Bug 1217619

Summary:	grub2 reproducible builds
Product:	[openSUSE] openSUSE Tumbleweed	Reporter:	Bernhard Wiedemann <bwiedemann>
Component:	Bootloader	Assignee:	Bootloader Maintainers <bootloader-maintainers>
Status:	IN_PROGRESS ---	QA Contact:	E-mail List <qa-bugs>
Severity:	Normal
Priority:	P5 - None	CC:	bwiedemann, mchang
Version:	Current
Target Milestone:	---
Hardware:	Other
OS:	All
Whiteboard:
Found By:	Development	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---

Description Bernhard Wiedemann 2023-11-29 04:39:40 UTC

While working on reproducible builds for openSUSE, I found that
our grub2 package's
/usr/share/grub2/x86_64-xen/grub.xen varies across builds.

I identified 2 issues:
The tar contains changing (octal) mtime values.
The tar stores files in random filesystem readdir order.


The first issue produces such diffs:
--- old /usr/share/grub2/x86_64-xen/grub.xen (objdump)
+++ new /usr/share/grub2/x86_64-xen/grub.xen (objdump)
@@ -4896,7 +4896,7 @@
  01319f 00000000 30303030 36303000 30303031  ....0000600.0001
  0131af 37353000 30303031 37353000 30303030  750.0001750.0000
  0131bf 30303033 34353000 31343533 31323035  0003450.14531205
- 0131cf 35303300 30303135 30313320 30000000  503.0015013 0...
+ 0131cf 36313200 30303135 30313420 30000000  612.0015014 0...
  0131df 00000000 00000000 00000000 00000000  ................
  0131ef 00000000 00000000 00000000 00000000  ................


and the second issue produced this diff:
  01311f 00000000 00000000 01000000 00000000  ................
  01312f 00000000 00000000 01000000 08b42600  ..............&.
  01313f 626f6f74 2f677275 622f7838 365f3634  boot/grub/x86_64
- 01314f 2d78656e 2f646973 6b2e6d6f 64000000  -xen/disk.mod...
+ 01314f 2d78656e 2f6c7378 656e2e6d 6f640000  -xen/lsxen.mod..
  01315f 00000000 00000000 00000000 00000000  ................


Here is a PoC patch for the first issue:
--- a/util/grub-mkstandalone.c
+++ b/util/grub-mkstandalone.c
@@ -264,6 +264,7 @@ add_tar_file (const char *from,
   memcpy (hd.gid, "0001750", 7);
 
   set_tar_value (hd.size, size, 12);
+  mtime = 42;
   set_tar_value (hd.mtime, mtime, 12);
   hd.typeflag = '0';
   memcpy (hd.magic, MAGIC, sizeof (hd.magic));


The second issue probably comes from grub_util_fd_readdir that would need to collect+sort entries before further processing.

Comment 1 Michael Chang 2023-12-01 06:03:22 UTC

Hi Bernhard,

Thanks for opening the issue and providing the information.

You are absolutely right, the varying timestamp and the nondeterministic readdir() result is the culprit. For that these two patches have been worked out to address them separately.  

https://build.opensuse.org/package/rdiff/home:michael-chang:branches:Base:System/grub2?opackage=grub2&oproject=Base%3ASystem&rev=3 

Would you mind to check if the preliminary version works for you ? I'd like to have your signed-off-by or test-by before doing the submission to factory and upstream. It is all up to you and if you are not available, I can just go ahead  and do the submit.

Comment 2 Bernhard Wiedemann 2023-12-04 08:56:13 UTC

I tested it successfully. It produces bit-identical results now.

The patches also look good to me.
The only thing I wondered why the free (tfp);
was not already there before.
That should have leaked some (insignificant) memory.

Comment 3 Michael Chang 2023-12-05 07:54:20 UTC

(In reply to Bernhard Wiedemann from comment #2)
> I tested it successfully. It produces bit-identical results now.
> 
> The patches also look good to me.

Thanks for your feedback. With the result I think it is ready to submit to factory and also upstream (with your signed-off-by).

> The only thing I wondered why the free (tfp);
> was not already there before.
> That should have leaked some (insignificant) memory.

I came across the memory leak issue while working on the patch, given it is trivial one to fix along I'm not bothered to create a separate patch for it.