Bug 1225754

Summary: samba: new non-determinism in 4.20.1
Product: [openSUSE] openSUSE Tumbleweed Reporter: Bernhard Wiedemann <bwiedemann>
Component: SambaAssignee: The 'Opening Windows to a Wider World' guys <samba-maintainers>
Status: RESOLVED FIXED QA Contact: The 'Opening Windows to a Wider World' guys <samba-maintainers>
Severity: Normal    
Priority: P5 - None CC: nopower, scabrero
Version: Current   
Target Milestone: ---   
Hardware: Other   
OS: All   
Whiteboard:
Found By: Development Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: upstream patch

Description Bernhard Wiedemann 2024-05-31 14:03:43 UTC
My tools for testing reproducible builds show me that there
is new non-determinism from ASLR and parallelism
that comes in via

--- /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/source3/librpc/gen_ndr/py_smbXsrv.c
+++ /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/source3/librpc/gen_ndr/py_smbXsrv.c
@@ -1695,7 +1695,7 @@
                object->raw_ev_ctx = NULL;
        } else {
                object->raw_ev_ctx = NULL;
-               PyErr_SetString(PyExc_TypeError, "Can not convert C Type struct HASH(0x555555e02eb0) from Python");
+               PyErr_SetString(PyExc_TypeError, "Can not convert C Type struct HASH(0x555555e02e20) from Python");

This was not an issue with previous version 4.19.5+git.342.57620c4f7e-304
Comment 1 Bernhard Wiedemann 2024-05-31 14:33:41 UTC
Here is a small reproducer:

cd ~/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2 &&
 for i in $(seq 3) ; do perl "pidl/pidl" "--quiet" "--includedir=/home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/librpc/idl" "--header" "--ndr-parser" "--client" "--python" "--outputdir" "bin/default/source3/librpc/gen_ndr" "--" "source3/librpc/idl/smbXsrv.idl" 2>/dev/null ; md5sum bin/default/source3/librpc/gen_ndr/py_smbXsrv.c ; done
59d8aaa03d6e823a3895c7bed7a1f299  bin/default/source3/librpc/gen_ndr/py_smbXsrv.c
b274520e585748db6cb33b025cfad66a  bin/default/source3/librpc/gen_ndr/py_smbXsrv.c
0864b694bb939ce7aecf1079134943e7  bin/default/source3/librpc/gen_ndr/py_smbXsrv.c

There is also plenty output on stderr such as
source3/librpc/idl/smbXsrv.idl:60: error: Unable to determine origin of type `struct db_record'
that might be related.
Comment 2 Noel Power 2024-05-31 15:08:05 UTC
(In reply to Bernhard Wiedemann from comment #0)
> My tools for testing reproducible builds show me that there
> is new non-determinism from ASLR and parallelism
> that comes in via
> 
> ---
> /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/
> source3/librpc/gen_ndr/py_smbXsrv.c
> +++
> /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/
> source3/librpc/gen_ndr/py_smbXsrv.c
> @@ -1695,7 +1695,7 @@
>                 object->raw_ev_ctx = NULL;
>         } else {
>                 object->raw_ev_ctx = NULL;
> -               PyErr_SetString(PyExc_TypeError, "Can not convert C Type
> struct HASH(0x555555e02eb0) from Python");
> +               PyErr_SetString(PyExc_TypeError, "Can not convert C Type
> struct HASH(0x555555e02e20) from Python");
> 
> This was not an issue with previous version 4.19.5+git.342.57620c4f7e-304

looks like an error in the perl where it looks like it is trying to generate an error message (but ends up printing out the address of the array containing the required string) or something like that
I'll try and find where this is going wrong
Comment 3 Samuel Cabrero 2024-05-31 16:21:22 UTC
(In reply to Noel Power from comment #2)
> (In reply to Bernhard Wiedemann from comment #0)
> > My tools for testing reproducible builds show me that there
> > is new non-determinism from ASLR and parallelism
> > that comes in via
> > 
> > ---
> > /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/
> > source3/librpc/gen_ndr/py_smbXsrv.c
> > +++
> > /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/
> > source3/librpc/gen_ndr/py_smbXsrv.c
> > @@ -1695,7 +1695,7 @@
> >                 object->raw_ev_ctx = NULL;
> >         } else {
> >                 object->raw_ev_ctx = NULL;
> > -               PyErr_SetString(PyExc_TypeError, "Can not convert C Type
> > struct HASH(0x555555e02eb0) from Python");
> > +               PyErr_SetString(PyExc_TypeError, "Can not convert C Type
> > struct HASH(0x555555e02e20) from Python");
> > 
> > This was not an issue with previous version 4.19.5+git.342.57620c4f7e-304
> 
> looks like an error in the perl where it looks like it is trying to generate
> an error message (but ends up printing out the address of the array
> containing the required string) or something like that
> I'll try and find where this is going wrong

Hey Noel, I also had a look. This happens for fields tagged as [ignore] in the IDL. Somehow it changes the data structure generated so NAME contains a HASH instead of a SCALAR, e.g.:

>  $VAR1 = {
>           'DATA' => {
>                       'TYPE' => 'STRUCT'
>                     },
>           'TYPE' => 'TYPEDEF',
>           'NAME' => {
>                       'ELEMENTS' => undef,
>                       'SURROUNDING_ELEMENT' => undef,
>                       'TYPE' => 'STRUCT',
>                       'NAME' => 'smbXsrv_pending_break',
>                       'ALIGN' => undef,
>                       'ORIGINAL' => {
>                                       'LINE' => 151,
>                                       'TYPE' => 'STRUCT',
>                                       'NAME' => 'smbXsrv_pending_break',
>                                       'FILE' => 'source3/librpc/idl/smbXsrv.idl'
>                                     },
>                       'PROPERTIES' => undef
>                     }
>         };

I Have a patch here: https://gitlab.suse.de/scabrero/suse-samba/-/commit/a4dc4d50ba6306272ddc008a62dc1fdb6a687cfa

If it passes CI I will submit to upstream.
Comment 4 Noel Power 2024-06-01 16:26:34 UTC
(In reply to Samuel Cabrero from comment #3)
> (In reply to Noel Power from comment #2)
> > (In reply to Bernhard Wiedemann from comment #0)
> > > My tools for testing reproducible builds show me that there
> > > is new non-determinism from ASLR and parallelism
> > > that comes in via
> > > 
> > > ---
> > > /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/
> > > source3/librpc/gen_ndr/py_smbXsrv.c
> > > +++
> > > /home/abuild/rpmbuild/BUILD/samba-4.20.1+git.335.0a46cdafe2/bin/default/
> > > source3/librpc/gen_ndr/py_smbXsrv.c
> > > @@ -1695,7 +1695,7 @@
> > >                 object->raw_ev_ctx = NULL;
> > >         } else {
> > >                 object->raw_ev_ctx = NULL;
> > > -               PyErr_SetString(PyExc_TypeError, "Can not convert C Type
> > > struct HASH(0x555555e02eb0) from Python");
> > > +               PyErr_SetString(PyExc_TypeError, "Can not convert C Type
> > > struct HASH(0x555555e02e20) from Python");
> > > 
> > > This was not an issue with previous version 4.19.5+git.342.57620c4f7e-304
> > 
> > looks like an error in the perl where it looks like it is trying to generate
> > an error message (but ends up printing out the address of the array
> > containing the required string) or something like that
> > I'll try and find where this is going wrong
> 
> Hey Noel, I also had a look. This happens for fields tagged as [ignore] in
> the IDL. Somehow it changes the data structure generated so NAME contains a
> HASH instead of a SCALAR, e.g.:
> 
> >  $VAR1 = {
> >           'DATA' => {
> >                       'TYPE' => 'STRUCT'
> >                     },
> >           'TYPE' => 'TYPEDEF',
> >           'NAME' => {
> >                       'ELEMENTS' => undef,
> >                       'SURROUNDING_ELEMENT' => undef,
> >                       'TYPE' => 'STRUCT',
> >                       'NAME' => 'smbXsrv_pending_break',
> >                       'ALIGN' => undef,
> >                       'ORIGINAL' => {
> >                                       'LINE' => 151,
> >                                       'TYPE' => 'STRUCT',
> >                                       'NAME' => 'smbXsrv_pending_break',
> >                                       'FILE' => 'source3/librpc/idl/smbXsrv.idl'
> >                                     },
> >                       'PROPERTIES' => undef
> >                     }
> >         };
> 
> I Have a patch here:
> https://gitlab.suse.de/scabrero/suse-samba/-/commit/
> a4dc4d50ba6306272ddc008a62dc1fdb6a687cfa
> 
> If it passes CI I will submit to upstream.

There is an upstream patch for this f3433f60b8ee83fc785a6e6838513de31bff5a6a (will attach)
Comment 5 Noel Power 2024-06-01 16:27:16 UTC
Created attachment 875255 [details]
upstream patch
Comment 6 James McDonough 2024-06-25 16:17:43 UTC
fixed, closing