Bug 1146736 - [Build 20190815] openQA test fails in yast2_samba - a problem occurred when connecting LDAP server
[Build 20190815] openQA test fails in yast2_samba - a problem occurred when c...
Status: RESOLVED INVALID
Classification: openSUSE
Product: openSUSE Tumbleweed
Classification: openSUSE
Component: Samba
Current
Other Other
: P3 - Medium : Normal (vote)
: ---
Assigned To: William Brown
The 'Opening Windows to a Wider World' guys
https://openqa.opensuse.org/tests/101...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2019-08-22 05:24 UTC by Joaquín Rivera
Modified: 2020-05-28 09:21 UTC (History)
9 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
Invalid credentials (60.13 KB, image/png)
2019-10-21 07:52 UTC, Joaquín Rivera
Details
journalctl (41.86 KB, application/x-xz)
2020-05-14 14:24 UTC, Jonathan Rivrain
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joaquín Rivera 2019-08-22 05:24:57 UTC
## Observation

openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-yast2_ncurses@64bit fails in
[yast2_samba](https://openqa.opensuse.org/tests/1012453/modules/yast2_samba/steps/72)
A pop-up appears about problem with connection to LDAP.

## Test suite description
Maintainer: zluo, riafarov Test for yast2 UI, ncurses only. Running on created gnome images which provides both text console for ncurses UI tests as well as the gnome environment for the GUI tests.
riafarov set TIMEOUT_SCALE to improve stability of the test.


## Reproducible

Fails since (at least) Build [20190202](https://openqa.opensuse.org/tests/846144)


## Expected result

Last good: [20190110](https://openqa.opensuse.org/tests/829581) (or more recent)


## Further details

Always latest result in this scenario: [latest](https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=yast2_ncurses&version=Tumbleweed)
Comment 1 Noel Power 2019-08-28 13:18:27 UTC
https://openqa.opensuse.org/tests/1012453#step/yast2_samba/20
would seem to indicate that the 389 Directory server didn't start up

...

[ 1488.342597] ns-slapd[14333]: [21/Aug/2019:10:25:52.908460519 -0400] - INFO - ldbm_instance_config_cachememsize_set - force a minimal value 512000
[ 1488.348667] ns-slapd[14333]: [21/Aug/2019:10:25:52.914522316 -0400] - NOTICE - ldbm_back_start - found 1514456k physical memory
[ 1488.350737] ns-slapd[14333]: [21/Aug/2019:10:25:52.916615511 -0400] - NOTICE - ldbm_back_start - found 561372k available
[ 1488.352619] ns-slapd[14333]: [21/Aug/2019:10:25:52.918455320 -0400] - NOTICE - ldbm_back_start - cache autosizing: db cache: 37861k
[ 1488.354355] ns-slapd[14333]: [21/Aug/2019:10:25:52.920190741 -0400] - NOTICE - ldbm_back_start - cache autosizing: userroot entry cache (1 total): 131072k
[ 1488.356751] ns-slapd[14333]: [21/Aug/2019:10:25:52.922570613 -0400] - NOTICE - ldbm_back_start - cache autosizing: userroot dn cache (1 total): 65536k
[ 1488.359266] ns-slapd[14333]: [21/Aug/2019:10:25:52.924609979 -0400] - NOTICE - ldbm_back_start - total cache size: 232342650 B;
[ 1488.362876] ns-slapd[14333]: [21/Aug/2019:10:25:52.926739280 -0400] - ERR - attrcrypt_fetch_private_key - Can't find certificate Server-Cert: -5950 - File not found.
[ 1488.364906] ns-slapd[14333]: [21/Aug/2019:10:25:52.928327899 -0400] - ERR - attrcrypt_fetch_private_key - Can't get private key from cert Server-Cert: -5950 - File not found.
[ 1488.367128] ns-slapd[14333]: [21/Aug/2019:10:25:52.929151480 -0400] - ERR - dblayer_instance_start - Unable to initialize attrcrypt system for userroot
[ 1488.370758] ns-slapd[14333]: [21/Aug/2019:10:25:52.930719825 -0400] - ERR - ldbm_back_start - Failed to start databases, err=-1 Unknown error: -1
[ 1488.372672] ns-slapd[14333]: [21/Aug/2019:10:25:52.932769607 -0400] - ERR - ldbm_back_start - Failed to allocate 31016058 byte dbcache.  Please reduce the value of nsslapd-cache-autosize and restart the server.
[ 1488.375374] ns-slapd[14333]: [21/Aug/2019:10:25:52.933580044 -0400] - ERR - plugin_dependency_startall - Failed to start database plugin ldbm database
[ 1488.377377] ns-slapd[14333]: [21/Aug/2019:10:25:52.936581062 -0400] - WARN - ldbm_instance_add_instance_entry_callback - ldbm instance userroot already exists
[ 1488.379667] ns-slapd[14333]: [21/Aug/2019:10:25:52.938334634 -0400] - ERR - ldbm_config_read_instance_entries - Failed to add instance entry cn=userroot,cn=ldbm database,cn=plugins,cn=config
[ 1488.381953] ns-slapd[14333]: [21/Aug/2019:10:25:52.939015682 -0400] - ERR - ldbm_config_load_dse_info - failed to read instance entries
[ 1488.383926] ns-slapd[14333]: [21/Aug/2019:10:25:52.940998216 -0400] - ERR - ldbm_back_start - Loading database configuration failed
[ 1488.385835] ns-slapd[14333]: [21/Aug/2019:10:25:52.941768786 -0400] - ERR - plugin_dependency_startall - Failed to start database plugin ldbm database
[ 1488.387807] ns-slapd[14333]: [21/Aug/2019:10:25:52.943535888 -0400] - ERR - plugin_dependency_startall - Failed to resolve plugin dependencies
[ 1488.389873] ns-slapd[14333]: [21/Aug/2019:10:25:52.945471478 -0400] - ERR - plugin_dependency_startall - betxnpreoperation plugin 7-bit check is not started
[ 1488.392085] ns-slapd[14333]: [21/Aug/2019:10:25:52.947725668 -0400] - ERR - plugin_dependency_startall - preoperation plugin Account Usability Plugin is not started
[ 1488.394219] ns-slapd[14333]: [21/Aug/2019:10:25:52.949404200 -0400] - ERR - plugin_dependency_startall - accesscontrol plugin ACL Plugin is not started
[ 1488.396124] ns-slapd[14333]: [21/Aug/2019:10:25:52.951200141 -0400] - ERR - plugin_dependency_startall - preoperation plugin ACL preoperation is not started
[ 1488.398395] ns-slapd[14333]: [21/Aug/2019:10:25:52.952129459 -0400] - ERR - plugin_dependency_startall - betxnpreoperation plugin Auto Membership Plugin is not started
[ 1488.410976] ns-slapd[14333]: [21/Aug/2019:10:25:52.953648749 -0400] - ERR - plugin_dependency_startall - object plugin Class of Service is not started
[ 1488.413697] ns-slapd[14333]: [21/Aug/2019:10:25:52.955287982 -0400] - ERR - plugin_dependency_startall - preoperation plugin deref is not started
[ 1488.428963] ns-slapd[14333]: [21/Aug/2019:10:25:52.955832582 -0400] - ERR - plugin_dependency_startall - preoperation plugin HTTP Client is not started
[ 1488.430504] ns-slapd[14333]: [21/Aug/2019:10:25:52.957716979 -0400] - ERR - plugin_dependency_startall - database plugin ldbm database is not started
[ 1488.433643] ns-slapd[14333]: [21/Aug/2019:10:25:52.958366192 -0400] - ERR - plugin_dependency_startall - betxnpreoperation plugin Linked Attributes is not started
[ 1488.437155] ns-slapd[14333]: [21/Aug/2019:10:25:52.959889529 -0400] - ERR - plugin_dependency_startall - betxnpreoperation plugin Managed Entries is not started
[ 1488.439670] ns-slapd[14333]: [21/Aug/2019:10:25:52.960729081 -0400] - ERR - plugin_dependency_startall - object plugin Multimaster Replication Plugin is not started
[ 1488.441017] ns-slapd[14333]: [21/Aug/2019:10:25:52.962080489 -0400] - ERR - plugin_dependency_startall - object plugin Roles Plugin is not started
[ 1488.442235] ns-slapd[14333]: [21/Aug/2019:10:25:52.963793009 -0400] - ERR - plugin_dependency_startall - object plugin Views is not started
[ 1488.443601] ns-slapd[14333]: [21/Aug/2019:10:25:52.964324391 -0400] - ERR - plugin_dependency_startall - extendedop plugin whoami is not started
[ 1488.445116] systemd[1]: dirsrv@openqatest.service: Main process exited, code=exited, status=1/FAILURE
[ 1488.446261] systemd[1]: dirsrv@openqatest.service: Failed with result 'exit-code'.
[ 1488.447459] systemd[1]: Failed to start 389 Directory Server openqatest..
yast2-ldap-server-status-0

That presumably is why there is a problem connecting to LDAP
Comment 2 Noel Power 2019-08-30 10:32:41 UTC
@william

could you have a look at this, I don't think this is a problem with the yast module but probably 389 Directory server (or maybe the test itself) I don't know enough about 389 to tell
Comment 3 William Brown 2019-09-25 00:25:57 UTC
[ 1488.362876] ns-slapd[14333]: [21/Aug/2019:10:25:52.926739280 -0400] - ERR - attrcrypt_fetch_private_key - Can't find certificate Server-Cert: -5950 - File not found.
[ 1488.364906] ns-slapd[14333]: [21/Aug/2019:10:25:52.928327899 -0400] - ERR - attrcrypt_fetch_private_key - Can't get private key from cert Server-Cert: -5950 - File not found.

^ It looks like you aren't able to find the system certificates. 

https://openqa.opensuse.org/tests/846144/modules/yast2_samba/steps/1/src

I'm going to try to reproduce this, but I'm having issues setting up the virtual machine.

I think that there was recently a change to yast-auth-server which changed a number of the UI fields. Can you confirm if this is the source of the problem on openqa since the tests may be out of sync. This could be related also to https://bugzilla.opensuse.org/show_bug.cgi?id=1146735 where it looks like there could be a package sync or an openqa update issue maybe?
Comment 4 Joaquín Rivera 2019-10-08 07:58:21 UTC
From my side, I don't see changes in the UI in this part in particular, for example this run is from 11 months ago: https://openqa.opensuse.org/tests/798945#step/yast2_samba/73
Comment 7 Joaquín Rivera 2019-10-21 07:52:02 UTC
I started to try to reproduce it with following steps in a VM:
1 - Get gnome .qcow2 image created in openQA for latest tw:
https://openqa.opensuse.org/tests/1060828/asset/hdd/opensuse-Tumbleweed-x86_64-20191018-gnome@64bit.qcow2
2 - Boot image, open terminal, login as root, password: nots3cr3t
2 - Download certs:
wget https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/console/samba_server_cert.p12
wget https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/data/console/samba_ca_cert.pem
3 - Configure host and hostname:
    `hostname openqa.ldaptest.org`
    `echo "127.0.0.1 openqa.ldaptest.org openqa" > /etc/hosts`
    `echo "openqa.ldaptest.org" > /etc/hostname`
4 - Launch yast2 ldap-server:
    Y2DEBUG=1 ZYPP_MEDIA_CURL_DEBUG=1 yast2 ldap-server
5 - Create new directory instance with following data: https://openqa.opensuse.org/tests/1060828#step/yast2_samba/16 (dir manager password: openqatest)
6 - There is an error, click OK and then Cancel.
7 - I could not run certutil command https://openqa.opensuse.org/tests/798945#step/yast2_samba/26 succesfully in the VM, I got "certutil: could not find certificate named "openqa.ldaptest.org. - Suse": SEC_ERROR_INVALID_ARGS: security library: invalid arguments. I cannot see in openQA that error, perhaps is silent when redirected to that tty. Anyway what I see also quite different in is the output of the status of the service with those error which cannot see in previous old run (https://openqa.opensuse.org/tests/798945#step/yast2_samba/26). Even that the service is up and running.
8 - Launch yast2 samba-server:
    Y2DEBUG=1 ZYPP_MEDIA_CURL_DEBUG=1 yast2 samba-server
9 - Workgroup: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/29
    Start-Up: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/39
    Shares: New share: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/44
    Identity: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/51
    Trusted domains: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/63 (password: testing) check error and cancel.
    LDAP Settings: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/66 and https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/68 (with admin password: openqatest)
    Expert LDAP Settings: https://openqa.opensuse.org/tests/1060828/#step/yast2_samba/71

Then finally I tried to Test the connection and tick "Show Details" and I got "invalid credentials". Attached screenshot.
Comment 8 Joaquín Rivera 2019-10-21 07:52:27 UTC
Created attachment 822014 [details]
Invalid credentials
Comment 20 Oliver Kurz 2020-04-03 12:29:57 UTC
Hi William Brown, you asked for support regarding openQA . In https://bugzilla.suse.com/show_bug.cgi?id=1146736#c7 I think jeriveramoya@suse.com already nicely explain what he did based on the openQA test definition. From the last comment showing a more recent test one can find https://openqa.suse.de/tests/4031826#step/yast2_samba/78 showing where the test failed. As this test is based on the YaST ncurses module the conducted steps are visible from screenshots. But also the logfiles can help as reachable over the web UI tab "Logs & Assets". E.g. https://openqa.suse.de/tests/4031826/file/autoinst-log.txt with all the test assertions and commands sent to the SUT. A bit verbose though :) How else can we help?
Comment 21 William Brown 2020-04-06 00:40:23 UTC
(In reply to Oliver Kurz from comment #20)
> Hi William Brown, you asked for support regarding openQA . In
> https://bugzilla.suse.com/show_bug.cgi?id=1146736#c7 I think
> jeriveramoya@suse.com already nicely explain what he did based on the openQA
> test definition. From the last comment showing a more recent test one can
> find https://openqa.suse.de/tests/4031826#step/yast2_samba/78 showing where
> the test failed. As this test is based on the YaST ncurses module the
> conducted steps are visible from screenshots. But also the logfiles can help
> as reachable over the web UI tab "Logs & Assets". E.g.
> https://openqa.suse.de/tests/4031826/file/autoinst-log.txt with all the test
> assertions and commands sent to the SUT. A bit verbose though :) How else
> can we help?


I think I already submitted a fix to openQA for this?

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9250

https://progress.opensuse.org/issues/61780#change-283738

It's just not had any more action I think ....
Comment 22 Oliver Kurz 2020-04-07 06:58:18 UTC
@mgriessmeier your PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9250 is pending for this bug.
Comment 25 Jonathan Rivrain 2020-05-14 14:24:23 UTC
Created attachment 837791 [details]
journalctl

(In reply to William Brown from comment #21)
> 
> I think I already submitted a fix to openQA for this?
> 
> https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/9250
> 
> https://progress.opensuse.org/issues/61780#change-283738
> 
> It's just not had any more action I think ....

The PR does not fix the issue. it just renames the variables, the used values are the same. It seems like he wanted to re-design the module then abandoned it. Not sure what he wanted to do.
We still have this "invalid credentials" error, see latest run: https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=yast2_ncurses&version=Tumbleweed. 

I could not find similar errors as in comment 1. So I re-upload the journalctl log from current build.
Comment 26 William Brown 2020-05-15 02:47:20 UTC
It appears you are putting in an invalid cert into the DS instance:

May 14 09:41:58 openqa.ldaptest.org ns-slapd[2735]: [14/May/2020:09:41:58.805919303 -0400] - INFO - Security Initialization - SSL info:         TLS_RSA_WITH_AES_256_CBC_SHA256: enabled
May 14 09:41:58 openqa.ldaptest.org ns-slapd[2735]: [14/May/2020:09:41:58.817107251 -0400] - WARN - Security Initialization - SSL alert: CERT_VerifyCertificateNow: verify certificate failed for cert Server-Cert of family cn=RSA,cn=encryption,cn=config (Netscape Portable Runtime error -8181 - Peer's Certificate has expired.)
Comment 27 William Brown 2020-05-15 02:48:17 UTC
More detailed response sorry: the p12 bundel you are putting in has expired, so that's why it's failing to start, which then causes the subsequent failures. 

So yeah, i think you should probably either remove the p12 from the test, or generate it as required?
Comment 28 Jonathan Rivrain 2020-05-22 16:28:31 UTC
I renewed the key, but that was not the issue - though at least it allowed me to go ahead. It looks like now we have to use "cn=Directory Manager" instead of "cn=root" : https://openqa.opensuse.org/tests/1275011#step/yast2_samba/62. We see this cn here : https://openqa.opensuse.org/tests/1275011#step/yast2_samba/15.
Comment 29 Jonathan Rivrain 2020-05-27 14:31:58 UTC
So I guess you can reseolve this report, as after all it was just due to some upstream change. Thanks for helping !
Comment 30 William Brown 2020-05-28 00:02:17 UTC
(In reply to Jonathan Rivrain from comment #29)
> So I guess you can reseolve this report, as after all it was just due to
> some upstream change. Thanks for helping !

Yep, it sounds like there has been a lot going on that's needed to resolve this, but let's stay in contact and I'll do my part to get it fixed upstream to make sure this works.
Comment 31 Jonathan Rivrain 2020-05-28 09:21:17 UTC
Test module should have been adapted, as the problem was due to a change upstream, see https://progress.opensuse.org/issues/61780#note-16.