Bug 1221678

What do you mean "can't provide right information"?

This is as nondescript as "doesn't work"; it doesn't say anything. This is not a useful bug subject. Please find an appropriate one that does not leave us guessing what this is all about.

This refers to (among others) this openQA test:

https://openqa.suse.de/tests/13816505

This test does not have any description. I just see an endless sequence of screenshots with command lines (!). The part that says "softfailed"  starts here:

https://openqa.suse.de/tests/13816505#step/yast_dns_server/1

and ends here:

https://openqa.suse.de/tests/13816505#step/yast_dns_server/172

If those numbers can be believed, that's 172 steps; all with some command lines.


This is the corresponding openQA test code:

https://openqa.suse.de/tests/13816505/modules/yast_dns_server/steps/1/src


What does that do? I have no idea. And I am not going to reverse-engineer it.

Please read

https://en.opensuse.org/openSUSE:How_to_Write_a_Good_Bugreport

BTW comment #0 here is not any more useful. There is no description what you did, just command lines. You don't describe what you do, why you do that, what you are trying to test, what goes wrong; what you expect, and what happens instead.

Those are the basics of describing any problem anywhere (not only in SUSE Bugzilla; globally, everywhere); as described in https://en.opensuse.org/openSUSE:How_to_Write_a_Good_Bugreport.

The error is that we can't find related information in the outpout.

Test steps:

1) Create DNS forwarder and DNS server, then create zone and host
2) We checked the dnsrecord
3) We add a mailserver by comman yast2 dns-server nameserver add zone=example.org  ns=ns2.example.com and we want to check if ns info was print in `yast2 dns-server nameserver show zone=example.org' but we can't find it.
   https://openqa.suse.de/tests/13817163#step/yast_dns_server/107


4) We add a mailserver by `yast2 dns-server mailserver add zone=example.org  priority=97 mx=mx001;` and want to check mx info in the yast2 dns-server mailserver show zone=example.org', but we can't find the mx in the output.
   https://openqa.suse.de/tests/13817163#step/yast_dns_server/119

5) we cheange the atboot by 'yast2 dns-server startup atboot', but it still shows manual
   https://openqa.suse.de/tests/13817163#step/yast_dns_server/134


So you can see for the bug title, we have three errors in this the bug that we need to describle and it's so hard to give a clear description in title. So it is better to file three bugs for it?

OK, how to put a problem in words, beginner course:

"We add a zone to the nameserver with the 'yast2 dns' command line, then we check if that new zone is there with another 'yast2 dns' command line."

THEN (after that!) it would be useful to show the exact commands used.


Is that really that difficult?

The appropriate subject for this would be "yast2 dns command line doesn't add nameserver zone".

Did you try the same in interactive mode, i.e. with the GUI of that YaST module?

Obviously, it worked before - there is a "last good" case. What changed in the meantime? I am pretty sure that the yast2-dns-server code did not change; that module is old and already declared obsolete in the master branch, i.e. in Factory / Tumbleweed. So it's very unlikely that it was the yast2-dns-server package that causes this problem.

It might be one of the underlying packages that changed, or some command line tools that are used by yast2-dns-server.

The last source code change of this package in the SLE-15-SP5 branch was 2 years ago:

  https://github.com/yast/yast-dns-server/commits/SLE-15-SP5/src

I don't see how this package can be responsible for a problem that appeared for the first time only a few days ago, after a "last good" 3 days ago.

Trying to decypher that Perl code from your openQA test:

https://openqa.suse.de/tests/13817163/modules/yast_dns_server/steps/1/src

I swore I'd never do this again: This is the job of QA, not ours. Yet with zero documentation and zero comments and nobody feeling able to put into words what that thing does, you are forcing us to reverse-engineer that stuff.


So this tries some of those "yast dns-server" command lines which basically take the arguments you give it, then writes them to some configuration file. Then that Perl code uses some "yast dns-server show" command to give you that same thing back that you just put in.

I don't see this ever even attempting to check IN THAT CONFIGURATION FILE if the correct changes were done; I don't see any file being read and compared with the expected output.

This completely relies on the "yast dns-server add ..." commands and then the corresponding "yast dns-server show ..." commands; sometimes also "yast dns-server remove ...".

This alone strikes me as a pretty absurd way of testing; if there is a discrepancy, was it a problem of the "add" or of the "show" command? Or did some parameters simply not survive a plausibility check somewhere and were rejected?


And then I see 

    systemctl("start named.service");

in various places; but not everywhere; why and when it does that is unclear to me. Who knows what that named.service is doing here.

What are you testing here? 'named' or the YaST dns-server module?

Created attachment 873665 [details]
The y2logs split into individual yast calls

> y2-00-dns-server.log:  08:31:15  yast dns-server forwarders add ip=10.0.2.3
> y2-01-dns-server.log:  08:31:19  yast dns-server forwarders show
> y2-02-dns-server.log:  08:31:22  yast dns-server forwarders remove ip=10.0.2.3
> y2-03-dns-server.log:  08:31:25  yast dns-server forwarders show
> y2-04-dns-server.log:  08:31:28  yast dns-server zones add zonetype=master name=example.org
> y2-05-dns-server.log:  08:31:31  yast dns-server zones show
> y2-06-dns-server.log:  08:31:34  yast dns-server zones add zonetype=master name=100.168.192.in-addr.arpa
> y2-07-dns-server.log:  08:31:38  yast dns-server zones show
> y2-08-dns-server.log:  08:31:41  yast dns-server host add zone=example.org ip=192.168.100.4 hostname=host02.example.org.
> y2-09-dns-server.log:  08:31:45  yast dns-server host show zone=example.org
> y2-10-dns-server.log:  08:31:48  yast dns-server host remove zone=example.org hostname=host02.example.org. ip=192.168.100.4
> y2-11-dns-server.log:  08:31:52  yast dns-server host show zone=example.org
> y2-12-dns-server.log:  08:31:55  yast dns-server logging set destination=file file=/var/log/named.log maxsize=100M maxversions=3
> y2-13-dns-server.log:  08:31:59  yast dns-server logging show
> y2-14-dns-server.log:  08:32:02  yast dns-server soa set zone=example.org serial=2019090502 expiry=2W retry=2H
> y2-15-dns-server.log:  08:32:06  yast dns-server soa show zone=example.org
> y2-16-dns-server.log:  08:32:10  yast dns-server dnsrecord add zone=example.org type=NS value=ns1 query=subdomain.example.org.
> y2-17-dns-server.log:  08:32:14  yast dns-server dnsrecord show zone=example.org
> y2-18-dns-server.log:  08:32:17  yast dns-server dnsrecord remove zone=example.org query=subdomain.example.org. value=ns1 type=NS
> y2-19-dns-server.log:  08:32:21  yast dns-server dnsrecord show zone=example.org
> y2-20-dns-server.log:  08:32:25  yast dns-server dnsrecord add zone=example.org type=A query=host1 value=192.168.100.3
> y2-21-dns-server.log:  08:32:29  yast dns-server dnsrecord show zone=example.org
> y2-22-dns-server.log:  08:32:33  yast dns-server dnsrecord remove zone=example.org query=host1 value=192.168.100.3 type=A
> y2-23-dns-server.log:  08:32:37  yast dns-server dnsrecord show zone=example.org
> y2-24-dns-server.log:  08:32:41  yast dns-server dnsrecord add zone=100.168.192.in-addr.arpa value=host1 query=123 type=PTR
> y2-25-dns-server.log:  08:32:45  yast dns-server dnsrecord show zone=100.168.192.in-addr.arpa
> y2-26-dns-server.log:  08:32:49  yast dns-server dnsrecord remove zone=100.168.192.in-addr.arpa type=PTR query=123 value=host1
> y2-27-dns-server.log:  08:32:54  yast dns-server dnsrecord show zone=100.168.192.in-addr.arpa
> y2-28-dns-server.log:  08:32:58  yast dns-server dnsrecord add zone=example.org value=server6.anywhere.net. query=ns6 type=CNAME
> y2-29-dns-server.log:  08:33:03  yast dns-server dnsrecord show zone=example.org
> y2-30-dns-server.log:  08:33:07  yast dns-server dnsrecord remove zone=example.org value=server6.anywhere.net. query=ns6 type=CNAME
> y2-31-dns-server.log:  08:33:11  yast dns-server dnsrecord show zone=example.org
> y2-32-dns-server.log:  08:33:16  yast dns-server mailserver add zone=example.org priority=97 mx=mx001
> y2-33-dns-server.log:  08:33:20  yast dns-server mailserver show zone=example.org
> y2-34-dns-server.log:  08:33:24  yast dns-server nameserver add zone=example.org ns=ns2.example.com.
> y2-35-dns-server.log:  08:33:29  yast dns-server nameserver show zone=example.org
> y2-36-dns-server.log:  08:33:33  yast dns-server startup atboot
> y2-37-dns-server.log:  08:33:38  yast dns-server startup show
> y2-38-dns-server.log:  08:33:43  yast dns-server startup manual
> y2-39-dns-server.log:  08:33:48  yast dns-server zones remove name=example.org zonetype=master
> y2-40-dns-server.log:  08:33:53  yast dns-server zones show
> y2-41-dns-server.log:  08:33:58  yast dns-server zones remove name=100.168.192.in-addr.arpa zonetype=master
> y2-42-dns-server.log:  08:34:03  yast dns-server zones show

I am not at all sure if the sequence of those commands can works like that. If something is added and then immediately removed, maybe; the normal sequence appears to be

  add
  show
  remove

But not always. Does it matter if a previous command left something behind? I don't know.

I don't know what that whole test is trying to do (and it's not documented anywhere). What is the point? Worse, what is the point of never checking if the configuration files really contain the expected values, or if the changes have the desired impact on the system that you are testing?

Until proven otherwise, the test is faulty, not the YaST module.

This test is based on the following document and the author and the link now are out of touch.

# All of cases is based on the reference:
# https://documentation.suse.com/sles/15-SP1/single-html/SLES-admin/#id-1.3.3.6.13.6.11

We are just care about these comands:
> y2-32-dns-server.log:  08:33:16  yast dns-server mailserver add zone=example.org priority=97 mx=mx001
> y2-33-dns-server.log:  08:33:20  yast dns-server mailserver show zone=example.org
> y2-34-dns-server.log:  08:33:24  yast dns-server nameserver add zone=example.org ns=ns2.example.com.
> y2-35-dns-server.log:  08:33:29  yast dns-server nameserver show zone=example.org
> y2-36-dns-server.log:  08:33:33  yast dns-server startup atboot

In the ouput of cmd `yast dns-server mailserver show zone=example.org`should show the mx info, but can't get them.

I've tried these cmd lines on the dev mode.

1)  # yast2 dns-server nameserver add zone=example.org  ns=ns2.example.com
    # yast2 dns-server nameserver show zone=example.org 2>&1
```
Name Servers:
-------------

    Zone         Name Server
    ------------------------
    example.org  susetest. 
``` 
** No ns info ns2.example.comin the output **

2) #yast2 dns-server mailserver add zone=example.org  priority=97 mx=mx001;
   #yast2 dns-server mailserver show zone=example.org
```   
Mail Servers:
-------------

    Zone  Mail Server  Priority
    --------------------------- 
```
**No mx info mx001 in the outout.**

And I've asked the help-yast and they said it should show mx/ns information in the output.

>Hi Yast experts, I have one question for this CLI, do you know what ns means in the CLI
> # yast2 dns-server nameserver add zone=example.org  ns=ns2.example.com
> and for this command
> # yast2 dns-server nameserver show zone=example.org 
> Can it show the ns information in the output?
> Thanks,

> Yes, ns stands for nameserver. For the second question, the output is zone and nameserver. So ns value is the second collumn.
> So from your first command it will be zone example.org and nameserver (aka ns)ns2.example.com

As https://bugzilla.suse.com/show_bug.cgi?id=1221678#c13, the output of corresponding yast2 cmds are not as expected, so re-open this bug, please check it again. Thanks.

No, you are not going to play this game with me - no Bugzilla ping-pong.


You are claiming that code that is UNCHANGED since two years (!) worked until some days ago, and now it doesn't work anymore? And that this must be the fault of this code? This is simply not possible.

If your test stopped working, and the code that you are testing didn't change, then it must be the test or the test environment or maybe some other system component that is now causing problems.

I explained that in great detail. Restoring old resolution.

(In reply to Huajian Luo from comment #13)
> This test is based on the following document and the author and the link now
> are out of touch.
> 
> # All of cases is based on the reference:
> #
> https://documentation.suse.com/sles/15-SP1/single-html/SLES-admin/#id-1.3.3.
> 6.13.6.11
> 
> We are just care about these comands:
> > y2-32-dns-server.log:  08:33:16  yast dns-server mailserver add zone=example.org priority=97 mx=mx001
> > y2-33-dns-server.log:  08:33:20  yast dns-server mailserver show zone=example.org
> > y2-34-dns-server.log:  08:33:24  yast dns-server nameserver add zone=example.org ns=ns2.example.com.
> > y2-35-dns-server.log:  08:33:29  yast dns-server nameserver show zone=example.org
> > y2-36-dns-server.log:  08:33:33  yast dns-server startup atboot
> 
> In the ouput of cmd `yast dns-server mailserver show zone=example.org`should
> show the mx info, but can't get them.
> 
> I've tried these cmd lines on the dev mode.
> 
> 1)  # yast2 dns-server nameserver add zone=example.org  ns=ns2.example.com
>     # yast2 dns-server nameserver show zone=example.org 2>&1
> ```
> Name Servers:
> -------------
> 
>     Zone         Name Server
>     ------------------------
>     example.org  susetest. 
> ``` 
> ** No ns info ns2.example.comin the output **
> 
> 2) #yast2 dns-server mailserver add zone=example.org  priority=97 mx=mx001;
>    #yast2 dns-server mailserver show zone=example.org
> ```   
> Mail Servers:
> -------------
> 
>     Zone  Mail Server  Priority
>     --------------------------- 
> ```
> **No mx info mx001 in the outout.**
> 
> And I've asked the help-yast and they said it should show mx/ns information
> in the output.
> 
> >Hi Yast experts, I have one question for this CLI, do you know what ns means in the CLI
> > # yast2 dns-server nameserver add zone=example.org  ns=ns2.example.com
> > and for this command
> > # yast2 dns-server nameserver show zone=example.org 
> > Can it show the ns information in the output?
> > Thanks,
> 
> > Yes, ns stands for nameserver. For the second question, the output is zone and nameserver. So ns value is the second collumn.
> > So from your first command it will be zone example.org and nameserver (aka ns)ns2.example.com


Repeating the same thing does not change anything. I explained it in great detail in my previous comments.

And I already wasted the better part of a whole working day taking this thing apart. Remember that the whole yast-dns-server module is already obsoleted in Factory; it is only still in SLE-15-SPx because we cannot discontinue it in a released product.

To the best of my knowledge, no customer ever complained about this anyway; this is a complete waste of time.

(In reply to Huajian Luo from comment #0)
> ## Expected result
> 
> Last good: [20240317-1](https://openqa.suse.de/tests/13808728) 
>            (or more recent)


So, this is the "last good" run of this exact same test, you write.
That test run was 3 days ago.


Now please explain how this is different from the one that failed, and what packages changed in the meantime.

So this "last good" was simply not true: There is no "last good". It never worked. That "last good" section in the bug description was wrong, and grossly misleadingly so.

This is an EXACT duplicate of bug #1151130 from 4 years ago, where we had requested y2logs and never received any. This is also a duplicate of bug #1151135, also from 4 years ago, that was also abandoned in NEEDINFO for a long time.

During all that time, not a single customer ever complained about it, and the whole yast-dns-server is now obsolete and dropped in Factory, only on life support for the remaining life time of SLE-15-SPx.

And now you opened yet another duplicate of this; a duplicate that does who knows what with a lot of commands before which may modify the state of the system in undefined ways.

Since for over 4 years nobody cared, in particular no customer, I don't think anybody will take any actions now.

*** This bug has been marked as a duplicate of bug 1151135 ***

Also, this way of testing is counterproductive: It uses the same command to do modifications and then to query them again.

It never checks if the correct modifications were done to the system; there is NO check at all of any system configuration files. The test doesn't even try.

It also violates another basic principle of good testing: Start each sub-test with a well-defined state. Do not rely on a previous command of the software under test to restore the old status. Yet this does:

   yast dns-server add ...
   yast dns-server show ...
   yast dns-server remove ...

And sometimes it doesn't do the 'remove' part; in those "soft failure" cases, for example. So this relies on HOPING that the system is back in a well-defined state, it doesn't make sure.


Also, this just fires off an endless sequence of similar tests in one single test step, producing some dozen of screenshots of command lines; which is completely useless to try verifying what each sub-test did, what the output was, and if that was the right thing.

This would be much better suited to a testing framework like DejaGNU or RSpec which are designed to check a command's actual output against the expected one.

In conclusion, this was a giant waste of time.