Bug 1216158 - [Build 20231011-1] oscap xccdf eval command stuck in test oscap_xccdf_eval_remote
Summary: [Build 20231011-1] oscap xccdf eval command stuck in test oscap_xccdf_eval_re...
Status: RESOLVED FIXED
Alias: None
Product: PUBLIC SUSE Linux Enterprise Server 15 SP5
Classification: openSUSE
Component: Security Certifications (show other bugs)
Version: unspecified
Hardware: S/390 Other
: P5 - None : Normal
Target Milestone: ---
Assignee: Certification Bugs
QA Contact:
URL: https://openqa.suse.de/tests/12463240...
Whiteboard: SCAP
Keywords:
Depends on:
Blocks:
 
Reported: 2023-10-12 06:32 UTC by Joaquín Rivera
Modified: 2024-05-06 12:35 UTC (History)
5 users (show)

See Also:
Found By: openQA
Services Priority:
Business Priority:
Blocker: Yes
Marketing QA Status: ---
IT Deployment: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joaquín Rivera 2023-10-12 06:32:06 UTC
## Observation

openQA test in scenario sle-15-SP5-Server-DVD-Updates-s390x-stig@s390x-kvm fails in
[oscap_xccdf_eval_remote](https://openqa.suse.de/tests/12463240/modules/oscap_xccdf_eval_remote/steps/42)

The command got stuck for 50 minutes (which is the script timeout) in this scenario running in maintenance updates.

See logs attached to openQA job in Logs and Assets tab.
Script code: https://github.com/search?q=repo%3Aos-autoinst%2Fos-autoinst-distri-opensuse%20oscap_evaluate_remote&type=code


## Expected result
Last good: [20231010-1](https://openqa.suse.de/tests/12450777) (or more recent)
Comment 1 Marcus Meissner 2023-10-12 07:13:24 UTC
It has probably run out of memory again, looking at the follow screen shots it seems in an OOM situation.

--fetch-remote-resources will really use multiple gigabytes of memory
Comment 2 Ricardo Branco 2023-10-12 07:36:26 UTC
Fwiw, in public cloud tests we download the compressed file, uncompress it and then run it with --local-files like this:

https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/publiccloud/hardened.pm
Comment 3 Marcus Meissner 2023-10-12 08:04:38 UTC
Did you see if it reduced memory usage of openscap?
Comment 4 Ricardo Branco 2023-10-12 08:16:00 UTC
(In reply to Marcus Meissner from comment #3)
> Did you see if it reduced memory usage of openscap?

I didn't analyze it but at least it worked.

It seems to me that oscap was trying to download the whole file which is 350MB because it was taking too much time, or perhaps downloaded the compressed file and tried to uncompress in memory itself.
Comment 5 Joaquín Rivera 2023-10-12 09:27:43 UTC
It succeed with less ram, it was quite overdimensioned, and no idea how guest shared memory in that situation, I set 8GiB to be conservative and the timeout needs to be bumped because in rare ocassion we made in 16' but normally was in the boundary of the timeout of 50'. https://openqa.suse.de/tests/12464688#step/oscap_xccdf_eval_remote/42
So no bug :) thanks for the feedback. we will keep in mind this local option for future development.
Comment 6 Ricardo Branco 2023-10-17 11:00:18 UTC
I could make it run in cloud with instances with 8G of RAM adding 4G of swap.
Comment 7 Joaquín Rivera 2023-10-17 12:46:29 UTC
I noticed that the installer when does it for you to apply the policies also download it first to apply stig, but then we wouldn't test that option if we don't use the fetch remote option.
Comment 8 Rumen Chikov 2023-11-13 07:53:48 UTC
Hello Joaquín 

Because in your message we have:
++
So no bug :) thanks for the feedback. we will keep in mind this local option for future development.
++

may we consider that this bug is closed and to change it status, otherwise we continue to have this bug in our list.

Thank you in advance for your feedback.
Have a nice day
Rumen
Comment 9 Marcela Maslanova 2024-05-06 12:30:56 UTC
I'm closing the bug based on the last comment.
Comment 10 Ricardo Branco 2024-05-06 12:35:40 UTC
Fwiw, the workaround with swap didn't work sometimes even with 16G RAM.  We had to increase our VM's to 32G.