Bug 1090849 (CVE-2018-3817)

Bryan can you review and add your assesement?

This is not a serious issue and  it does not need an urgent patch.  We should update to a recent version, or at least a non-ancient version, of logstash for Cloud 8 MU1.

@Johannes: Logstash is used by Monasca do you know if we can update to a newer version?

I'm not 100% sure, but I wouldn't be surprised if the Monasca plugin for Logstash depended on 2.4.x. I'll build a fresh cloud and take a closer look at it and/or test a later logstash version. I'm not optimistic, though...maybe better to cherry-pick this patch.

If Monasca depends on 2.4.x we should at least consider making the changes in Monasca to support a more recent version of logstash instead of cherry-picking vulnerability fixes for logstash and back-porting them to older logstash versions. Presumably Monasca will eventually support a more recent version of logstash, so the work to support the more recent version will need to be done eventually. Given that the work needs to be done eventually, there is less total work performed if we do that work now because we avoid the work of back-porting vulnerability fixes for logstash.

It is also sometimes the right trade-off to do more work in total in the long term to achieve the benefit of doing less work in the short term. I leave that trade-off to others to decide, but I want to make sure someone is making an informed choice and understanding how much short-term benefit is gained by doing more work in total.

In any case, if whatever solution we pursue will take some time this does not need to be fixed in MU1 or MU2, as long as we do work on it and get it fixed when appropriate.

@Joe: Can you please create a Jira ticket for tracking this as part of the Monitoring Squad? We should try to get this fixed within the next 4 weeks.

@Joe: Any news?

Created attachment 771981 [details]
Patch that gets security:logging/logstash running

I took a quick look at this while waiting for a Crowbar run to finish. Here's what I found so far:

* The more (far more, actually) recent logstash package in security:logging is currently broken and its logstash executable won't even run (the attached patch fixes this; I'll check if it's upstream and try to submit it upstream/against the package later)
* The systemd service file for monasca-log-agent will need a few changes (path to log stash executable, removal of `agent` command). This will affect monasca-log-transformer and monasca-log-metrics as well since they are logstash based.
* With both fixes in place, logstash fails with the following stacktrace:

May 31 13:30:01 d52-54-77-77-01-04 logstash[31335]: 13:30:01.439 [LogStash::Runner] FATAL logstash.runner - An unexpected error occurred! {:error=>java.nio.file.FileAlreadyExistsException: /usr/share/logstash/data, :backtrace=>["sun.nio.fs.UnixException.translateToIOException(sun/nio/fs/UnixException.java:88)", "sun.nio.fs.UnixException.rethrowAsIOException(sun/nio/fs/UnixException.java:102)", "sun.nio.fs.UnixException.rethrowAsIOException(sun/nio/fs/UnixException.java:107)", "sun.nio.fs.UnixFileSystemProvider.createDirectory(sun/nio/fs/UnixFileSystemProvider.java:384)", "java.nio.file.Files.createDirectory(java/nio/file/Files.java:674)", "java.nio.file.Files.createAndCheckIsDirectory(java/nio/file/Files.java:781)", "java.nio.file.Files.createDirectories(java/nio/file/Files.java:727)", "org.logstash.FileLockFactory.obtainLock(org/logstash/FileLockFactory.java:66)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.execute(/usr/share/logstash/logstash-core/lib/logstash/runner.rb:270)", "RUBY.run(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67)", "RUBY.run(/usr/share/logstash/logstash-core/lib/logstash/runner.rb:185)", "RUBY.run(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132)", "usr.share.logstash.lib.bootstrap.environment.(root)(/usr/share/logstash/lib/bootstrap/environment.rb:71)", "usr.share.logstash.lib.bootstrap.environment.(root)(usr/share/logstash/lib/bootstrap//usr/share/logstash/lib/bootstrap/environment.rb:71)"]}

Haven't had a closer look at this, yet since my Crowbar run finished now...

https://jira.prv.suse.net/browse/SCRD-3561

I backported the patch to 2.4.1 now:

https://build.opensuse.org/package/show/home:jgrassler:branches:Cloud:OpenStack:Master/logstash

I also encountered a little trouble when testing the upgrade: in its current shape our logstash package overwrites the logstash plugin registry upon upgrade, thus effectively deregistering the Monasca output plugin. I'll give this another spin once OBS has published the package and submit requests once I'm sure this works out.

As for upgrading to a more recent logtash: I think we should do this, too, but I don't think we should mix it with fixing this CVE for that will needlessly slow down the CVE fix.

Also, this problem affects Cloud 7 as well, hence I'll clone this bug for Cloud 7 (please keep all technical comments in regards to fixing this issue on here - the Cloud 7 bug is just for tracking the Cloud 7 backport).

Tested and works. Here are the requests for all Cloud repositories except for Cloud:OpenStack:Queens (we don't have a logstash package in there, yet):

https://build.opensuse.org/request/show/614863
https://build.opensuse.org/request/show/614866
https://build.opensuse.org/request/show/614867
https://build.opensuse.org/request/show/614868

For SOC7 and Crowbar flavoured SOC 8 we'll need to ensure openstack-monasca-log-metrics, openstack-monasca-log-transformer (on the Monasca node) and openstack-monasca-log-agent (on all nodes with the monasca-log-agent role) are restarted. I'll look into cobbling up something to pull that off with. If worse comes to worst we'll just need to document the need to restart these services in the release notes.

It would be just about workable to restart openstack-monasca-log-agent automatically but it's pretty wobbly. And getting monasca-installer to run and take care of openstack-monasca-log-{metrics,transformer} would require manual intervention as well. Hence I'd prefer adding the following to the release notes for the maintance update that includes this fix:

"Logstash is used by the openstack-monasca-log-metrics, openstack-monasca-log-transformer and the openstack-monasca-log-agent services. To ensure all of these services run the updated logstash version proceed as follows:

1) Update logstash package on all machines
2) Run `systemctl restart openstack-monasca-log-agent` on all machines with the monasca-log-agent Crowbar role
3) Run `systemctl restart openstack-monasca-log-metrics openstack-monasca-log-transformer` on the machine with the monasca-server Crowbar role."

Johannes, Has a fix for this been merged?  Can we resolve it and the associated Jira ticket SCRD-3561 ?

Joseph, no the fix has not been fully merged, yet: while it is in OBS and in our internal repositories already, this will need to go through QA first. Once QA signs off on this (depends on how busy they are), this will make into a maintenance update and thus into the product media. Once the maintenance update is out, I'll update/resolve the bug.

SUSE-SU-2018:2317-1: An update that solves two vulnerabilities and has 5 fixes is now available.

Category: security (moderate)
Bug References: 1090336,1090849,1094448,1095603,1096985,1097847,1101366
CVE References: CVE-2018-12099,CVE-2018-3817
Sources used:
SUSE OpenStack Cloud Crowbar 8 (src):    grafana-4.5.1-4.3.1, kafka-0.9.0.1-5.3.1, logstash-2.4.1-5.4.1, openstack-monasca-installer-20180622_15.06-3.6.1
SUSE OpenStack Cloud 8 (src):    grafana-4.5.1-4.3.1, kafka-0.9.0.1-5.3.1, logstash-2.4.1-5.4.1, openstack-monasca-installer-20180622_15.06-3.6.1
HPE Helion Openstack 8 (src):    grafana-4.5.1-4.3.1, kafka-0.9.0.1-5.3.1, logstash-2.4.1-5.4.1, openstack-monasca-installer-20180622_15.06-3.6.1

SUSE-SU-2018:2536-1: An update that solves three vulnerabilities and has 5 fixes is now available.

Category: security (moderate)
Bug References: 1086909,1090192,1090343,1090849,1094448,1095603,1096985,1102920
CVE References: CVE-2018-12099,CVE-2018-1288,CVE-2018-3817
Sources used:
SUSE OpenStack Cloud 7 (src):    grafana-4.5.1-1.8.1, kafka-0.10.2.2-5.1, logstash-2.4.1-5.1, monasca-installer-20180608_12.47-9.1

Updated versions merged, so closing.

clearing needinfo requests