Bug 1122623 (CVE-2019-3816)

Summary: VUL-0: CVE-2019-3816, CVE-2019-3833: openwsman: Information disclosure and denial of service in openwsman
Product: [Novell Products] SUSE Security Incidents Reporter: Marcus Meissner <meissner>
Component: IncidentsAssignee: Security Team bot <security-team>
Status: RESOLVED FIXED QA Contact: Security Team bot <security-team>
Severity: Normal    
Priority: P3 - Medium CC: amajer, amaris, atoptsoglou, cbuissar, gboiko, karol, kkaempf, meissner, rfrohl
Version: unspecified   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: patch
filter path relative to service directory
Restrict urls to those registered by wsman, deny all other http(s) operations
Restrict urls to those registered by wsman, deny all other http(s) operations, and don't crash on things

Description Marcus Meissner 2019-01-21 11:15:08 UTC
via report to security@suse.de

Hello Klaus,

openwsman upstream has been recently notified of a vulnerability in
openwsman-server allowing arbitrary file disclosure via GET request. Our
maintainer proposed a patch that allows only POST and NULL request methods,
disallowing the rest of them. Could you please confirm?

First of all, we'd like to coordinate the disclosure date and ask about CVE
assignment - will SUSE handle CVE assignment or can we (Red Hat) assign CVE?

Secondly, I believe the currently proposed patch won't work as it is
possible to do the same even via POST method according to my tests on
latest upstream version. Moreover, it is also possible to easily trigger a
denial of service to openwsman-server. I haven't tracked down the root
cause yet, but I have reproducers to share.

Best Regards,

Adam Mariš,
Red Hat Product Security
qq
Comment 4 Klaus Kämpf 2019-01-21 15:36:50 UTC
(In reply to Adam Majer from comment #3)
> I'll look into it. I should have time starting on Wednesday.
> 
> Would be nice to have those reproducers. Did they send us any?

You can craft a GET request with a path like "../../../../etc/passwd" (number of '..' are not accurate ;-)) to read any file :-/
Comment 5 Marcus Meissner 2019-01-23 15:56:55 UTC
Hi,                                                                                                                                                                                          
                                                                                                                                                                                             
When running openwsmand with -S option, this will output partial content of /etc/shadow:                                                                                                     
                                                                                                                                                                                             
$ echo -e "POST /../../../../../etc/shadow HTTP/1.1\r\nContent-Type: text/plain\r\nContent-Length: 6\r\nHello\r\n\r\n" | openssl s_client -connect localhost:5986 -quiet                     
                                                                                                                                                                                             
It also works from different machine, not just locally. Moreover, this will make openwsmand to run into infinite loop:                                                                       
                                                                                                                                                                                             
$ echo -e "POST /../../../../../etc/shadow HTTP/1.1\r\nContent-Type: text/plain\r\nContent-Length: 6\r\n\r\nHello\r\n\r\n" | openssl s_client -connect localhost:5986 -quiet                 
                                                                                                                                                                                             
I tested versions since 2.2.3 up to latest upstream version and all are affected. Running without -S doesn't help.                                                                           
                                                                                                                                                                                             
Best Regards,                                                                                                                                                                                
                                                                                                                                                                                             
Adam Mariš,                                                                                                                                                                                  
Red Hat Product Security
Comment 6 Adam Maris 2019-01-25 15:14:56 UTC
Traversing the directory via double dots is actually not necessary, the problem is that program calls chdir("/") in wsmand.c so there's no need for traversing. So "POST /etc/shadow ..." works just as well.

By the way, decide_what_to_do() function actually calls remove_double_dots(), however it is still possible to traverse one level above if one puts a dot at the beginning, like "POST ./etc/shadow ..." because root directory will be prepended to this path which is set to ".", hence resulting into double dots (separate CVE?)

Another problem is that POST is behaving like GET requests. Maybe I'm misunderstanding the purpose of this package and it's designed for such system management. Nevertheless, disclosing arbitrary file without authorization (which is used for PUT and DELETE requests) is still a problem.

Another problem is the denial of service caused by invalid request which results into infinite loop (process_connection() calls connection_desctructor() which calls process_connection() again, etc.). But this seems to be unrelated to this one, so probably it requires a separate CVE.
Comment 7 Adam Majer 2019-01-29 17:29:00 UTC
(In reply to Adam Maris from comment #6)
> Traversing the directory via double dots is actually not necessary, the
> problem is that program calls chdir("/") in wsmand.c so there's no need for
> traversing. So "POST /etc/shadow ..." works just as well.

When running in foreground mode as `openwsman -d`, this chdir() doesn't occur and all subsequent calls are relative to where `openwsman -d` was started from. So if testing in foreground,

cd /
openwsman -d

> echo -e "GET /etc/shadow HTTP/1.1\r\nContent-Type: text/plain\r\n\rContent-Length: 6\r\nHello\r\n" | nc localhost 5985


The solution here would be to chdir to another subdirectory, but to me this is unclear as to *where* as there is no settings where it's suppose to change to. Maybe service_path() ? Then filter all paths that do not start with service_path() ? The attached patch does not filter the supplied paths, only does chroot, so it will fail in a real scenarios.


> By the way, decide_what_to_do() function actually calls
> remove_double_dots(), however it is still possible to traverse one level
> above if one puts a dot at the beginning, like "POST ./etc/shadow ..."
> because root directory will be prepended to this path which is set to ".",
> hence resulting into double dots (separate CVE?)

This is trivial fix for that function to just remove all dots, including leading dots. The constructs like /../......./ are already filtered. Only the first leading dots are not, and then another is prepended. But why prepend a dot considering it's in root directory?

I'm uncertain what is the purpose of this function as it strips out leading dots from dot files too, so I've renamed it.


> Another problem is that POST is behaving like GET requests. Maybe I'm
> misunderstanding the purpose of this package and it's designed for such
> system management. Nevertheless, disclosing arbitrary file without
> authorization (which is used for PUT and DELETE requests) is still a problem.
> 
> Another problem is the denial of service caused by invalid request which
> results into infinite loop (process_connection() calls
> connection_desctructor() which calls process_connection() again, etc.). But
> this seems to be unrelated to this one, so probably it requires a separate
> CVE.

Indeed, this resulted in a segfault from recursive call. This seems to be made on purpose. For example,

> echo -e "GET /etc/shadow HTTP/1.1\r\nContent-Type: text/plain\r\n\rContent-Length: 4444446\r\nContent-Type: t\n\nHello\r\n" | nc localhost 5985


I'm made a naive patch that addresses these issues. The connection is not closed, but it seems it's possible to avoid recursive calls and the connection is processed in due time. Looks like 1 second? Would that cause problems on workloads? In either case, it should not be doing this via recursion.

Alternative is to close the connection on error regardless.
Comment 8 Adam Majer 2019-01-29 17:30:06 UTC
Created attachment 795538 [details]
patch
Comment 11 Adam Majer 2019-02-05 13:30:50 UTC
Created attachment 796008 [details]
patch

Attached is a proposed patch to address these problems.

1. I've renamed the remove_double_dots to remove_all_leading_dots, as that is what the function attempts to do but fails to account for the initial leading dots

2. in parse_http_request(), the remote io_inc_tail() is never called when part of the request is already parsed and c->loc is closed. Move this before the if() to avoid reparsing the same thing over and over.

3. I've removed infinite recursion bomb between connection_desctructor and process_connection. It's a little ugly where some casts are required, but return value is ignored anyway except to avoid this potential recursive bomb. Note that recursive bomb was triggered by #2 problem in the first place but it could happen independently with a crafted request.

4. I've added a check to verify that remote connection is not closed in the do_close flag... because if it's closed an assert in process_connection then fails or do_select() ends up with an error condition because of invalid socket (socket already closed)

5. I've changed the cwd, which is the base for all requests, to service_path instead of / . I've also moved this prior to checking for the foreground option so running openwsmand with -d param or without results in same cwd.

The `cwd` will probably result in some surprises... alternative is to filter all requests to only this path? Which is preferred? What is expected by users of openwsmand?

I'd welcome feedback on this and whether this fixes problems addressed in Comment #6 *without* causing some side effects for users.
Comment 12 Adam Maris 2019-02-11 13:26:14 UTC
> 
> I'd welcome feedback on this and whether this fixes problems addressed in
> Comment #6 *without* causing some side effects for users.

Thanks for the patch, it looks good to me regarding fixing these issues. I can't comment much on side effects. However, the default value for service_path is /wsman if not specified otherwise and if it doesn't exist (it's not created by default) then it asserts. Should the creation of this directory be rather addressed in distro specific code (e.g. spec file)?

I think it would be reasonable to decouple this into two CVEs, one for the disclosure addressed by 1 and 5, and another one for the rest, causing DoS. I can get the second CVE. Would that work for you?
Comment 13 Marcus Meissner 2019-02-18 13:30:50 UTC
Redhat assigned:

CVE-2019-3833 openwsman: Infinite loop in process_connection() allows denial of service
Comment 15 Marcus Meissner 2019-02-21 15:15:34 UTC
CRD: 2019-03-04
Comment 16 Adam Majer 2019-03-04 15:57:59 UTC
Created attachment 798778 [details]
filter path relative to service directory

I've added another patch to make sure that all paths are filtered based on the service path (default: /wsman). This should keep functionality the same for paths that have service path in them, but returns 403 for all requests under other paths.

Without this, the daemon will function like in a chroot which may not be what is expected.
Comment 17 Adam Majer 2019-03-04 15:59:19 UTC
Klaus, cam be look at the patches and verify that these will not break things too badly?

The unit tests in the current package fail "equally" irrespective of applied patches, so I don't have much confidence in them.
Comment 18 Klaus Kämpf 2019-03-08 10:33:26 UTC
(In reply to Adam Majer from comment #17)
> Klaus, cam be look at the patches and verify that these will not break
> things too badly?
> 
> The unit tests in the current package fail "equally" irrespective of applied
> patches, so I don't have much confidence in them.

The service path approach, while a good one, doesn't work for all cases. wsman standard describes /wsman (authorized) and /wsman-anon (non-authorized) as allowed cases.

The restriction on POST is certainly the right step but incomplete (comment #5) :-/
Comment 19 Klaus Kämpf 2019-03-08 11:36:16 UTC
Created attachment 799384 [details]
Restrict urls to those registered by wsman, deny all other http(s) operations

Improved patch (for openwsman HEAD). All tests pass with this one.
Comment 21 Adam Majer 2019-03-08 13:30:29 UTC
Created attachment 799402 [details]
Restrict urls to those registered by wsman, deny all other http(s) operations, and don't crash on things

Thank you Klaus. I've merged the patches for the crashes with your patch.

openwsman will now run in / as before, but filters to registered URIs.
Comment 23 Karol Babioch 2019-03-12 10:15:13 UTC
Making public.
Comment 29 Swamp Workflow Management 2019-03-20 14:10:37 UTC
SUSE-SU-2019:0654-1: An update that fixes two vulnerabilities is now available.

Category: security (important)
Bug References: 1092206,1122623
CVE References: CVE-2019-3816,CVE-2019-3833
Sources used:
SUSE Linux Enterprise Module for Server Applications 15 (src):    openwsman-2.6.7-3.3.1
SUSE Linux Enterprise Module for Open Buildservice Development Tools 15 (src):    openwsman-2.6.7-3.3.1
Comment 30 Swamp Workflow Management 2019-03-20 14:13:41 UTC
SUSE-SU-2019:0656-1: An update that fixes two vulnerabilities is now available.

Category: security (important)
Bug References: 1122623
CVE References: CVE-2019-3816,CVE-2019-3833
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP4 (src):    openwsman-2.4.11-21.8.1
SUSE Linux Enterprise Software Development Kit 12-SP3 (src):    openwsman-2.4.11-21.8.1
SUSE Linux Enterprise Server 12-SP4 (src):    openwsman-2.4.11-21.8.1
SUSE Linux Enterprise Server 12-SP3 (src):    openwsman-2.4.11-21.8.1
SUSE Linux Enterprise Desktop 12-SP4 (src):    openwsman-2.4.11-21.8.1
SUSE Linux Enterprise Desktop 12-SP3 (src):    openwsman-2.4.11-21.8.1
Comment 31 Swamp Workflow Management 2019-04-02 16:17:59 UTC
openSUSE-SU-2019:1111-1: An update that fixes two vulnerabilities is now available.

Category: security (important)
Bug References: 1092206,1122623
CVE References: CVE-2019-3816,CVE-2019-3833
Sources used:
openSUSE Leap 15.0 (src):    openwsman-2.6.7-lp150.2.3.1

*** NOTE: This information is not intended to be used for external
    communication, because this may only be a partial fix.
    If you have questions please reach out to maintenance coordination.
Comment 32 Marcus Meissner 2019-04-03 05:22:59 UTC
Adam, openSUSE Leap 42.3 fix is missing, as its not auto imported from SLES.
Comment 33 Swamp Workflow Management 2019-04-03 09:20:15 UTC
This is an autogenerated message for OBS integration:
This bug (1122623) was mentioned in
https://build.opensuse.org/request/show/690896 42.3 / openwsman
Comment 34 Adam Majer 2019-04-03 10:39:45 UTC
(In reply to Marcus Meissner from comment #32)
> Adam, openSUSE Leap 42.3 fix is missing, as its not auto imported from SLES.

Sorry, didn't notice that. I've submitted the Leap 15.0 update to Leap 42.3 now. The version updates are just bug fixes since Openwsman has been under maintenance-only for number of years now.
Comment 36 Marcus Meissner 2019-04-05 15:19:38 UTC
42.3 will be released soon, rest is done.
Comment 37 Swamp Workflow Management 2019-04-16 19:09:10 UTC
openSUSE-SU-2019:1217-1: An update that fixes two vulnerabilities is now available.

Category: security (important)
Bug References: 1092206,1122623
CVE References: CVE-2019-3816,CVE-2019-3833
Sources used:
openSUSE Leap 42.3 (src):    openwsman-2.6.7-4.3.1