Bug 1062571 - libvirt segfault: concurrent access between pool refresh and volume create
libvirt segfault: concurrent access between pool refresh and volume create
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE Distribution
Classification: openSUSE
Component: Virtualization:Other
Leap 42.3
Other Other
: P3 - Medium : Normal (vote)
: ---
Assigned To: Cédric Bosdonnat
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2017-10-10 14:00 UTC by Flavio Castelli
Modified: 2018-01-31 23:47 UTC (History)
2 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---


Attachments
valgrind logs (8.98 MB, text/plain)
2017-10-10 14:02 UTC, Flavio Castelli
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Flavio Castelli 2017-10-10 14:00:38 UTC
libvirt crashes when creating multiple volumes inside of the same pool and asking the pool to refresh.

According to some debugging I did with Cedric, this could be happening because the pool refresh function frees the volume data structures that are already in use.

The issue is happening on openSUSE 42.3, but I think it can be easily reproduced somewhere else.

These are the libvirt packages I've installed on my system:

  * libvirt-daemon-driver-storage-logical-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-scsi-3.3.0-6.1.x86_64
  * libvirt-python-3.3.0-1.2.x86_64
  * libvirt-daemon-3.3.0-6.1.x86_64
  * libvirt-daemon-debuginfo-3.3.0-6.1.x86_64
  * libvirt-libs-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-qemu-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-interface-3.3.0-6.1.x86_64
  * libvirt-libs-debuginfo-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-nwfilter-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-network-debuginfo-3.3.0-6.1.x86_64
  * libvirt-doc-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-disk-3.3.0-6.1.x86_64
  * libvirt-glib-1_0-0-0.2.3-3.1.x86_64
  * libvirt-daemon-driver-storage-core-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-mpath-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-disk-debuginfo-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-secret-3.3.0-6.1.x86_64
  * libvirt-debugsource-3.3.0-6.1.x86_64
  * libvirt-devel-3.3.0-6.1.x86_64
  * libvirt-daemon-qemu-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-rbd-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-network-3.3.0-6.1.x86_64
  * libvirt-client-debuginfo-3.3.0-6.1.x86_64
  * libvirt-client-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-iscsi-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-nodedev-3.3.0-6.1.x86_64
  * libvirt-daemon-driver-storage-core-debuginfo-3.3.0-6.1.x86_64

How to reproduce the issue:

  * run libvirt locally
  * download terraform 0.10.* from https://www.terraform.io/downloads.html
  * add the terraform binary to $PATH
  * install go 1.9.* (devel:languages:go has it)
  * ensure you have the libvirt development headers available
  * `mkdir -p ~/go/src/github.com/dmacvicar`
  * `cd ~/go/src/github.com/dmacvicar`
  * checkout this git repository branch https://github.com/dmacvicar/terraform-provider-libvirt/tree/reproduce-libvirt-crash
  * enter the `terraform-provider-libvirt` and run `go install`; if needed `export GOPATH=~/go:$GOPATH`
  * ensure `~/go/bin/terraform-provider-libvirt` exists
  * create `~/.terraformrc` with the following contents:

```
providers {
  libvirt = "/home/FIXME/go/bin/terraform-provider-libvirt"
}
```

  * create a `test` directory and download this [1] file inside of it
  * run `terraform apply`


[1] https://gist.github.com/flavio/28388f85aa83a82bd64b28cce4faa5af
Comment 1 Flavio Castelli 2017-10-10 14:02:10 UTC
Created attachment 743746 [details]
valgrind logs
Comment 2 Cédric Bosdonnat 2017-11-22 13:14:16 UTC
Will be in the next 12SP3 and 42.3 updates
Comment 3 Cédric Bosdonnat 2018-01-10 12:47:34 UTC
fix is upstream and pending maintenance update: closing
Comment 4 Flavio Castelli 2018-01-14 20:49:19 UTC
Can you tell me which version of libvirt got the fix? I need that to update the requirements of our terraform-libvirt provider.

I tested the code against the latest packages available from obs://Virtualization (libvirt-daemon-3.10.0-709.3.x86_64) and I cannot reproduce the issue anymore (yay!).

Unfortunately the packages inside of the updates repositories of Leap 42.3 haven't got the patch yet.

I tried to look inside of our RPM changelog but I couldn't find any reference to this bug number.

I've also looked upstream to all your commits but I couldn't find it :(
Comment 5 Cédric Bosdonnat 2018-01-15 08:07:44 UTC
It's not yet in the Maintenance pipe, it's stacked for the next maintenance update, not sure when that one will be.
Comment 6 Flavio Castelli 2018-01-15 08:54:45 UTC
I spoke with Cedric this morning:
  * The fix got merged with https://libvirt.org/git/?p=libvirt.git;a=commit;h=2dd70901db8b7fd62592b1332370148e97062431
  * Upstream has the fix starting with libvirt 3.10 release
Comment 7 James Fehlig 2018-01-15 17:28:17 UTC
(In reply to Cédric Bosdonnat from comment #5)
> It's not yet in the Maintenance pipe, it's stacked for the next maintenance
> update, not sure when that one will be.

I submitted libvirt for a SLE12 SP3 update last week. It feeds 42.3 maintenance and will appear shortly after the SP3 update.

https://build.suse.de/request/show/150994
Comment 8 Swamp Workflow Management 2018-01-30 11:13:19 UTC
SUSE-SU-2018:0279-1: An update that solves one vulnerability and has 9 fixes is now available.

Category: security (moderate)
Bug References: 1035442,1052825,1062571,1062760,1064947,1065766,1070130,1072887,1073973,1076500
CVE References: CVE-2018-5748
Sources used:
SUSE Linux Enterprise Software Development Kit 12-SP3 (src):    libvirt-3.3.0-5.13.1
SUSE Linux Enterprise Server 12-SP3 (src):    libvirt-3.3.0-5.13.1
SUSE Linux Enterprise Desktop 12-SP3 (src):    libvirt-3.3.0-5.13.1
Comment 9 Swamp Workflow Management 2018-01-31 23:12:09 UTC
openSUSE-SU-2018:0322-1: An update that solves one vulnerability and has 9 fixes is now available.

Category: security (moderate)
Bug References: 1035442,1052825,1062571,1062760,1064947,1065766,1070130,1072887,1073973,1076500
CVE References: CVE-2018-5748
Sources used:
openSUSE Leap 42.3 (src):    libvirt-3.3.0-12.1