Bugzilla – Bug 1197415
gdb fails to find debugging symbols for libraries
Last modified: 2022-03-24 12:22:28 UTC
Trying to debug a binary, like this: `gdb /usr/libexec/virtiofsd 7596` I see the following: > Reading symbols from /usr/libexec/virtiofsd... > Reading symbols from /usr/lib/debug/usr/libexec/virtiofsd-6.2.0-42.1.x86_64.debug... > Attaching to program: /usr/libexec/virtiofsd, process 7596 > [New LWP 7600] > Error while mapping shared library sections: > Could not open `target:/lib64/libseccomp.so.2' as an executable file: No such file or directory > Error while mapping shared library sections: > Could not open `target:/lib64/libcap-ng.so.0' as an executable file: No such file or directory > Error while mapping shared library sections: > Could not open `target:/lib64/libgmodule-2.0.so.0' as an executable file: No such file or directory > Error while mapping shared library sections: > Could not open `target:/lib64/libglib-2.0.so.0' as an executable file: No such file or directory > Error while mapping shared library sections: > Could not open `target:/lib64/libc.so.6' as an executable file: No such file or directory > Error while mapping shared library sections: > Could not open `target:/lib64/ld-linux-x86-64.so.2' as an executable file: No such file or directory > Error while mapping shared library sections: > Could not open `target:/lib64/libpcre.so.1' as an executable file: No such file or directory And this is despite the fact that (I think!) I have the debugging symbols installed > Solace:~ # rpm -qa|grep 'glibc\|libseccomp' > glibc-2.35-2.1.x86_64 > libseccomp2-2.5.3-4.5.x86_64 > glibc-locale-2.35-2.1.x86_64 > glibc-extra-2.35-2.1.x86_64 > glibc-locale-base-2.35-2.1.x86_64 > glibc-debugsource-2.35-2.1.x86_64 > glibc-debuginfo-2.35-2.1.x86_64 > libseccomp-debugsource-2.5.3-4.5.x86_64 > libseccomp2-debuginfo-2.5.3-4.5.x86_64 > glibc-extra-debuginfo-2.35-2.1.x86_64 And: > Solace:~ # ls -l /lib64/libc.so.6 > -rwxr-xr-x 1 root root 2551568 Mar 2 18:58 /lib64/libc.so.6 > Solace:~ # ls -l /usr/lib64/libc.so.6 > -rwxr-xr-x 1 root root 2551568 Mar 2 18:58 /usr/lib64/libc.so.6 > Solace:~ # ls -l /usr/lib/debug/usr/lib64/libc.so.6* > -rw-r--r-- 1 root root 6211808 Mar 2 18:58 /usr/lib/debug/usr/lib64/libc.so.6-2.35-2.1.x86_64.debug As we see, it fails with libraries, while it seems to work with binaries. I tried installing `elfutils-debuginfod-0.186-2.1.x86_64` and starting the `debuginfod` service, but this does not change much. Am I missing anything or misuing something?
Could not open `target:/lib64/libseccomp.so.2' as an executable file: No such file or directory I'm not sure why gdb(?) tries to open this but for me it's in /usr/lib64, not /lib64. Is virtiofsd maybe run inside some container/namespace that confuses gdb?
At least virtiofsd supports sandboxes/chroots: % /usr/libexec/virtiofsd --help ... -o sandbox=namespace|chroot sandboxing mode: - namespace: mount, pid, and net namespaces with pivot_root(2) into shared directory - chroot: chroot(2) into shared directory (use in containers) default: namespace ... (in particular: note the default!). Attaching to running processes requires some heuristics in gdb in order to find the correct files that the process has loaded, and in this case it seems that those heuristics fail. In some cases those heuristics can't find out the correct rootdir at all, think about a process that loads "/foo/bar.so", then chroots into "/X", loads "/a.so", then chroots into "/Y", then loads "/b.so". The process image of the process will say it loaded "/b.so" and "/a.so", but there's no way to know that the latter was "/X/a.so", not "/X/Y/a.so" (should that exist). So, it might be that gdb's heuristics could be made to work in this scenario, until then you need to help gdb, e.g. by setting gdb's sysroot to "/": (gdb) show sysroot The current system root is "target:". (gdb) set sysroot / Giving to Tom for investigating if something could be improved on gdb's side for this case. For that, Dario: please provide a snippet of how you (or qemu) starts virtiofsd, I didn't manage to start it myself to see your example break.
(In reply to Richard Biener from comment #1) > Could not open `target:/lib64/libseccomp.so.2' as an executable file: No > such file or directory > > I'm not sure why gdb(?) tries to open this but for me it's in /usr/lib64, > not /lib64. > It's in both places, here in my system: > # ls -l /usr/lib64/libseccomp.so.2 > lrwxrwxrwx 1 root root 19 Feb 19 22:52 /usr/lib64/libseccomp.so.2 -> libseccomp.so.2.5.3 > # ls -l /lib64/libseccomp.so.2 > lrwxrwxrwx 1 root root 19 Feb 19 22:52 /lib64/libseccomp.so.2 -> libseccomp.so.2.5.3 Maybe due to this? > # ls -l /|grep lib64 > lrwxrwxrwx 1 root root 9 Mar 9 20:00 lib64 -> usr/lib64 > Is virtiofsd maybe run inside some container/namespace that > confuses gdb? > cgroups are involved for sure, due to how libvirt configures things: > # pstree -pa |grep virtiofs > |-virtiofsd,8045 --fd=34 -o source=/home/test/share -d > | |-virtiofsd,8072 --fd=34 -o source=/home/test/share -d > | | |-{virtiofsd},8073 > | | |-{virtiofsd},8395 > | | `-{virtiofsd},8396 > | `-{virtiofsd},8048 # cat /proc/8045/task/*/cgroup 0::/machine.slice/machine-qemu\x2d1\x2dopensusetumbleweed\x2djeos.scope/libvirt/emulator 0::/machine.slice/machine-qemu\x2d1\x2dopensusetumbleweed\x2djeos.scope/libvirt/emulator # cat /proc/8072/task/*/cgroup 0::/machine.slice/machine-qemu\x2d1\x2dopensusetumbleweed\x2djeos.scope/libvirt/emulator 0::/machine.slice/machine-qemu\x2d1\x2dopensusetumbleweed\x2djeos.scope/libvirt/emulator 0::/machine.slice/machine-qemu\x2d1\x2dopensusetumbleweed\x2djeos.scope/libvirt/emulator 0::/machine.slice/machine-qemu\x2d1\x2dopensusetumbleweed\x2djeos.scope/libvirt/emulator Namespace wise, can you tell what kind of information I can share, to help you understand the situation? For now: > # readlink /proc/8045/task/*/ns/* | sort -u > cgroup:[4026531835] > ipc:[4026531839] > mnt:[4026531840] > mnt:[4026532396] > net:[4026531992] > net:[4026532399] > pid:[4026531836] > pid:[4026532397] > time:[4026531834] > user:[4026531837] > uts:[4026531838] > # readlink /proc/8072/task/*/ns/* | sort -u > cgroup:[4026531835] > ipc:[4026531839] > mnt:[4026532396] > net:[4026532399] > pid:[4026532397] > time:[4026531834] > user:[4026531837] > uts:[4026531838]
(In reply to Michael Matz from comment #2) > So, it might be that gdb's heuristics could be made to work in this scenario, > until then you need to help gdb, e.g. by setting gdb's sysroot to "/": > > (gdb) show sysroot > The current system root is "target:". > (gdb) set sysroot / > > Giving to Tom for investigating if something could be improved on gdb's side > for this case. For that, Dario: please provide a snippet of how you (or > qemu) > starts virtiofsd, I didn't manage to start it myself to see your example > break. > In theory, with something like this should work: > # /usr/libexec/virtiofsd --socket-path=/tmp/vsock -o source=/home/test/share -o cache=always > # qemu-system-x86_64 -M pc -cpu host --enable-kvm -smp 2 -m 4G \ > -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on -numa node,memdev=mem \ > -chardev socket,id=char0,path=/tmp/vsock -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs \ > -chardev stdio,mux=on,id=mon -mon chardev=mon,mode=readline -device virtio-serial-pci -device virtconsole,chardev=mon \ > -vga none -display none -drive if=virtio,file=/home/test/openSUSE-Tumbleweed-JeOS.x86_64-kvm-and-xen.qcow2 It's different than my usecase as, e.g., libvirt is not there creating the cgroups etc. But I still see the problem if I try to attach with gdb. Instead, in order to replicate exactly what I'm using, I guess the best way is to create a simple VM, e.g., with virt-manager, and then add a virtiofs shared filesystem, either from the GUI or adding this to the XML: > <filesystem type="mount" accessmode="passthrough"> > <driver type="virtiofs"/> > <source dir="/home/test/share"/> > <target dir="host_share"/> > </filesystem> Let me know if this helps.