Bug 1214292

Summary:	zypper signal handler causes deadlock
Product:	[openSUSE] openSUSE Tumbleweed	Reporter:	Fabian Vogt <fabian>
Component:	libzypp	Assignee:	E-mail List <zypp-maintainers>
Status:	RESOLVED FIXED	QA Contact:	E-mail List <qa-bugs>
Severity:	Normal
Priority:	P5 - None	CC:	arozyev, bzeller, fvogt
Version:	Current
Target Milestone:	---
Hardware:	Other
OS:	Other
Whiteboard:
Found By:	---	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---

Description Fabian Vogt 2023-08-15 13:11:42 UTC

I pressed Ctrl-C while zypper was busy calculating the "dup". This apparently caused a deadlock, the process does not react to any input anymore.

With gdb, it's visible that the cause is that a signal handler calls "malloc", which is not allowed (not async-signal-safe).

(gdb) bt
#0  0x00007f2c72e8c79b in __lll_lock_wait_private () at /lib64/libc.so.6
#1  0x00007f2c72ea13e0 in malloc () at /lib64/libc.so.6
#2  0x00007f2c732bc9bc in operator new(unsigned long) () at /lib64/libstdc++.so.6
#3  0x00007f2c739739d2 in  () at /lib64/libzypp.so.1722
#4  0x00007f2c73bb2018 in  () at /lib64/libzypp.so.1722
#5  0x00007f2c733459dd in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) () at /lib64/libstdc++.so.6
#6  0x000055aa51d6f085 in  ()
#7  0x000055aa51d90dc9 in Zypper::instance() ()
#8  0x000055aa51d84309 in signal_handler(int) ()
#9  0x00007f2c72e3f1f0 in <signal handler called> () at /lib64/libc.so.6
#10 0x00007f2c72e9fa25 in _int_malloc () at /lib64/libc.so.6
#11 0x00007f2c72ea11ad in malloc () at /lib64/libc.so.6
#12 0x00007f2c732bc9bc in operator new(unsigned long) () at /lib64/libstdc++.so.6
#13 0x00007f2c73aa9bf1 in zypp::PoolItem::PoolItem(zypp::PoolItem::Impl*) () at /lib64/libzypp.so.1722
#14 0x00007f2c73ab0b56 in zypp::PoolItem::makePoolItem(zypp::sat::Solvable const&) () at /lib64/libzypp.so.1722
#15 0x00007f2c73afff1d in  () at /lib64/libzypp.so.1722
#16 0x00007f2c73b04cc5 in zypp::ResPool::setHardLockQueries(std::__cxx11::list<zypp::PoolQuery, std::allocator<zypp::PoolQuery> > const&) () at /lib64/libzypp.so.1722
#17 0x00007f2c73a2f531 in zypp::target::TargetImpl::load(bool) () at /lib64/libzypp.so.1722
#18 0x000055aa51db9707 in load_target_resolvables(Zypper&) ()
#19 0x000055aa51db9998 in load_resolvables(Zypper&) ()
#20 0x000055aa51dd11e8 in ZypperBaseCommand::defaultSystemSetup(Zypper&, zypp::base::Flags<SetupSystemBits>) ()
#21 0x000055aa51dd16af in ZypperBaseCommand::run(Zypper&) ()
#22 0x000055aa51d92903 in Zypper::doCommand(int, char**, int) ()
#23 0x000055aa51d67dfa in Zypper::main(int, char**) ()
#24 0x000055aa51d66a21 in main ()

Comment 1 Benjamin Zeller 2023-08-21 12:24:29 UTC

Seems you managed to hit Ctrl+C when malloc had a lock aquired, unfortunately the signal handlers are doing too much things that are not signal safe and call into malloc as well , which deadlocks.

Comment 2 Fabian Vogt 2023-08-21 12:44:49 UTC

(In reply to Benjamin Zeller from comment #1)
> Seems you managed to hit Ctrl+C when malloc had a lock aquired,
> unfortunately the signal handlers are doing too much things that are not
> signal safe and call into malloc as well , which deadlocks.

Yes. The signal handlers can't really do much of their own, but instead influence the program to handle the signal after returning or terminate the program: https://www.gnu.org/software/libc/manual/html_node/Defining-Handlers.html#Defining-Handlers

Comment 3 Benjamin Zeller 2023-08-21 14:10:36 UTC

I reworked the SIGINT handler to only set a flag, and updated some of the code to check regularly for the flag. 

SIGPIPE handlers still are problematic, but we need them to write to the log for now... But they are invoked only rarely. 

This PR should fix the SIGINT problems though:

https://github.com/openSUSE/zypper/pull/499

Comment 4 Michael Andres 2023-08-31 16:12:24 UTC

fixed in zypper-1.14.64

Comment 7 Maintenance Automation 2023-10-03 12:30:06 UTC

SUSE-RU-2023:3937-1: An update that has four fixes can now be installed.

Category: recommended (moderate)
Bug References: 1213854, 1214292, 1214395, 1215007
Sources used:
SUSE Linux Enterprise High Performance Computing 15 SP1 LTSS 15-SP1 (src): zypper-1.14.64-150100.3.87.1
SUSE Linux Enterprise Server 15 SP1 LTSS 15-SP1 (src): zypper-1.14.64-150100.3.87.1
SUSE Linux Enterprise Server for SAP Applications 15 SP1 (src): zypper-1.14.64-150100.3.87.1
SUSE CaaS Platform 4.0 (src): zypper-1.14.64-150100.3.87.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.

Comment 8 Maintenance Automation 2023-10-05 12:29:37 UTC

SUSE-RU-2023:3973-1: An update that has four fixes can now be installed.

Category: recommended (moderate)
Bug References: 1213854, 1214292, 1214395, 1215007
Sources used:
openSUSE Leap 15.4 (src): zypper-1.14.64-150400.3.32.1
openSUSE Leap 15.5 (src): zypper-1.14.64-150400.3.32.1
SUSE Linux Enterprise Micro for Rancher 5.3 (src): zypper-1.14.64-150400.3.32.1
SUSE Linux Enterprise Micro 5.3 (src): zypper-1.14.64-150400.3.32.1
SUSE Linux Enterprise Micro for Rancher 5.4 (src): zypper-1.14.64-150400.3.32.1
SUSE Linux Enterprise Micro 5.4 (src): zypper-1.14.64-150400.3.32.1
Basesystem Module 15-SP4 (src): zypper-1.14.64-150400.3.32.1
Basesystem Module 15-SP5 (src): zypper-1.14.64-150400.3.32.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.

Comment 9 Maintenance Automation 2023-10-09 08:28:52 UTC

SUSE-RU-2023:4006-1: An update that has four fixes can now be installed.

Category: recommended (moderate)
Bug References: 1213854, 1214292, 1214395, 1215007
Sources used:
SUSE Manager Retail Branch Server 4.2 (src): zypper-1.14.64-150200.62.1
SUSE Manager Server 4.2 (src): zypper-1.14.64-150200.62.1
SUSE Enterprise Storage 7.1 (src): zypper-1.14.64-150200.62.1
SUSE Enterprise Storage 7 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Micro 5.1 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Micro 5.2 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Micro for Rancher 5.2 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise High Performance Computing 15 SP2 LTSS 15-SP2 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise High Performance Computing ESPOS 15 SP3 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise High Performance Computing LTSS 15 SP3 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Server 15 SP2 LTSS 15-SP2 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Server 15 SP3 LTSS 15-SP3 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Server for SAP Applications 15 SP2 (src): zypper-1.14.64-150200.62.1
SUSE Linux Enterprise Server for SAP Applications 15 SP3 (src): zypper-1.14.64-150200.62.1
SUSE Manager Proxy 4.2 (src): zypper-1.14.64-150200.62.1

NOTE: This line indicates an update has been released for the listed product(s). At times this might be only a partial fix. If you have questions please reach out to maintenance coordination.