Bugzilla – Bug 324760
delegate tests hang on buildbot on QueueUserAPC
Last modified: 2008-05-21 22:45:54 UTC
---- Reported by lupus@ximian.com 2007-07-12 12:22:36 MST ---- This is the status of a hung process (running tests/delegate1.exe on the x86 buildbot). Starting program: /tmp/scratch/BUILD/mono-81872/mono/mini/mono /tmp/delegate1.exe [Thread debugging using libthread_db enabled] [New Thread 16384 (LWP 27988)] [New Thread 32769 (LWP 27989)] [New Thread 16386 (LWP 27990)] [New Thread 32771 (LWP 27991)] [New Thread 49156 (LWP 27992)] [New Thread 65541 (LWP 27993)] [New Thread 81926 (LWP 27994)] [New Thread 98311 (LWP 27995)] Test.F from delegate: 1 Test.F from delegate: 3 Test.F from delegate: 2 Test.F from delegate: 4 Async Callback STATE1 Result = 1 Async Callback STATE4 Async Callback STATE2 Async Callback STATE3 cant execute EndInvoke twice ... OK completed1: True completed2: True completed3: True completed4: True Program received signal SIGINT, Interrupt. [Switching to Thread 32771 (LWP 27991)] 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 (gdb) info threads 8 Thread 98311 (LWP 27995) 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 7 Thread 81926 (LWP 27994) 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 6 Thread 65541 (LWP 27993) 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 5 Thread 49156 (LWP 27992) 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 * 4 Thread 32771 (LWP 27991) 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 3 Thread 16386 (LWP 27990) 0x400a0d36 in nanosleep () from /lib/i686/libpthread.so.0 2 Thread 32769 (LWP 27989) 0x401b526a in poll () from /lib/i686/libc.so.6 1 Thread 16384 (LWP 27988) 0x400a0d36 in nanosleep () from /lib/i686/libpthread.so.0 (gdb) t a a bt Thread 8 (Thread 98311 (LWP 27995)): #0 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 #1 0x4009bd08 in __pthread_wait_for_restart_signal () from /lib/i686/libpthread.so.0 #2 0x4009e8bd in __pthread_alt_lock () from /lib/i686/libpthread.so.0 #3 0x4009b8b5 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #4 0x08119628 in QueueUserAPC (apc_callback=0xfffffffc, handle=0x40c, param=0xfffffffc) at threads.c:1065 #5 0x080cd69e in mono_thread_request_interruption (running_managed=0) at threads.c:2881 #6 0x08135054 in sigusr1_signal_handler (_dummy=36, info=0x41101418, context=0x41101498) at mini.c:11170 #7 0x400a0149 in __pthread_sighandler_rt () from /lib/i686/libpthread.so.0 #8 <signal handler called> #9 0x4012f494 in sigprocmask () from /lib/i686/libc.so.6 #10 0x4012f0e9 in siglongjmp () from /lib/i686/libc.so.6 #11 0x4009ba74 in siglongjmp () from /lib/i686/libpthread.so.0 #12 0x4009c8cf in pthread_handle_sigrestart () from /lib/i686/libpthread.so.0 #13 <signal handler called> #14 0x400a0d36 in nanosleep () from /lib/i686/libpthread.so.0 #15 0x00000001 in ?? () #16 0x4009bbe0 in __pthread_timedsuspend_new () from /lib/i686/libpthread.so.0 #17 0x40099323 in pthread_cond_timedwait@GLIBC_2.0 () from /lib/i686/libpthread.so.0 #18 0x08108a27 in timedwait_signal_wait_cond (cond=0x4059902c, mutex=0x40599014, timeout=0x0, alertable=1, waiting_on=0x400) at handles.c:1422 #19 0x0811c26c in WaitForSingleObjectEx (handle=0x400, timeout=10000, alertable=1) at wait.c:202 #20 0x080cf834 in async_invoke_thread (data=0x0) at threadpool.c:1228 #21 0x080cd325 in start_wrapper (data=0x8281268) at threads.c:319 #22 0x0811b0d0 in thread_start_routine (args=0x405eb8a0) at threads.c:253 #23 0x081318e4 in GC_start_routine () #24 0x4009a021 in pthread_start_thread () from /lib/i686/libpthread.so.0 #25 0x4009a1de in pthread_start_thread_event () from /lib/i686/libpthread.so.0 #26 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 7 (Thread 81926 (LWP 27994)): #0 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 #1 0x4009bd08 in __pthread_wait_for_restart_signal () from /lib/i686/libpthread.so.0 #2 0x4009e8bd in __pthread_alt_lock () from /lib/i686/libpthread.so.0 #3 0x4009b8b5 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #4 0x08119628 in QueueUserAPC (apc_callback=0xfffffffc, handle=0x40b, param=0xfffffffc) at threads.c:1065 #5 0x080cd69e in mono_thread_request_interruption (running_managed=0) at threads.c:2881 #6 0x08135054 in sigusr1_signal_handler (_dummy=36, info=0x41000860, context=0x410008e0) at mini.c:11170 #7 0x400a0149 in __pthread_sighandler_rt () from /lib/i686/libpthread.so.0 #8 <signal handler called> #9 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 #10 0x4009bd08 in __pthread_wait_for_restart_signal () from /lib/i686/libpthread.so.0 #11 0x4009e8bd in __pthread_alt_lock () from /lib/i686/libpthread.so.0 #12 0x4009b8b5 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #13 0x400993bc in pthread_cond_timedwait@GLIBC_2.0 () from /lib/i686/libpthread.so.0 #14 0x08108a27 in timedwait_signal_wait_cond (cond=0x4059902c, mutex=0x40599014, timeout=0xffffffff, alertable=1, waiting_on=0x400) at handles.c:1422 #15 0x0811c26c in WaitForSingleObjectEx (handle=0x400, timeout=10000, alertable=1) at wait.c:202 #16 0x080cf834 in async_invoke_thread (data=0x0) at threadpool.c:1228 #17 0x080cd325 in start_wrapper (data=0x8281268) at threads.c:319 #18 0x0811b0d0 in thread_start_routine (args=0x405eb768) at threads.c:253 #19 0x081318e4 in GC_start_routine () #20 0x4009a021 in pthread_start_thread () from /lib/i686/libpthread.so.0 #21 0x4009a1de in pthread_start_thread_event () from /lib/i686/libpthread.so.0 #22 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 6 (Thread 65541 (LWP 27993)): #0 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 #1 0x4009bd08 in __pthread_wait_for_restart_signal () from /lib/i686/libpthread.so.0 #2 0x4009e8bd in __pthread_alt_lock () from /lib/i686/libpthread.so.0 #3 0x4009b8b5 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #4 0x081196da in QueueUserAPC (apc_callback=0xfffffffc, handle=0x40a, param=0xfffffffc) at handles-private.h:254 #5 0x080cd69e in mono_thread_request_interruption (running_managed=0) at threads.c:2881 #6 0x08135054 in sigusr1_signal_handler (_dummy=36, info=0x40dbb920, context=0x40dbb9a0) at mini.c:11170 #7 0x400a0149 in __pthread_sighandler_rt () from /lib/i686/libpthread.so.0 #8 <signal handler called> #9 0x08107ab1 in _wapi_lookup_handle (handle=0x40a, type=WAPI_HANDLE_THREAD, handle_specific=0x40dbbcd0) at handles.c:621 #10 0x081088b0 in _wapi_handle_current_thread_set_waiting_on (waiting_on=0x0) at handles-private.h:192 #11 0x08108a3e in timedwait_signal_wait_cond (cond=0x4059902c, mutex=<value optimized out>, timeout=<value optimized out>, alertable=1, waiting_on=0x400) at handles.c:1429 #12 0x0811c26c in WaitForSingleObjectEx (handle=0x400, timeout=10000, alertable=1) at wait.c:202 #13 0x080cf834 in async_invoke_thread (data=0x0) at threadpool.c:1228 #14 0x080cd325 in start_wrapper (data=0x821e240) at threads.c:319 #15 0x0811b0d0 in thread_start_routine (args=0x405eb630) at threads.c:253 #16 0x081318e4 in GC_start_routine () #17 0x4009a021 in pthread_start_thread () from /lib/i686/libpthread.so.0 #18 0x4009a1de in pthread_start_thread_event () from /lib/i686/libpthread.so.0 #19 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 5 (Thread 49156 (LWP 27992)): #0 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 #1 0x4009bd08 in __pthread_wait_for_restart_signal () from /lib/i686/libpthread.so.0 #2 0x4009e8bd in __pthread_alt_lock () from /lib/i686/libpthread.so.0 #3 0x4009b8b5 in pthread_mutex_lock () from /lib/i686/libpthread.so.0 #4 0x08119628 in QueueUserAPC (apc_callback=0xfffffffc, handle=0x409, param=0xfffffffc) at threads.c:1065 #5 0x080cd69e in mono_thread_request_interruption (running_managed=0) at threads.c:2881 #6 0x08135054 in sigusr1_signal_handler (_dummy=36, info=0x40cba418, context=0x40cba498) at mini.c:11170 #7 0x400a0149 in __pthread_sighandler_rt () from /lib/i686/libpthread.so.0 #8 <signal handler called> #9 0x4012f494 in sigprocmask () from /lib/i686/libc.so.6 #10 0x4012f0e9 in siglongjmp () from /lib/i686/libc.so.6 #11 0x4009ba74 in siglongjmp () from /lib/i686/libpthread.so.0 #12 0x4009c8cf in pthread_handle_sigrestart () from /lib/i686/libpthread.so.0 #13 <signal handler called> #14 0x400a0d36 in nanosleep () from /lib/i686/libpthread.so.0 #15 0x00000000 in ?? () #16 0x4009bbe0 in __pthread_timedsuspend_new () from /lib/i686/libpthread.so.0 #17 0x40099323 in pthread_cond_timedwait@GLIBC_2.0 () from /lib/i686/libpthread.so.0 #18 0x08108a27 in timedwait_signal_wait_cond (cond=0x4059902c, mutex=0x40599014, timeout=0x0, alertable=1, waiting_on=0x400) at handles.c:1422 #19 0x0811c26c in WaitForSingleObjectEx (handle=0x400, timeout=10000, alertable=1) at wait.c:202 #20 0x080cf834 in async_invoke_thread (data=0x0) at threadpool.c:1228 #21 0x080cd325 in start_wrapper (data=0x821e258) at threads.c:319 #22 0x0811b0d0 in thread_start_routine (args=0x405eb4f8) at threads.c:253 #23 0x081318e4 in GC_start_routine () #24 0x4009a021 in pthread_start_thread () from /lib/i686/libpthread.so.0 #25 0x4009a1de in pthread_start_thread_event () from /lib/i686/libpthread.so.0 #26 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 4 (Thread 32771 (LWP 27991)): #0 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 #1 0x4009bd08 in __pthread_wait_for_restart_signal () from /lib/i686/libpthread.so.0 #2 0x400995f0 in pthread_cond_wait@GLIBC_2.0 () from /lib/i686/libpthread.so.0 #3 0x08108a5f in timedwait_signal_wait_cond (cond=0x405991dc, mutex=0x405991c4, timeout=0x0, alertable=0, waiting_on=0x404) at handles.c:1424 #4 0x08108c0a in _wapi_handle_wait_signal_handle (handle=0xfffffffc, alertable=-4) at handles.c:1486 #5 0x0811c341 in WaitForSingleObjectEx (handle=0x404, timeout=4294967295, alertable=0) at wait.c:200 #6 0x080ada6b in finalizer_thread (unused=0x0) at gc.c:835 #7 0x080cd325 in start_wrapper (data=0x822bc10) at threads.c:319 #8 0x0811b0d0 in thread_start_routine (args=0x405eb3c0) at threads.c:253 #9 0x081318e4 in GC_start_routine () #10 0x4009a021 in pthread_start_thread () from /lib/i686/libpthread.so.0 #11 0x4009a1de in pthread_start_thread_event () from /lib/i686/libpthread.so.0 #12 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 3 (Thread 16386 (LWP 27990)): #0 0x400a0d36 in nanosleep () from /lib/i686/libpthread.so.0 #1 0x00000000 in ?? () #2 0x0810586e in collection_thread (unused=0x0) at collection.c:34 #3 0x4009a021 in pthread_start_thread () from /lib/i686/libpthread.so.0 #4 0x4009a1de in pthread_start_thread_event () from /lib/i686/libpthread.so.0 #5 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 2 (Thread 32769 (LWP 27989)): #0 0x401b526a in poll () from /lib/i686/libc.so.6 #1 0x4009a6f0 in __pthread_manager () from /lib/i686/libpthread.so.0 #2 0x4009b0d3 in __pthread_manager_event () from /lib/i686/libpthread.so.0 #3 0x401bdf5a in clone () from /lib/i686/libc.so.6 Thread 1 (Thread 16384 (LWP 27988)): #0 0x400a0d36 in nanosleep () from /lib/i686/libpthread.so.0 #1 0x00000000 in ?? () #2 0x08108702 in timedwait_signal_poll (handle=0xffffffff, timeout=0x0) at handles-private.h:407 #3 0x0811c95a in WaitForMultipleObjectsEx (numobjects=4, handles=0x8287408, waitall=1, timeout=4294967295, alertable=0) at wait.c:679 #4 0x080cbd30 in wait_for_tids (wait=0x8287408, timeout=<value optimized out>) at threads.c:1972 #5 0x080cc160 in mono_thread_manage () at threads.c:2185 #6 0x08058eaa in mono_main (argc=1, argv=0xbfffd904) at driver.c:1029 #7 0x08057f8e in main (argc=-4, argv=0xfffffffc) at main.c:6 #0 0x4009d224 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0 ---- Additional Comments From lupus@ximian.com 2007-07-12 12:29:35 MST ---- Other runs just hang with the following traces (note info threads doesn't report anything): Starting program: /tmp/scratch/BUILD/mono-81872/mono/mini/mono /tmp/delegate1.exe Test.F from delegate: 3 Test.F from delegate: 1 Test.F from delegate: 4 Test.F from delegate: 2 Async Callback STATE1 Async Callback STATE2 Async Callback STATE3 Result = 1 Async Callback STATE4 cant execute EndInvoke twice ... OK completed1: True completed2: True completed3: True completed4: True Program received signal SIGINT, Interrupt. 0x400a0d36 in ?? () (gdb) bt #0 0x400a0d36 in ?? () #1 0x00000001 in ?? () #2 0x0810b743 in _wapi_handle_spin (ms=100) at handles-private.h:407 #3 0x0810a433 in timedwait_signal_poll (handle=0xffffffff, timeout=0x0) at handles.c:1458 #4 0x0810a46d in _wapi_handle_wait_signal (shared=1) at handles.c:1465 #5 0x0811c85a in WaitForMultipleObjectsEx (numobjects=4, handles=0x8287628, waitall=1, timeout=4294967295, alertable=0) at wait.c:679 #6 0x080cc2da in wait_for_tids (wait=0x8287628, timeout=4294967295) at threads.c:1972 #7 0x080cc9ea in mono_thread_manage () at threads.c:2185 #8 0x08058eda in mono_main (argc=1, argv=0xbfffef04) at driver.c:1029 #9 0x08057fbe in main (argc=-4, argv=0xfffffffc) at main.c:6 (gdb) info threads (gdb) As expected setting the MONO_NO_SMP env var makes the race/deadlock go away. Unknown operating system unknown. Setting to default OS "Other".
*** This bug has been marked as a duplicate of bug 379524 ***