Re: [gtk-osx-users] Lion, NSUserDefaults, Threading, Python and Deadlock?

[Richard Procter <richard n procter gmail com>]

Hi John, 

Thanks for the info. As this doesn't look to be a specifically pygtk or gtk-osx related issue I won't clog up the list further until I resolve it one way or the other. 

In the meantime I'll respond to your points below for anyone following this.  

On 10/12/2011, at 9:42 AM, John Ralls wrote:
> 
> The GIL is a python thing, AFAIK, so neither gdk-quartz nor AppKit know anything about it, and neither calls into Python.

That was my understanding, too, but note the stack traces below. Although it's an unthreaded pygtk application, in the 'wqthread' (i.e. non Py_Main()) thread we see a call into PyEval_RestoreThread() during an invocation of [CFXPreferencesPropertyListSource synchronizeInBackgroundWithCompletionBlock:], which, if I am reading it right, is blocking trying to acquire the GIL. 

The trace suggests to me that the preference system is keeping some object alive until the next synchronisation (in a helper thread), whereupon its PyObjC destructor calls into python and blocks on the GIL. 

That said, I would expect the _wrap_gtk_main() call to have released the GIL (via its call to pyg_begin_allow_threads, which releases the GIL via PyEval_SaveThread()), which should be sufficient to prevent the deadlock scenario I outlined. 

> Notifications are to the NSApplicationDelegate, not to an NSUserDefaults object, so that's not likely to be the problem either.

> CFPreferences has always handled its own access serialization, that release note only says that they've made it better.

Right. They go on to say "Additionally, automatic synchronization is now non-blocking in most cases." 

If they introduced a second thread (pthread work queue) for that, it creates possibility for the sort of deadlock (I currently believe) I am seeing on 10.7 and would explain why I am not having any problem with 10.6. 

best, 
Richard. 


  Thread 0x412b      
  User stack:
    21 ??? (in python) [0x2ced9]
      21 Py_Main + 2519 (in Python) [0xcc55b]
        21 PyObject_Call + 46 (in Python) [0x3a270]
          21 ??? (in Python) [0x53c57]
            21 PyEval_EvalCodeEx + 1734 (in Python) [0xa89aa]
              21 PyEval_EvalFrameEx + 12046 (in Python) [0xa6176]
                21 ??? (in Python) [0xa8b97]
                  21 PyEval_EvalCodeEx + 1734 (in Python) [0xa89aa]
                    21 PyEval_EvalFrameEx + 12046 (in Python) [0xa6176]
                      21 ??? (in Python) [0xa8b97]
                        21 PyEval_EvalCodeEx + 1734 (in Python) [0xa89aa]
                          21 PyEval_EvalFrameEx + 6950 (in Python) [0xa4d8e]
                            21 PyEval_EvalCode + 87 (in Python) [0xa8a5b]
                              21 PyEval_EvalCodeEx + 1734 (in Python) [0xa89aa]
                                21 PyEval_EvalFrameEx + 11348 (in Python) [0xa5ebc]
                                  21 _wrap_gtk_main + 119 (in _gtk.so) [0x7f693d]
                                    21 gtk_main + 172 (in libgtk-quartz-2.0.0.dylib) [0x9fe932]
                                      21 g_main_loop_run + 605 (in libglib-2.0.0.dylib) [0x42a3e4]
                                        21 g_main_context_iterate + 954 (in libglib-2.0.0.dylib) [0x42a0bd]
                                          21 poll_func + 1091 (in libgdk-quartz-2.0.0.dylib) [0xd4b546]
                                            21 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113 (in AppKit) [0x94ccf8ab]
                                              21 _DPSNextEvent + 678 (in AppKit) [0x94cd0040]
                                                21 BlockUntilNextEventMatchingListInMode + 88 (in HIToolbox) [0x97088c0a]
                                                  21 ReceiveNextEventCommon + 381 (in HIToolbox) [0x97088d9b]
                                                    21 RunCurrentEventLoopInMode + 318 (in HIToolbox) [0x97081a7f]
                                                      21 CFRunLoopRunInMode + 120 (in CoreFoundation) [0x9a566798]
                                                        21 CFRunLoopRunSpecific + 332 (in CoreFoundation) [0x9a5668ec]
                                                          21 __CFRunLoopRun + 2003 (in CoreFoundation) [0x9a567453]
                                                            21 _dispatch_main_queue_callback_4CF + 216 (in libdispatch.dylib) [0x98c51d75]
                                                              21 _dispatch_queue_invoke + 61 (in libdispatch.dylib) [0x98c5164a]
                                                                21 _dispatch_source_invoke + 672 (in libdispatch.dylib) [0x98c54c6c]
                                                                  21 ____NSHysteresisBlock_block_invoke_1 + 252 (in AppKit) [0x94d2e0f6]
                                                                    21 __-[NSPersistentUIManager acquireDirtyState]_block_invoke_1 + 215 (in AppKit) [0x94d2e1d5]
                                                                      21 -[NSPersistentUIManager flushPersistentStateAndClose:waitingUntilDone:] + 201 (in AppKit) [0x94d2e2e3]
                                                                        21 -[NSPersistentUIManager flushAllChangesOptionallyWaitingUntilDone:updatingSnapshots:] + 350 (in AppKit) [0x94d2e46b]
                                                                          21 _NSGetBoolAppConfig + 118 (in AppKit) [0x94cc98fa]
                                                                            21 -[NSUserDefaults(NSUserDefaults) objectForKey:] + 36 (in Foundation) [0x9a92b02a]
                                                                              21 CFPreferencesCopyAppValue + 153 (in CoreFoundation) [0x9a583fc9]
                                                                                21 __psynch_rw_rdlock + 10 (in libsystem_kernel.dylib) [0x98aa98ca]
  Kernel stack:
    21 hndl_unix_scall + 281 (in mach_kernel) [0xffffff80002d7f39]
      21 unix_syscall + 472 (in mach_kernel) [0xffffff80005ca468]
        21 psynch_rw_rdlock + 1739 (in mach_kernel) [0xffffff800059e60b]
          21 ksyn_block_thread_locked + 67 (in mach_kernel) [0xffffff800059a7f3]
            21 thread_block_reason + 299 (in mach_kernel) [0xffffff800022f39b]
              21 thread_continue + 1661 (in mach_kernel) [0xffffff800022f11d]
                21 machine_switch_context + 361 (in mach_kernel) [0xffffff80002c0939]

  Thread 0x4140       DispatchQueue 1701273966
  User stack:
    21 _dispatch_mgr_thread + 53 (in libdispatch.dylib) [0x98c506a7]
      21 kevent + 10 (in libsystem_kernel.dylib) [0x98aaa90a]
  Kernel stack:
    21 kqueue_scan + 416 (in mach_kernel) [0xffffff8000537d40]

  Thread 0x415b       DispatchQueue 1853444896
  User stack:
    21 start_wqthread + 30 (in libsystem_c.dylib) [0x97df06fe]
      21 _pthread_wqthread + 346 (in libsystem_c.dylib) [0x97deeb24]
        21 _dispatch_worker_thread2 + 187 (in libdispatch.dylib) [0x98c50e44]
          21 _dispatch_queue_invoke + 47 (in libdispatch.dylib) [0x98c5163c]
            21 _dispatch_queue_drain + 224 (in libdispatch.dylib) [0x98c51797]
              21 _dispatch_call_block_and_release + 15 (in libdispatch.dylib) [0x98c4fe11]
                21 __-[CFXPreferencesPropertyListSource synchronizeInBackgroundWithCompletionBlock:]_block_invoke_1 + 191 (in CoreFoundation) [0x9a5bb8cf]
                  21 -[CFXPreferencesPropertyListSource _assimilateSync:] + 79 (in CoreFoundation) [0x9a578c6f]
                    21 CFRelease + 577 (in CoreFoundation) [0x9a5348e1]
                      21 __CFBasicHashDrain + 528 (in CoreFoundation) [0x9a538ad0]
                        21 __CFDictionaryStandardReleaseKey + 79 (in CoreFoundation) [0x9a542c7f]
                          21 CFRelease + 169 (in CoreFoundation) [0x9a534749]
                            21 ??? (in _objc.so) [0x26da523]
                              21 PyGILState_Ensure + 109 (in Python) [0xbfbae]
                                21 PyEval_RestoreThread + 73 (in Python) [0xa8c5e]
                                  21 PyThread_acquire_lock + 119 (in Python) [0xca6e4]
                                    21 pthread_cond_wait$UNIX2003 + 71 (in libsystem_c.dylib) [0x97da142c]
                                      21 __psynch_cvwait + 10 (in libsystem_kernel.dylib) [0x98aa983e]
  Kernel stack:
    21 hndl_unix_scall + 281 (in mach_kernel) [0xffffff80002d7f39]
      21 unix_syscall + 472 (in mach_kernel) [0xffffff80005ca468]
        21 psynch_cvwait + 1913 (in mach_kernel) [0xffffff800059f169]
          21 ksyn_block_thread_locked + 67 (in mach_kernel) [0xffffff800059a7f3]
            21 thread_block_reason + 299 (in mach_kernel) [0xffffff800022f39b]
              21 thread_continue + 1661 (in mach_kernel) [0xffffff800022f11d]
                21 machine_switch_context + 361 (in mach_kernel) [0xffffff80002c0939]