Hi all,

I have an issue with my daemon hanging when left overnight.(Daemon
undergoing stress testing at the moment). I am unable to narrow down the
issue.

I have taken the sample of my daemon when the hang happened and below is
log. One of the thread did try using curl_easy_perform post which ended up
with OSSpinLockunlock.

I was not able to gather more information from it as to what might have
gone wrong. I went through the Multi-threading issue and have set
CURLOPT_NOSIGNAL option to .

Can any one please help me out

thanks in advance.

Analysis of sampling sefiltd (pid 161) every 1 millisecond
Process: sefiltd [161]
Path: /usr/sbin/sefiltd
Load Address: 0x1000
Identifier: sefiltd
Version: ??? (???)
Code Type: X86 (Native)
Parent Process: launchd [1]

Date/Time: 2013-05-24 11:29:30.446 +0530
OS Version: Mac OS X 10.8.3 (12D78)
Report Version: 7

Call graph:
    2663 Thread_1126 DispatchQueue_1: com.apple.main-thread (serial)
    + 2663 ??? (in sefiltd) load address 0x1000 + 0x2469 [0x3469]
    + 2663 ??? (in sefiltd) load address 0x1000 + 0x2a00 [0x3a00]
    + 2620 ??? (in sefiltd) load address 0x1000 + 0x1053b [0x1153b]
    + ! 2619 usleep$UNIX2003 (in libsystem_c.dylib) + 60 [0x910ca91e]
    + ! : 2618 nanosleep$UNIX2003 (in libsystem_c.dylib) + 189
[0x910caa55]
    + ! : | 2618 __semwait_signal (in libsystem_kernel.dylib) + 10
[0x943e9c72]
    + ! : 1 cerror_nocancel (in libsystem_kernel.dylib) + 52
[0x943eb4b3]
    + ! : 1 cthread_set_errno_self (in libsystem_c.dylib) + 1
[0x910484e4]
    + ! 1 usleep$UNIX2003 (in libsystem_c.dylib) + 0 [0x910ca8e2]
    + 39 ??? (in sefiltd) load address 0x1000 + 0x10687 [0x11687]
    + ! 32 ??? (in sefiltd) load address 0x1000 + 0xf8be [0x108be]
    + ! : 32 __sysctl (in libsystem_kernel.dylib) + 10 [0x943e9fce]
    + ! 7 ??? (in sefiltd) load address 0x1000 + 0xf8fb [0x108fb]
    + ! 7 __sysctl (in libsystem_kernel.dylib) + 10 [0x943e9fce]
    + 3 ??? (in sefiltd) load address 0x1000 + 0x10519 [0x11519]
    + ! 2 ??? (in sefiltd) load address 0x1000 + 0x271d2 [0x281d2]
    + ! : 2 ??? (in sefiltd) load address 0x1000 + 0x1c561 [0x1d561]
    + ! : 2 ??? (in sefiltd) load address 0x1000 + 0x1c507
[0x1d507]
    + ! : 2 send$UNIX2003 (in libsystem_c.dylib) + 54 [0x910cabfa]
    + ! : 2 __sendto (in libsystem_kernel.dylib) + 10
[0x943e9ce2]
    + ! 1 ??? (in sefiltd) load address 0x1000 + 0x2721c [0x2821c]
    + ! 1 ??? (in sefiltd) load address 0x1000 + 0x236d9 [0x246d9]
    + ! 1 std::basic_string<char, std::char_traits<char>,
std::allocator<char> >::basic_string(char const*, std::allocator<char>
const&) (in libstdc++.6.dylib) + 28 [0x99b9eb2c]
    + ! 1 strlen (in libsystem_c.dylib) + 20 [0x9102c644]
    + 1 ??? (in sefiltd) load address 0x1000 + 0x100af [0x110af]
    2663 Thread_1128 DispatchQueue_2: com.apple.libdispatch-manager
(serial)
    + 2663 _dispatch_mgr_thread (in libdispatch.dylib) + 53 [0x93d7e7a9]
    + 2663 _dispatch_mgr_invoke (in libdispatch.dylib) + 993
[0x93d7ec71]
    + 2663 kevent (in libsystem_kernel.dylib) + 10 [0x943ea9ae]
    2663 Thread_1132
    + 2663 thread_start (in libsystem_c.dylib) + 34 [0x9102ad4e]
    + 2663 _pthread_start (in libsystem_c.dylib) + 344 [0x910405b7]
    + 2663 ??? (in sefiltd) load address 0x1000 + 0x10c71 [0x11c71]
    + 2663 ??? (in sefiltd) load address 0x1000 + 0x37bf [0x47bf]
    + 2663 pthread_cond_wait$UNIX2003 (in libsystem_c.dylib) + 71
[0x910cb095]
    + 2663 _pthread_cond_wait (in libsystem_c.dylib) + 833
[0x91045280]
    + 2663 __psynch_cvwait (in libsystem_kernel.dylib) + 10
[0x943e98e2]
    2663 Thread_1133
    + 2663 thread_start (in libsystem_c.dylib) + 34 [0x9102ad4e]
    + 2663 _pthread_start (in libsystem_c.dylib) + 344 [0x910405b7]
    + 2663 ??? (in sefiltd) load address 0x1000 + 0x10c71 [0x11c71]
    + 2051 ??? (in sefiltd) load address 0x1000 + 0x394f [0x494f]
    + ! 2051 pthread_cond_wait$UNIX2003 (in libsystem_c.dylib) + 71
[0x910cb095]
    + ! 2051 _pthread_cond_wait (in libsystem_c.dylib) + 833
[0x91045280]
    + ! 2051 __psynch_cvwait (in libsystem_kernel.dylib) + 10
[0x943e98e2]
    + 612 ??? (in sefiltd) load address 0x1000 + 0x3a86 [0x4a86]
    + 612 ??? (in sefiltd) load address 0x1000 + 0x448e8
[0x458e8]
    + 389 ??? (in sefiltd) load address 0x1000 + 0x44525
[0x45525]
    + : 388 ??? (in sefiltd) load address 0x1000 + 0x443ce
[0x453ce]
    + : | 388 ??? (in sefiltd) load address 0x1000 + 0x43b43
[0x44b43]
    + : | 388 __select (in libsystem_kernel.dylib) + 10
[0x943e9be6]
    + : 1 ??? (in sefiltd) load address 0x1000 + 0x443b6
[0x453b6]
    + : 1 ??? (in sefiltd) load address 0x1000 + 0x12075
[0x13075]
    + : 1 ??? (in sefiltd) load address 0x1000 + 0x11a48
[0x12a48]
    + : 1 strftime$UNIX2003 (in libsystem_c.dylib) + 74
[0x910b1af2]
    + : 1 strftime_l$UNIX2003 (in libsystem_c.dylib) +
51 [0x910b09c7]
    + : 1 tzset (in libsystem_c.dylib) + 18
[0x910af327]
    + : 1 tzsetwall_basic (in libsystem_c.dylib) +
242 [0x910ae2b1]
    + : 1 tzload (in libsystem_c.dylib) + 345
[0x910ae82e]
    + : 1 access (in libsystem_kernel.dylib) +
10 [0x943ea15e]
    + 223 ??? (in sefiltd) load address 0x1000 + 0x444fb
[0x454fb]
    + 223 ??? (in sefiltd) load address 0x1000 + 0x44018
[0x45018]
    + 222 ??? (in sefiltd) load address 0x1000 + 0x43c7d
[0x44c7d]
    + | 222 __select (in libsystem_kernel.dylib) + 10
[0x943e9be6]
    + 1 ??? (in sefiltd) load address 0x1000 + 0x43d2e
[0x44d2e]
    + 1 ??? (in sefiltd) load address 0x1000 + 0x11c1c
[0x12c1c]
    + 1 ??? (in sefiltd) load address 0x1000 + 0x11a29
[0x12a29]
    + 1 localtime (in libsystem_c.dylib) + 231
[0x910af8f5]
    + 1 tzsetwall_basic (in libsystem_c.dylib) +
242 [0x910ae2b1]
    + 1 tzload (in libsystem_c.dylib) + 12
[0x910ae6e1]
    2663 Thread_1134
      2663 thread_start (in libsystem_c.dylib) + 34 [0x9102ad4e]
        2663 _pthread_start (in libsystem_c.dylib) + 344 [0x910405b7]
          2663 ??? (in sefiltd) load address 0x1000 + 0x10c71 [0x11c71]
            2663 ??? (in sefiltd) load address 0x1000 + 0xf850 [0x10850]
              2663 ??? (in sefiltd) load address 0x1000 + 0xf594
[0x10594]
                2663 ??? (in sefiltd) load address 0x1000 + 0x9a35
[0xaa35]
                  2663 ??? (in sefiltd) load address 0x1000 + 0x2b992
[0x2c992]
                    2663 ??? (in sefiltd) load address 0x1000 + 0x2b49d
[0x2c49d]
                      2663 ??? (in sefiltd) load address 0x1000 +
0x31a8e [0x32a8e]
                        2663 ??? (in sefiltd) load address 0x1000 +
0x2d7ab [0x2e7ab]
                          2663 Curl_do_perform (in libcurl.4.dylib) + 153
[0x9009e802]
                            2663 Curl_resolver_wait_resolv (in
libcurl.4.dylib) + 35 [0x900b57d0]
                              2663 Curl_thread_join (in libcurl.4.dylib) +
28 [0x900b4691]
                                2663 pthread_join$UNIX2003 (in
libsystem_c.dylib) + 87 [0x910cac56]
                                  2663 _pthread_lookup_thread (in
libsystem_c.dylib) + 65 [0x91040d95]
                                    2315 _pthread_find_thread (in
libsystem_c.dylib) + 92 [0x91040f6d]
                                    ! 2315 swtch_pri (in
libsystem_kernel.dylib) + 10,0 [0x943e78aa,0x943e78a0]
                                    264 _pthread_find_thread (in
libsystem_c.dylib) + 98,92,... [0x91040f73,0x91040f6d,...]
                                    62 _spin_lock$VARIANT$mp (in
libsystem_c.dylib) + 13,4,... [0x9102d3bd,0x9102d3b4,...]
                                    7 DYLD-STUB$$_spin_lock (in
libsystem_c.dylib) + 0 [0x910dad9c]
                                    6 DYLD-STUB$$swtch_pri (in
libsystem_c.dylib) + 0 [0x910dab9e]
                                    5 _sysenter_trap (in
libsystem_kernel.dylib) + 0 [0x943eb4c2]
                                    3 DYLD-STUB$$_spin_unlock (in
libsystem_c.dylib) + 0 [0x910dadb4]
                                    1 _pthread_find_thread (in
libsystem_c.dylib) + 80 [0x91040f61]
                                      1 OSSpinLockUnlock (in
libsystem_c.dylib) + 0 [0x9102d3e4]

Total number in stack (recursive counted multiple, when >=5):

Sort by top of stack, same collapsed (when >= 5):
        __psynch_cvwait (in libsystem_kernel.dylib) 4714
        kevent (in libsystem_kernel.dylib) 2663
        __semwait_signal (in libsystem_kernel.dylib) 2618
        swtch_pri (in libsystem_kernel.dylib) 2315
        __select (in libsystem_kernel.dylib) 610
        _pthread_find_thread (in libsystem_c.dylib) 264
        _spin_lock$VARIANT$mp (in libsystem_c.dylib) 62
        __sysctl (in libsystem_kernel.dylib) 39
        DYLD-STUB$$_spin_lock (in libsystem_c.dylib) 7
        DYLD-STUB$$swtch_pri (in libsystem_c.dylib) 6
        _sysenter_trap (in libsystem_kernel.dylib) 5

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2013-05-24