[svn-commits] lmadsen: branch 1.6.1 r257429 - /branches/1.6.1/doc/backtrace.txt

SVN commits to the Digium repositories svn-commits at lists.digium.com
Thu Apr 15 14:43:19 CDT 2010


Author: lmadsen
Date: Thu Apr 15 14:43:17 2010
New Revision: 257429

URL: http://svnview.digium.com/svn/asterisk?view=rev&rev=257429
Log:
Merged revisions 257427 via svnmerge from 
https://origsvn.digium.com/svn/asterisk/trunk

................
  r257427 | lmadsen | 2010-04-15 14:41:05 -0500 (Thu, 15 Apr 2010) | 21 lines
  
  Merged revisions 257426 via svnmerge from 
  https://origsvn.digium.com/svn/asterisk/branches/1.4
  
  ........
    r257426 | lmadsen | 2010-04-15 14:40:33 -0500 (Thu, 15 Apr 2010) | 13 lines
    
    Update backtrace.txt documentation.
    
    Update the backtrace.txt documentation so it conforms to the same layout as
    other documents we've been working on recently. Additionally, add a bunch of
    new information about gathering backtraces for crashes and deadlocks, along
    with ways of verifying your file before uploading it. Create a couple of one
    line commands for people to generate the files we need.
    
    (closes issue #17190)
    Reported by: lmadsen
    Patches: 
          backtrace.txt.patch-2 uploaded by lmadsen (license 10)
    Tested by: lmadsen, pabelanger
  ........
................

Modified:
    branches/1.6.1/doc/backtrace.txt

Modified: branches/1.6.1/doc/backtrace.txt
URL: http://svnview.digium.com/svn/asterisk/branches/1.6.1/doc/backtrace.txt?view=diff&rev=257429&r1=257428&r2=257429
==============================================================================
--- branches/1.6.1/doc/backtrace.txt (original)
+++ branches/1.6.1/doc/backtrace.txt Thu Apr 15 14:43:17 2010
@@ -1,12 +1,25 @@
+===============================================================================
+===
+=== Collecting Backtrace Information
+===
+=== Last updated: 2010-04-12
+===============================================================================
+
 This document is intended to provide information on how to obtain the
 backtraces required on the asterisk bug tracker, available at
-http://issues.asterisk.org. 
-
-The information is required by developers to
-help fix problem with bugs of any kind. Backtraces provide information
-about what was wrong when a program crashed; in our case,
-Asterisk. There are two kind of backtraces (aka 'bt') which are
-useful: bt and bt full.
+https://issues.asterisk.org. 
+
+-------------------------------------------------------------------------------
+--- Overview
+-------------------------------------------------------------------------------
+
+The backtrace information is required by developers to help fix problem with 
+bugs of any kind. Backtraces provide information about what was wrong when a 
+program crashed; in our case, Asterisk. 
+
+-------------------------------------------------------------------------------
+--- Preparing Asterisk To Produce Core Files On Crash
+-------------------------------------------------------------------------------
 
 First of all, when you start Asterisk, you MUST start it with option
 -g. This tells Asterisk to produce a core file if it crashes.
@@ -28,6 +41,10 @@
 optimization or the backtrace will be (nearly) unusable. This can be
 done by selecting the 'DONT_OPTIMIZE' option in the Compiler Flags
 submenu in the 'make menuselect' tree before building Asterisk.
+
+Running a production server with DONT_OPTIMIZE is generally safe.
+You'll notice the binary files may be a bit larger, but in terms of
+Asterisk performance, and impact should be negligible.
 
 After Asterisk crashes, a core file will be "dumped" in your /tmp/
 directory. To make sure it's really there, you can just type the
@@ -46,11 +63,21 @@
 above example), it is important to look at the file timestamps in
 order to determine which one you really intend to look at.
 
-Now that we've verified the core file has been written to disk, the
-final part is to extract 'bt' from the core file. Core files are
-pretty big, don't be scared, it's normal.
-
-*** NOTE: Don't attach core files on the bug tracker, we only need the bt and bt full. ***
+-------------------------------------------------------------------------------
+--- Getting Information After A Crash
+-------------------------------------------------------------------------------
+
+There are two kind of backtraces (aka 'bt') which are useful: bt and bt full.
+
+Now that we've verified the core file has been written to disk, the final part 
+is to extract 'bt' from the core file. Core files are pretty big, don't be 
+scared, it's normal.
+
+******************************************************************************
+*** NOTE: Don't attach core files on the bug tracker as they are only useful *
+***       on the machine they were generated on. We only need the output of  *
+***       the 'bt' and 'bt full.'                                            *
+******************************************************************************
 
 For extraction, we use a really nice tool, called gdb. To verify that
 you have gdb installed on your system:
@@ -65,11 +92,25 @@
 This GDB was configured as "i386-linux".
 debian:/tmp#
 
-Which is great, we can continue. If you don't have gdb installed, go install gdb.
-
-Now load the core file in gdb, as follows:
-
-debian:/tmp# gdb asterisk /tmp/core.26252
+If you don't have gdb installed, go install gdb. You should be able to install
+using something like: apt-get install gdb --or-- yum install gdb
+
+Now load the core file in gdb with the following command. This will also save
+the output of gdb to the /tmp/backtract.txt file.
+
+# gdb -se "asterisk" -c /tmp/core.26252 | tee /tmp/backtrace.txt
+
+******************************************************************************
+*** TIP!
+***     Just run the following command to get the output into the
+***     backtrace.txt file, ready for uploading to the issue tracker. Be sure
+***     to change the name of the core file to your actual core dump file:
+***
+*** gdb -se "asterisk" -ex "bt full" -ex "thread apply all bt" --batch -c /tmp/core.26252 > /tmp/backtrace.txt
+***
+******************************************************************************
+
+
 [...]
 (You would see a lot of output here.)
 [...]
@@ -85,6 +126,7 @@
 
 Now at the gdb prompt, type: bt
 You would see output similar to:
+
 (gdb) bt
 #0  0x29b45d7e in ?? ()
 #1  0x08180bf8 in ?? ()
@@ -110,6 +152,7 @@
 The bt's output is the information that we need on the bug tracker.
 
 Now do a bt full as follows:
+
 (gdb) bt full
 #0  0x29b45d7e in ?? ()
 No symbol table info available.
@@ -150,13 +193,9 @@
 No symbol table info available.
 (gdb)
 
-We also need gdb's output. That output gives more details compared to
-the simple "bt". So we recommend that you use bt full instead of bt.
-But, if you could include both, we appreciate that.
-
-The final "extraction" would be to know all traces by all
-threads. Even if asterisk runs on the same thread for each call, it
-could have created some new threads.
+The final "extraction" would be to know all traces by all threads. Even if 
+Asterisk runs on the same thread for each call, it could have created some new
+threads.
 
 To make sure we have the correct information, just do:
 (gdb) thread apply all bt
@@ -185,15 +224,54 @@
 
 That output tells us crucial information about each thread.
 
-Now, if you turned on logging upload your gdb.txt file. Otherwise,
-create an output.txt file and dump your "bt full" (and/or "bt")
-ALONG WITH "thread apply all bt" into it.
-
-Note: Please ATTACH your output, DO NOT paste it as a note.
-
-And you're ready for upload on the bug tracker.
-
-
-If you have questions or comments regarding this documentation, feel
-free to pass by the #asterisk-bugs channel on irc.freenode.net.
-
+-------------------------------------------------------------------------------
+--- Getting Information For A Deadlock
+-------------------------------------------------------------------------------
+
+Whenever supplying information about a deadlock (i.e. when you run the
+'core show locks' command on the Asterisk console), it is useful to also have
+additional information about the threads. We can generate this information by
+attaching to a running Asterisk process and gathering that information.
+
+You can easily attach to a running Asterisk process, gather the output required
+and then detach from the process all in a single step. Execute the following
+command and upload the resulting backtrace-threads.txt file to the Asterisk
+issue tracker:
+
+   gdb -ex "thread apply all bt" --batch /usr/sbin/asterisk `pidof asterisk` > /tmp/backtrace-threads.txt
+
+Note that this gathers information from the running Asterisk process, so you
+want to make sure you run this command immediately before or after gathering
+the output of 'core show locks'. You can gather that information by running the
+following command:
+
+   asterisk -rx "core show locks" > /tmp/core-show-locks.txt
+
+-------------------------------------------------------------------------------
+--- Verify Your Backtraces
+-------------------------------------------------------------------------------
+
+Before uploading your backtraces to the issue tracker, you should double check
+to make sure the data you have is of use to the developers. Check your
+backtrace files to make sure you're not seeing several of the following:
+
+   <value optimized out>
+
+If you are, then you likely haven't compiled with DONT_OPTIMIZE. The impact of
+DONT_OPTIMIZE is negligible on most systems. Be sure you've enabled the
+DONT_OPTIMIZE flag within the Compiler Flags section of menuselect. After
+doing so, be sure to run 'make install' and restart Asterisk.
+
+-------------------------------------------------------------------------------
+--- Uploading Your Information To The Issue Tracker
+-------------------------------------------------------------------------------
+
+You're now ready to upload your files to the Asterisk issue tracker (located at
+https://issues.asterisk.org).
+
+******************************************************************************
+*** NOTE: Please ATTACH your output! DO NOT paste it as a note!              *
+******************************************************************************
+
+If you have questions or comments regarding this documentation, feel free to
+pass by the #asterisk-bugs channel on irc.freenode.net.




More information about the svn-commits mailing list