<p>George Joseph has uploaded this change for <strong>review</strong>.</p><p><a href="https://gerrit.asterisk.org/8388">View Change</a></p><pre style="font-family: monospace,monospace; white-space: pre-wrap;">ast_coredumper: Minor fixes<br><br>* Fix --tarball-config so the option doesn't cause an error.<br><br>* Allow for missing /etc/os-release.<br><br>* Add a sleep between tarballing the coredump and removing the<br> output directory to allow the filesystem to settle.<br><br>Change-Id: I73e03b13087978bcc7f6bc9f45753990f82d9d77<br>---<br>M contrib/scripts/ast_coredumper<br>1 file changed, 131 insertions(+), 38 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">git pull ssh://gerrit.asterisk.org:29418/asterisk refs/changes/88/8388/1</pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;">diff --git a/contrib/scripts/ast_coredumper b/contrib/scripts/ast_coredumper<br>index 81e94e9..38d95cc 100755<br>--- a/contrib/scripts/ast_coredumper<br>+++ b/contrib/scripts/ast_coredumper<br>@@ -15,8 +15,9 @@<br> $prog [ --help ] [ --running | --RUNNING ] [ --latest ]<br> [ --tarball-coredumps ] [ --delete-coredumps-after ]<br> [ --tarball-results ] [ --delete-results-after ]<br>- [ --tarball-uniqueid="<uniqueid>" ]<br>+ [ --tarball-config ] [ --tarball-uniqueid="<uniqueid>" ]<br> [ --no-default-search ] [ --append-coredumps ]<br>+ [ --asterisk-bin="path" ]<br> [ <coredump> | <pattern> ... ]<br> <br> DESCRIPTION<br>@@ -50,7 +51,8 @@<br> Create a coredump from the running asterisk instance and<br> process it along with any other coredumps found (if any).<br> WARNING: This WILL interrupt call processing. You will be<br>- asked to confirm.<br>+ asked to confirm. The coredump will be written to /tmp if<br>+ $OUTPUTDIR is not defined.<br> <br> --RUNNING<br> Same as --running but without the confirmation prompt.<br>@@ -63,9 +65,16 @@<br> from the existing coredumps.<br> <br> --tarball-coredumps<br>- Creates a gzipped tarball of all coredumps processed.<br>- The tarball name will be:<br>- /tmp/asterisk.<timestamp>.coredumps.tar.gz<br>+ Creates a gzipped tarball of coredumps processed, their<br>+ results txt files and copies of /etc/os-release,<br>+ /usr/sbin/asterisk, /usr/lib(64)/libasterisk* and<br>+ /usr/lib(64)/asterisk as those files are needed to properly<br>+ examine the coredump. The file will be named<br>+ $OUTPUTDIR/asterisk.<timestamp>.coredumps.tar.gz or<br>+ $OUTPUTDIR/asterisk-<uniqueid>.coredumps.tar.gz if<br>+ --tarball-uniqueid was specified.<br>+ WARNING: This file could 1gb in size!<br>+ Mutually exclusive with --tartball-results<br> <br> --delete-coredumps-after<br> Deletes all processed coredumps regardless of whether<br>@@ -74,13 +83,18 @@<br> --tarball-results<br> Creates a gzipped tarball of all result files produced.<br> The tarball name will be:<br>- /tmp/asterisk.<timestamp>.results.tar.gz<br>+ $OUTPUTDIR/asterisk.<timestamp>.results.tar.gz<br>+ Mutually exclusive with --tartball-coredumps<br> <br> --delete-results-after<br> Deletes all processed results regardless of whether<br> a tarball was created. It probably doesn't make sense<br> to use this option unless you have also specified<br> --tarball-results.<br>+<br>+ --tarball-config<br>+ Adds the contents of /etc/asterisk to the tarball created<br>+ with --tarball-coredumps or --tarball-results.<br> <br> --tarball-uniqueid="<uniqueid>"<br> Normally DATEFORMAT is used to make the tarballs unique<br>@@ -95,6 +109,10 @@<br> --append-coredumps<br> Append any coredumps specified on the command line to the<br> config file specified ones instead of overriding them.<br>+<br>+ --asterisk-binary<br>+ Path to the asterisk binary. Default: look for asterisk<br>+ in the PATH.<br> <br> <coredump> | <pattern><br> A list of coredumps or coredump search patterns. Unless<br>@@ -118,6 +136,10 @@<br> <br> NOTES<br> You must be root to use $prog.<br>+<br>+ $OUTPUTDIR can be read from the current environment or from the<br>+ ast_debug_tools.conf file described below. If not specified,<br>+ work products are placed in the same directory as the core file.<br> <br> The script relies on not only bash, but also recent GNU date and<br> gdb with python support. *BSD operating systems may require<br>@@ -155,6 +177,12 @@<br> # anyway.<br> COREDUMPS=(/tmp/core[-._]asterisk!(*.txt) /tmp/core[-._]\$(hostname)!(*.txt))<br> <br>+ # The directory to contain output files and work directories.<br>+ # For output from existing core files, the default is the<br>+ # directory that the core file is found in. For core files<br>+ # produced from a running process, the default is /tmp.<br>+ OUTPUTDIR=/some/directory<br>+<br> # Date command for the "running" coredump and tarballs.<br> # DATEFORMAT will be executed to get the timestamp.<br> # Don't put quotes around the format string or they'll be<br>@@ -188,6 +216,7 @@<br> RUNNING=false<br> latest=false<br> tarball_coredumps=false<br>+tarball_config=false<br> delete_coredumps_after=false<br> tarball_results=false<br> delete_results_after=false<br>@@ -214,6 +243,13 @@<br> if [ -z "$GDB" ] ; then<br> echo "No suitable gdb was found in $PATH"<br> exit 1<br>+fi<br>+<br>+if [ -n "$OUTPUTDIR" ] ; then<br>+ if [ ! -d "$OUTPUTDIR" ] ; then<br>+ echo "OUTPUTDIR $OUTPUTDIR doesn't exists or is not a directory"<br>+ exit 1<br>+ fi<br> fi<br> <br> if [ ${#COREDUMPS[@]} -eq 0 ] ; then<br>@@ -246,6 +282,9 @@<br> --tarball-coredumps)<br> tarball_coredumps=true<br> ;;<br>+ --tarball-config)<br>+ tarball_config=true<br>+ ;;<br> --delete-coredumps-after)<br> delete_coredumps_after=true<br> ;;<br>@@ -260,6 +299,9 @@<br> ;;<br> --tarball-uniqueid=*)<br> tarball_uniqueid=${a#*=}<br>+ ;;<br>+ --asterisk-bin=*)<br>+ asterisk_bin=${a#*=}<br> ;;<br> --help|-*)<br> print_help<br>@@ -312,6 +354,10 @@<br> # Timestamp to use for output files<br> df=${tarball_uniqueid:-$(${DATEFORMAT})}<br> <br>+if [ -z "$asterisk_bin" ]; then<br>+ asterisk_bin=$(which asterisk)<br>+fi<br>+<br> if $running || $RUNNING ; then<br> # We need to go through some gyrations to find the pid of the running<br> # MAIN asterisk process and not someone or something running asterisk -r.<br>@@ -337,9 +383,9 @@<br> read -p "WARNING: Taking a core dump of the running asterisk instance will suspend call processing while the dump is saved. Do you wish to continue? (y/N) " answer<br> fi<br> if [[ "$answer" =~ ^[Yy] ]] ; then<br>- cf="/tmp/core-asterisk-running-$df"<br>+ cf="${OUTPUTDIR:-/tmp}/core-asterisk-running-$df"<br> echo "Dumping running asterisk process to $cf"<br>- ${GDB} -p $pid -q --batch --ex "gcore $cf" >/dev/null 2>&1<br>+ ${GDB} ${asterisk_bin} -p $pid -q --batch --ex "gcore $cf" >/dev/null 2>&1<br> COREDUMPS+=("$cf")<br> else<br> echo "Skipping dump of running process"<br>@@ -356,17 +402,22 @@<br> # and save them to /tmp/.gdbinit<br> <br> ss=`egrep -n "^#@@@SCRIPTSTART@@@" $0 |cut -f1 -d:`<br>-tail -n +${ss} $0 >/tmp/.ast_coredumper.gdbinit<br>+tail -n +${ss} $0 >${OUTPUTDIR:-/tmp}/.ast_coredumper.gdbinit<br> <br> # Now iterate over the coredumps and dump the debugging info<br> for i in ${!COREDUMPS[@]} ; do<br> cf=${COREDUMPS[$i]}<br> echo "Processing $cf"<br>- ${GDB} -n --batch -q --ex "source /tmp/.ast_coredumper.gdbinit" $(which asterisk) "$cf" 2>/dev/null | (<br>+<br>+ cfdir=`dirname ${cf}`<br>+ cfname=`basename ${cf}`<br>+ outputdir=${OUTPUTDIR:-${cfdir}}<br>+<br>+ ${GDB} -n --batch -q --ex "source ${OUTPUTDIR:-/tmp}/.ast_coredumper.gdbinit" "$asterisk_bin" "$cf" 2>/dev/null | (<br> of=/dev/null<br> while IFS= read line ; do<br> if [[ "$line" =~ !@!@!@!\ ([^\ ]+)\ !@!@!@! ]] ; then<br>- of=${cf}-${BASH_REMATCH[1]}<br>+ of=${outputdir}/${cfname}-${BASH_REMATCH[1]}<br> of=${of//:/-}<br> rm -f "$of"<br> echo "Creating $of"<br>@@ -374,37 +425,59 @@<br> echo -e $"$line" >> "$of"<br> done<br> )<br>-done<br> <br>-if $tarball_coredumps ; then<br>- tf=/tmp/asterisk-$df.coredumps.tar<br>- echo "Creating $tf.gz"<br>- for i in ${!COREDUMPS[@]} ; do<br>- tar -uvf $tf "${COREDUMPS[@]}" 2>/dev/null<br>- done<br>- gzip $tf<br>-fi<br>+ if $tarball_coredumps ; then<br>+ cfname=${cfname//:/-}<br>+ tf=${outputdir}/${cfname}.tar.gz<br>+ echo "Creating ${tf}"<br>+<br>+ dest=${outputdir}/${cfname}.output<br>+ rm -rf ${dest} 2>/dev/null || :<br>+<br>+ libdir=usr/lib<br>+ [ -d /usr/lib64 ] && libdir+=64<br>+ mkdir -p ${dest}/tmp ${dest}/${libdir}/asterisk ${dest}/etc ${dest}/usr/sbin<br>+<br>+ ln -s ${cf} ${dest}/tmp/${cfname}<br>+ cp ${outputdir}/${cfname}*.txt ${dest}/tmp/<br>+ [ -f /etc/os-release ] && cp /etc/os-release ${dest}/etc/<br>+ if $tarball_config ; then<br>+ cp -a /etc/asterisk ${dest}/etc/<br>+ fi<br>+ cp -a /${libdir}/libasterisk* ${dest}/${libdir}/<br>+ cp -a /${libdir}/asterisk/* ${dest}/${libdir}/asterisk/<br>+ cp -a /usr/sbin/asterisk ${dest}/usr/sbin<br>+ rm -rf ${tf}<br>+ tar -chzf ${tf} --transform="s/^[.]/${cfname}/" -C ${dest} .<br>+ sleep 3<br>+ rm -rf ${dest}<br>+ echo "Created $tf"<br>+ elif $tarball_results ; then<br>+ cfname=${cfname//:/-}<br>+ tf=${outputdir}/${cfname}.tar.gz<br>+ echo "Creating ${tf}"<br>+<br>+ dest=${outputdir}/${cfname}.output<br>+ rm -rf ${dest} 2>/dev/null || :<br>+ mkdir -p ${dest}<br>+ cp ${outputdir}/${cfname}*.txt ${dest}/<br>+ if $tarball_config ; then<br>+ mkdir -p ${dest}/etc<br>+ cp -a /etc/asterisk ${dest}/etc/<br>+ fi<br>+ tar -chzf ${tf} --transform="s/^[.]/${cfname}/" -C ${dest} .<br>+ rm -rf ${dest}<br>+ echo "Created $tf"<br>+ fi<br> <br> if $delete_coredumps_after ; then<br>- for i in ${!COREDUMPS[@]} ; do<br>- rm -rf "${COREDUMPS[$i]}"<br>- done<br>-fi<br>+ rm -rf "${cf}"<br>+ fi<br> <br>-if $tarball_results ; then<br>- tf=/tmp/asterisk-$df-results.tar<br>- echo "Creating $tf.gz"<br>- for i in ${!COREDUMPS[@]} ; do<br>- tar -uvf $tf "${COREDUMPS[$i]//:/-}"-{brief,full,thread1,locks}.txt 2>/dev/null<br>- done<br>- gzip $tf<br>-fi<br>-<br>-if $delete_results_after ; then<br>- for i in ${!COREDUMPS[@]} ; do<br>- rm -rf "${COREDUMPS[$i]//:/-}"-{brief,full,thread1,locks}.txt<br>- done<br>-fi<br>+ if $delete_results_after ; then<br>+ rm -rf "${cf//:/-}"-{brief,full,thread1,locks}.txt<br>+ fi<br>+done<br> <br> exit<br> <br>@@ -431,21 +504,41 @@<br> pass<br> print("!@!@!@! thread1.txt !@!@!@!\n")<br> try:<br>+ gdb.execute("p $_siginfo", from_tty)<br>+ gdb.execute("info signal $_siginfo.si_signo")<br>+ except:<br>+ pass<br>+ try:<br> gdb.execute("thread apply 1 bt full", from_tty)<br> except:<br> pass<br> print("!@!@!@! brief.txt !@!@!@!\n")<br>+ try:<br>+ gdb.execute("p $_siginfo", from_tty)<br>+ gdb.execute("info signal $_siginfo.si_signo")<br>+ except:<br>+ pass<br> try:<br> gdb.execute("thread apply all bt", from_tty)<br> except:<br> pass<br> print("!@!@!@! full.txt !@!@!@!\n")<br> try:<br>+ gdb.execute("p $_siginfo", from_tty)<br>+ gdb.execute("info signal $_siginfo.si_signo")<br>+ except:<br>+ pass<br>+ try:<br> gdb.execute("thread apply all bt full", from_tty)<br> except:<br> pass<br> print("!@!@!@! locks.txt !@!@!@!\n")<br> try:<br>+ gdb.execute("p $_siginfo", from_tty)<br>+ gdb.execute("info signal $_siginfo.si_signo")<br>+ except:<br>+ pass<br>+ try:<br> gdb.execute("show_locks", from_tty)<br> except:<br> pass<br></pre><p>To view, visit <a href="https://gerrit.asterisk.org/8388">change 8388</a>. To unsubscribe, visit <a href="https://gerrit.asterisk.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.asterisk.org/8388"/><meta itemprop="name" content="View Change"/></div></div>
<div style="display:none"> Gerrit-Project: asterisk </div>
<div style="display:none"> Gerrit-Branch: certified/13.18 </div>
<div style="display:none"> Gerrit-MessageType: newchange </div>
<div style="display:none"> Gerrit-Change-Id: I73e03b13087978bcc7f6bc9f45753990f82d9d77 </div>
<div style="display:none"> Gerrit-Change-Number: 8388 </div>
<div style="display:none"> Gerrit-PatchSet: 1 </div>
<div style="display:none"> Gerrit-Owner: George Joseph <gjoseph@digium.com> </div>