<p>George Joseph <strong>merged</strong> this change.</p><p><a href="https://gerrit.asterisk.org/10477">View Change</a></p><div style="white-space:pre-wrap">Approvals:
  Richard Mudgett: Looks good to me, but someone else must approve
  George Joseph: Looks good to me, approved; Approved for Submit

</div><pre style="font-family: monospace,monospace; white-space: pre-wrap;">refdebug: Create refstats.py script.<br><br>This allows us to process AO2 statistics for total objects, memory<br>usage, memory overhead and lock usage.<br><br>* Install refstats.py and reflocks.py into the Asterisk scripts folder.<br>* Enable support for reflocks.py without DEBUG_THREADS.<br><br>Steal a bit from the ao2 magic to flag when an object lock is used.<br>Remove 'lockobj' from reflocks.py since we can now record 'used' or<br>'unused' for those objects.<br><br>Add comments to explain thread safety of the 'struct __priv_data'<br>bitfields.<br><br>Change-Id: I84e9d679cc86d772cc97c888d9d856a17e0d3a4a<br>---<br>M contrib/Makefile<br>M contrib/scripts/refcounter.py<br>M contrib/scripts/reflocks.py<br>A contrib/scripts/refstats.py<br>M main/astobj2.c<br>5 files changed, 243 insertions(+), 47 deletions(-)<br><br></pre><pre style="font-family: monospace,monospace; white-space: pre-wrap;"><span>diff --git a/contrib/Makefile b/contrib/Makefile</span><br><span>index 9512de5..074a12b 100644</span><br><span>--- a/contrib/Makefile</span><br><span>+++ b/contrib/Makefile</span><br><span>@@ -25,9 +25,13 @@</span><br><span>   $(INSTALL) -m 755 scripts/ast_loggrabber "$(DESTDIR)$(ASTDATADIR)/scripts/ast_loggrabber"</span><br><span>  $(INSTALL) -m 755 scripts/ast_coredumper "$(DESTDIR)$(ASTDATADIR)/scripts/ast_coredumper"</span><br><span>  $(INSTALL) -m 755 scripts/refcounter.py "$(DESTDIR)$(ASTDATADIR)/scripts/refcounter.py"</span><br><span style="color: hsl(120, 100%, 40%);">+     $(INSTALL) -m 755 scripts/refstats.py "$(DESTDIR)$(ASTDATADIR)/scripts/refstats.py"</span><br><span style="color: hsl(120, 100%, 40%);">+ $(INSTALL) -m 755 scripts/reflocks.py "$(DESTDIR)$(ASTDATADIR)/scripts/reflocks.py"</span><br><span> </span><br><span> uninstall:</span><br><span>      -rm -f "$(DESTDIR)$(ASTDATADIR)/scripts/ast_logescalator"</span><br><span>  -rm -f "$(DESTDIR)$(ASTDATADIR)/scripts/ast_loggrabber"</span><br><span>    -rm -f "$(DESTDIR)$(ASTDATADIR)/scripts/ast_coredumper"</span><br><span>    -rm -f "$(DESTDIR)$(ASTDATADIR)/scripts/refcounter.py"</span><br><span style="color: hsl(120, 100%, 40%);">+      -rm -f "$(DESTDIR)$(ASTDATADIR)/scripts/refstats.py"</span><br><span style="color: hsl(120, 100%, 40%);">+        -rm -f "$(DESTDIR)$(ASTDATADIR)/scripts/reflocks.py"</span><br><span>diff --git a/contrib/scripts/refcounter.py b/contrib/scripts/refcounter.py</span><br><span>index 36fb341..146a024 100755</span><br><span>--- a/contrib/scripts/refcounter.py</span><br><span>+++ b/contrib/scripts/refcounter.py</span><br><span>@@ -108,8 +108,10 @@</span><br><span>             else:</span><br><span>                 current_objects[obj]['curcount'] += int(parsed_line['delta'])</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-            if 'destructor' in parsed_line['state']:</span><br><span style="color: hsl(0, 100%, 40%);">-                # refcounter.py doesn't care about lock-state.</span><br><span style="color: hsl(120, 100%, 40%);">+            # Suppress object sizes and lock-state from output logs.</span><br><span style="color: hsl(120, 100%, 40%);">+            if 'constructor' in parsed_line['state']:</span><br><span style="color: hsl(120, 100%, 40%);">+                parsed_line['state'] = '**constructor**'</span><br><span style="color: hsl(120, 100%, 40%);">+            elif 'destructor' in parsed_line['state']:</span><br><span>                 parsed_line['state'] = '**destructor**'</span><br><span> </span><br><span>             current_objects[obj]['log'].append(</span><br><span>diff --git a/contrib/scripts/reflocks.py b/contrib/scripts/reflocks.py</span><br><span>index 7d8edfc..db1985f 100755</span><br><span>--- a/contrib/scripts/reflocks.py</span><br><span>+++ b/contrib/scripts/reflocks.py</span><br><span>@@ -48,7 +48,6 @@</span><br><span>                     object_types[obj_type] = {</span><br><span>                         'used': 0,</span><br><span>                         'unused': 0,</span><br><span style="color: hsl(0, 100%, 40%);">-                        'lockobj': 0,</span><br><span>                         'none': 0</span><br><span>                     }</span><br><span>                 objects[addr] = obj_type</span><br><span>@@ -62,8 +61,6 @@</span><br><span>                     object_types[obj_type]['unused'] += 1</span><br><span>                 elif '**lock-state:used**' in state:</span><br><span>                     object_types[obj_type]['used'] += 1</span><br><span style="color: hsl(0, 100%, 40%);">-                elif '**lock-state:lockobj**' in state:</span><br><span style="color: hsl(0, 100%, 40%);">-                    object_types[obj_type]['lockobj'] += 1</span><br><span>                 elif '**lock-state:none**' in state:</span><br><span>                     object_types[obj_type]['none'] += 1</span><br><span> </span><br><span>@@ -75,8 +72,6 @@</span><br><span>             stats.append("%d used" % info['used'])</span><br><span>         if info['unused'] > 0:</span><br><span>             stats.append("%d unused" % info['unused'])</span><br><span style="color: hsl(0, 100%, 40%);">-        if info['lockobj'] > 0 and options.lockobj:</span><br><span style="color: hsl(0, 100%, 40%);">-            stats.append("%d lockobj" % info['lockobj'])</span><br><span>         if info['none'] > 0 and options.none:</span><br><span>             stats.append("%d none" % info['none'])</span><br><span>         if len(stats) == 0:</span><br><span>@@ -103,9 +98,6 @@</span><br><span>     parser.add_option("-n", "--show-none", action="store_true",</span><br><span>                       dest="none", default=False,</span><br><span>                       help="Show counts of objects with no locking.")</span><br><span style="color: hsl(0, 100%, 40%);">-    parser.add_option("-o", "--show-lockobj", action="store_true",</span><br><span style="color: hsl(0, 100%, 40%);">-                      dest="lockobj", default=False,</span><br><span style="color: hsl(0, 100%, 40%);">-                      help="Show counts of objects with a lockobj.")</span><br><span> </span><br><span>     (options, args) = parser.parse_args(argv)</span><br><span> </span><br><span>diff --git a/contrib/scripts/refstats.py b/contrib/scripts/refstats.py</span><br><span>new file mode 100755</span><br><span>index 0000000..93df02f</span><br><span>--- /dev/null</span><br><span>+++ b/contrib/scripts/refstats.py</span><br><span>@@ -0,0 +1,180 @@</span><br><span style="color: hsl(120, 100%, 40%);">+#!/usr/bin/env python</span><br><span style="color: hsl(120, 100%, 40%);">+"""Process a ref debug log for memory usage</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ This will provide information about total and peak</span><br><span style="color: hsl(120, 100%, 40%);">+ allocations.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ See http://www.asterisk.org for more information about</span><br><span style="color: hsl(120, 100%, 40%);">+ the Asterisk project. Please do not directly contact</span><br><span style="color: hsl(120, 100%, 40%);">+ any of the maintainers of this project for assistance;</span><br><span style="color: hsl(120, 100%, 40%);">+ the project provides a web site, mailing lists and IRC</span><br><span style="color: hsl(120, 100%, 40%);">+ channels for your use.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ This program is free software, distributed under the terms of</span><br><span style="color: hsl(120, 100%, 40%);">+ the GNU General Public License Version 2. See the LICENSE file</span><br><span style="color: hsl(120, 100%, 40%);">+ at the top of the source tree.</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+ Copyright (C) 2018, CFWare, LLC</span><br><span style="color: hsl(120, 100%, 40%);">+ Corey Farrell <git@cfware.com></span><br><span style="color: hsl(120, 100%, 40%);">+"""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from __future__ import print_function</span><br><span style="color: hsl(120, 100%, 40%);">+import sys</span><br><span style="color: hsl(120, 100%, 40%);">+import os</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+from optparse import OptionParser</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def create_stats():</span><br><span style="color: hsl(120, 100%, 40%);">+    """Create statistics object"""</span><br><span style="color: hsl(120, 100%, 40%);">+    return {</span><br><span style="color: hsl(120, 100%, 40%);">+        'count': 0,</span><br><span style="color: hsl(120, 100%, 40%);">+        'overhead': 0,</span><br><span style="color: hsl(120, 100%, 40%);">+        'user_data': 0,</span><br><span style="color: hsl(120, 100%, 40%);">+        'totalmem': 0</span><br><span style="color: hsl(120, 100%, 40%);">+    }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def update_stats(current, peak, total, key, direction, delta):</span><br><span style="color: hsl(120, 100%, 40%);">+    """Update statistics objects"""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    if direction == 1:</span><br><span style="color: hsl(120, 100%, 40%);">+        total[key] += delta</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    delta *= direction</span><br><span style="color: hsl(120, 100%, 40%);">+    current[key] += delta</span><br><span style="color: hsl(120, 100%, 40%);">+    if current[key] > peak[key]:</span><br><span style="color: hsl(120, 100%, 40%);">+        peak[key] = current[key]</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def process_file(options):</span><br><span style="color: hsl(120, 100%, 40%);">+    """The routine that kicks off processing a ref file"""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    current = create_stats()</span><br><span style="color: hsl(120, 100%, 40%);">+    total = create_stats()</span><br><span style="color: hsl(120, 100%, 40%);">+    peak = create_stats()</span><br><span style="color: hsl(120, 100%, 40%);">+    object_types = {}</span><br><span style="color: hsl(120, 100%, 40%);">+    objects = {}</span><br><span style="color: hsl(120, 100%, 40%);">+    filename = options.filepath</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    with open(filename, 'r') as ref_file:</span><br><span style="color: hsl(120, 100%, 40%);">+        for line in ref_file:</span><br><span style="color: hsl(120, 100%, 40%);">+            if 'constructor' not in line and 'destructor' not in line:</span><br><span style="color: hsl(120, 100%, 40%);">+                continue</span><br><span style="color: hsl(120, 100%, 40%);">+            # The line format is:</span><br><span style="color: hsl(120, 100%, 40%);">+            # addr,delta,thread_id,file,line,function,state,tag</span><br><span style="color: hsl(120, 100%, 40%);">+            # Only addr, file, line, function, state are used by reflocks.py</span><br><span style="color: hsl(120, 100%, 40%);">+            tokens = line.strip().split(',', 7)</span><br><span style="color: hsl(120, 100%, 40%);">+            addr = tokens[0]</span><br><span style="color: hsl(120, 100%, 40%);">+            state = tokens[6]</span><br><span style="color: hsl(120, 100%, 40%);">+            if 'constructor' in state:</span><br><span style="color: hsl(120, 100%, 40%);">+                split_state = state.split("**")</span><br><span style="color: hsl(120, 100%, 40%);">+                if len(split_state) < 4:</span><br><span style="color: hsl(120, 100%, 40%);">+                    print("File does not contain object size information", file=sys.stderr)</span><br><span style="color: hsl(120, 100%, 40%);">+                    sys.exit(1)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+                obj_type = '%s:%s:%s' % (tokens[3], tokens[4], tokens[5])</span><br><span style="color: hsl(120, 100%, 40%);">+                if obj_type not in object_types:</span><br><span style="color: hsl(120, 100%, 40%);">+                    object_types[obj_type] = {</span><br><span style="color: hsl(120, 100%, 40%);">+                        'used': 0,</span><br><span style="color: hsl(120, 100%, 40%);">+                        'unused': 0,</span><br><span style="color: hsl(120, 100%, 40%);">+                        'none': 0</span><br><span style="color: hsl(120, 100%, 40%);">+                    }</span><br><span style="color: hsl(120, 100%, 40%);">+                overhead = int(split_state[2])</span><br><span style="color: hsl(120, 100%, 40%);">+                user_data = int(split_state[3])</span><br><span style="color: hsl(120, 100%, 40%);">+                obj = objects[addr] = {</span><br><span style="color: hsl(120, 100%, 40%);">+                    'overhead': overhead,</span><br><span style="color: hsl(120, 100%, 40%);">+                    'user_data': user_data,</span><br><span style="color: hsl(120, 100%, 40%);">+                    'obj_type': obj_type</span><br><span style="color: hsl(120, 100%, 40%);">+                }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+                direction = 1</span><br><span style="color: hsl(120, 100%, 40%);">+            else:</span><br><span style="color: hsl(120, 100%, 40%);">+                if addr not in objects:</span><br><span style="color: hsl(120, 100%, 40%);">+                    # This error would be reported by refcounter.py.</span><br><span style="color: hsl(120, 100%, 40%);">+                    continue</span><br><span style="color: hsl(120, 100%, 40%);">+                obj = objects[addr]</span><br><span style="color: hsl(120, 100%, 40%);">+                del objects[addr]</span><br><span style="color: hsl(120, 100%, 40%);">+                direction = -1</span><br><span style="color: hsl(120, 100%, 40%);">+                obj_type = obj['obj_type']</span><br><span style="color: hsl(120, 100%, 40%);">+                if '**lock-state:unused**' in state:</span><br><span style="color: hsl(120, 100%, 40%);">+                    object_types[obj_type]['unused'] += 1</span><br><span style="color: hsl(120, 100%, 40%);">+                elif '**lock-state:used**' in state:</span><br><span style="color: hsl(120, 100%, 40%);">+                    object_types[obj_type]['used'] += 1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+            # Increment current and peak usage</span><br><span style="color: hsl(120, 100%, 40%);">+            update_stats(current, peak, total, 'count', direction, 1)</span><br><span style="color: hsl(120, 100%, 40%);">+            update_stats(current, peak, total, 'overhead', direction, obj['overhead'])</span><br><span style="color: hsl(120, 100%, 40%);">+            update_stats(current, peak, total, 'user_data', direction, obj['user_data'])</span><br><span style="color: hsl(120, 100%, 40%);">+            update_stats(current, peak, total, 'totalmem', direction, obj['overhead'] + obj['user_data'])</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    print("Total usage statistics:")</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d" % ("Count", total['count']))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d" % ("Total Memory (k)", total['totalmem'] / 1024))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d (%.2f%%)" % ("Overhead (k)", total['overhead'] / 1024, total['overhead'] * 100.0 / total['totalmem']))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d" % ("User Data (k)", total['user_data'] / 1024))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("")</span><br><span style="color: hsl(120, 100%, 40%);">+    print("Peak usage statistics:")</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d" % ("Count", peak['count']))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d" % ("Total Memory (k)", peak['totalmem'] / 1024))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d (%.2f%%)" % ("Overhead (k)", peak['overhead'] / 1024, peak['overhead'] * 100.0 / peak['totalmem']))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d" % ("User Data (k)", peak['user_data'] / 1024))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    lockbyobj = {'used': 0, 'total': 0}</span><br><span style="color: hsl(120, 100%, 40%);">+    lockbytype = {'used': 0, 'total': 0}</span><br><span style="color: hsl(120, 100%, 40%);">+    for (allocator, info) in object_types.items():</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbyobj['used'] += info['used']</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbyobj['total'] += info['used'] + info['unused']</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+        if info['used'] != 0:</span><br><span style="color: hsl(120, 100%, 40%);">+            lockbytype['used'] += 1</span><br><span style="color: hsl(120, 100%, 40%);">+        elif info['unused'] == 0:</span><br><span style="color: hsl(120, 100%, 40%);">+            # This object type doesn't have locking.</span><br><span style="color: hsl(120, 100%, 40%);">+            continue</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbytype['total'] += 1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    print("Lock usage statistics:")</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d of %d used (%.2f%%)" % (</span><br><span style="color: hsl(120, 100%, 40%);">+        "By object",</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbyobj['used'],</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbyobj['total'],</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbyobj['used'] * 100.0 / lockbyobj['total']))</span><br><span style="color: hsl(120, 100%, 40%);">+    print("%20s: %d of %d used (%.2f%%)" % (</span><br><span style="color: hsl(120, 100%, 40%);">+        "By type",</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbytype['used'],</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbytype['total'],</span><br><span style="color: hsl(120, 100%, 40%);">+        lockbytype['used'] * 100.0 / lockbytype['total']))</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+def main(argv=None):</span><br><span style="color: hsl(120, 100%, 40%);">+    """Main entry point for the script"""</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    ret_code = 0</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    if argv is None:</span><br><span style="color: hsl(120, 100%, 40%);">+        argv = sys.argv</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    parser = OptionParser()</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    parser.add_option("-f", "--file", action="store", type="string",</span><br><span style="color: hsl(120, 100%, 40%);">+                      dest="filepath", default="/var/log/asterisk/refs",</span><br><span style="color: hsl(120, 100%, 40%);">+                      help="The full path to the refs file to process")</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    (options, args) = parser.parse_args(argv)</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    if not os.path.isfile(options.filepath):</span><br><span style="color: hsl(120, 100%, 40%);">+        print("File not found: %s" % options.filepath, file=sys.stderr)</span><br><span style="color: hsl(120, 100%, 40%);">+        return -1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    try:</span><br><span style="color: hsl(120, 100%, 40%);">+        process_file(options)</span><br><span style="color: hsl(120, 100%, 40%);">+    except (KeyboardInterrupt, SystemExit, IOError):</span><br><span style="color: hsl(120, 100%, 40%);">+        print("File processing cancelled", file=sys.stderr)</span><br><span style="color: hsl(120, 100%, 40%);">+        return -1</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+    return ret_code</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span style="color: hsl(120, 100%, 40%);">+if __name__ == "__main__":</span><br><span style="color: hsl(120, 100%, 40%);">+    sys.exit(main(sys.argv))</span><br><span>diff --git a/main/astobj2.c b/main/astobj2.c</span><br><span>index d6a8a4e..ebc0720 100644</span><br><span>--- a/main/astobj2.c</span><br><span>+++ b/main/astobj2.c</span><br><span>@@ -61,15 +61,40 @@</span><br><span> #endif</span><br><span>   /*! Number of references held for this object */</span><br><span>     int32_t ref_counter;</span><br><span style="color: hsl(0, 100%, 40%);">-    /*! The ao2 object option flags */</span><br><span style="color: hsl(120, 100%, 40%);">+    /*!</span><br><span style="color: hsl(120, 100%, 40%);">+    * \brief The ao2 object option flags.</span><br><span style="color: hsl(120, 100%, 40%);">+         *</span><br><span style="color: hsl(120, 100%, 40%);">+     * \note This field is constant after object creation.  It shares</span><br><span style="color: hsl(120, 100%, 40%);">+      *       a uint32_t with \ref lockused and \ref magic.</span><br><span style="color: hsl(120, 100%, 40%);">+         */</span><br><span>  uint32_t options:2;</span><br><span style="color: hsl(0, 100%, 40%);">-     /*! magic number.  This is used to verify that a pointer passed in is a</span><br><span style="color: hsl(0, 100%, 40%);">-  *  valid astobj2 or ao2_weak reference */</span><br><span style="color: hsl(0, 100%, 40%);">-      uint32_t magic:30;</span><br><span style="color: hsl(120, 100%, 40%);">+    /*!</span><br><span style="color: hsl(120, 100%, 40%);">+    * \brief Set to 1 when the lock is used if refdebug is enabled.</span><br><span style="color: hsl(120, 100%, 40%);">+       *</span><br><span style="color: hsl(120, 100%, 40%);">+     * \note This bit-field may be modified after object creation.  It</span><br><span style="color: hsl(120, 100%, 40%);">+     *       shares a uint32_t with \ref options and \ref magic.</span><br><span style="color: hsl(120, 100%, 40%);">+   */</span><br><span style="color: hsl(120, 100%, 40%);">+   uint32_t lockused:1;</span><br><span style="color: hsl(120, 100%, 40%);">+  /*!</span><br><span style="color: hsl(120, 100%, 40%);">+    * \brief Magic number.</span><br><span style="color: hsl(120, 100%, 40%);">+        *</span><br><span style="color: hsl(120, 100%, 40%);">+     * This is used to verify that a pointer is a valid astobj2 or ao2_weak</span><br><span style="color: hsl(120, 100%, 40%);">+        * reference.</span><br><span style="color: hsl(120, 100%, 40%);">+  *</span><br><span style="color: hsl(120, 100%, 40%);">+     * \note This field is constant after object creation.  It shares</span><br><span style="color: hsl(120, 100%, 40%);">+      *       a uint32_t with \ref options and \ref lockused.</span><br><span style="color: hsl(120, 100%, 40%);">+       *</span><br><span style="color: hsl(120, 100%, 40%);">+     * \warning Stealing bits for any additional writable fields would cause</span><br><span style="color: hsl(120, 100%, 40%);">+       *          reentrancy issues if using bitfields.  If any additional</span><br><span style="color: hsl(120, 100%, 40%);">+   *          writable bits are required in the future we will need to put</span><br><span style="color: hsl(120, 100%, 40%);">+       *          all bitfields into a single 'uint32_t flags' field and use</span><br><span style="color: hsl(120, 100%, 40%);">+         *          atomic operations from \file lock.h to perform writes.</span><br><span style="color: hsl(120, 100%, 40%);">+     */</span><br><span style="color: hsl(120, 100%, 40%);">+   uint32_t magic:29;</span><br><span> };</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-#define     AO2_MAGIC       0x3a70b123</span><br><span style="color: hsl(0, 100%, 40%);">-#define       AO2_WEAK        0x3a70b122</span><br><span style="color: hsl(120, 100%, 40%);">+#define     AO2_MAGIC       0x1a70b123</span><br><span style="color: hsl(120, 100%, 40%);">+#define     AO2_WEAK        0x1a70b122</span><br><span> #define IS_AO2_MAGIC_BAD(p) (AO2_MAGIC != (p->priv_data.magic | 1))</span><br><span> </span><br><span> /*!</span><br><span>@@ -206,6 +231,10 @@</span><br><span>               return -1;</span><br><span>   }</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ if (ref_log) {</span><br><span style="color: hsl(120, 100%, 40%);">+                obj->priv_data.lockused  = 1;</span><br><span style="color: hsl(120, 100%, 40%);">+      }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span>  switch (obj->priv_data.options & AO2_ALLOC_OPT_LOCK_MASK) {</span><br><span>   case AO2_ALLOC_OPT_LOCK_MUTEX:</span><br><span>               obj_mutex = INTERNAL_OBJ_MUTEX(user_data);</span><br><span>@@ -322,6 +351,10 @@</span><br><span>            return -1;</span><br><span>   }</span><br><span> </span><br><span style="color: hsl(120, 100%, 40%);">+ if (ref_log) {</span><br><span style="color: hsl(120, 100%, 40%);">+                obj->priv_data.lockused  = 1;</span><br><span style="color: hsl(120, 100%, 40%);">+      }</span><br><span style="color: hsl(120, 100%, 40%);">+</span><br><span>  switch (obj->priv_data.options & AO2_ALLOC_OPT_LOCK_MASK) {</span><br><span>   case AO2_ALLOC_OPT_LOCK_MUTEX:</span><br><span>               obj_mutex = INTERNAL_OBJ_MUTEX(user_data);</span><br><span>@@ -369,7 +402,6 @@</span><br><span>             return -1;</span><br><span>   }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-</span><br><span>        return res;</span><br><span> }</span><br><span> </span><br><span>@@ -473,9 +505,7 @@</span><br><span>   int32_t current_value;</span><br><span>       int32_t ret;</span><br><span>         struct ao2_weakproxy *weakproxy = NULL;</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span>  const char *lock_state;</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span> </span><br><span>    if (obj == NULL) {</span><br><span>           if (ref_log && user_data) {</span><br><span>@@ -595,33 +625,25 @@</span><br><span>  switch (obj->priv_data.options & AO2_ALLOC_OPT_LOCK_MASK) {</span><br><span>   case AO2_ALLOC_OPT_LOCK_MUTEX:</span><br><span>               obj_mutex = INTERNAL_OBJ_MUTEX(user_data);</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span style="color: hsl(0, 100%, 40%);">-          lock_state = obj_mutex->mutex.lock.flags.setup ? "used" : "unused";</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span style="color: hsl(120, 100%, 40%);">+              lock_state = obj->priv_data.lockused ? "used" : "unused";</span><br><span>             ast_mutex_destroy(&obj_mutex->mutex.lock);</span><br><span> </span><br><span>                ast_free(obj_mutex);</span><br><span>                 break;</span><br><span>       case AO2_ALLOC_OPT_LOCK_RWLOCK:</span><br><span>              obj_rwlock = INTERNAL_OBJ_RWLOCK(user_data);</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span style="color: hsl(0, 100%, 40%);">-                lock_state = obj_rwlock->rwlock.lock.flags.setup ? "used" : "unused";</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span style="color: hsl(120, 100%, 40%);">+            lock_state = obj->priv_data.lockused ? "used" : "unused";</span><br><span>             ast_rwlock_destroy(&obj_rwlock->rwlock.lock);</span><br><span> </span><br><span>             ast_free(obj_rwlock);</span><br><span>                break;</span><br><span>       case AO2_ALLOC_OPT_LOCK_NOLOCK:</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span>          lock_state = "none";</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span>                 ast_free(obj);</span><br><span>               break;</span><br><span>       case AO2_ALLOC_OPT_LOCK_OBJ:</span><br><span>                 obj_lockobj = INTERNAL_OBJ_LOCKOBJ(user_data);</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span style="color: hsl(0, 100%, 40%);">-              lock_state = "lockobj";</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span style="color: hsl(120, 100%, 40%);">+               lock_state = obj->priv_data.lockused ? "used" : "unused";</span><br><span>             ao2_t_ref(obj_lockobj->lockobj.lock, -1, "release lockobj");</span><br><span> </span><br><span>                ast_free(obj_lockobj);</span><br><span>@@ -629,22 +651,13 @@</span><br><span>       default:</span><br><span>             ast_log(__LOG_ERROR, file, line, func,</span><br><span>                       "Invalid lock option on ao2 object %p\n", user_data);</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span>          lock_state = "invalid";</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span>              break;</span><br><span>       }</span><br><span> </span><br><span>        if (ref_log && tag) {</span><br><span style="color: hsl(0, 100%, 40%);">-           fprintf(ref_log, "%p,%d,%d,%s,%d,%s,**destructor%s%s**,%s\n",</span><br><span style="color: hsl(0, 100%, 40%);">-                 user_data, delta, ast_get_tid(), file, line, func,</span><br><span style="color: hsl(0, 100%, 40%);">-#ifdef DEBUG_THREADS</span><br><span style="color: hsl(0, 100%, 40%);">-                  "**lock-state:",</span><br><span style="color: hsl(0, 100%, 40%);">-                      lock_state,</span><br><span style="color: hsl(0, 100%, 40%);">-#else</span><br><span style="color: hsl(0, 100%, 40%);">-                        "", "",</span><br><span style="color: hsl(0, 100%, 40%);">-#endif</span><br><span style="color: hsl(0, 100%, 40%);">-                       tag);</span><br><span style="color: hsl(120, 100%, 40%);">+         fprintf(ref_log, "%p,%d,%d,%s,%d,%s,**destructor**lock-state:%s**,%s\n",</span><br><span style="color: hsl(120, 100%, 40%);">+                    user_data, delta, ast_get_tid(), file, line, func, lock_state, tag);</span><br><span>                 fflush(ref_log);</span><br><span>     }</span><br><span> </span><br><span>@@ -673,10 +686,12 @@</span><br><span>        struct astobj2_lock *obj_mutex;</span><br><span>      struct astobj2_rwlock *obj_rwlock;</span><br><span>   struct astobj2_lockobj *obj_lockobj;</span><br><span style="color: hsl(120, 100%, 40%);">+  size_t overhead;</span><br><span> </span><br><span>         switch (options & AO2_ALLOC_OPT_LOCK_MASK) {</span><br><span>     case AO2_ALLOC_OPT_LOCK_MUTEX:</span><br><span style="color: hsl(0, 100%, 40%);">-          obj_mutex = __ast_calloc(1, sizeof(*obj_mutex) + data_size, file, line, func);</span><br><span style="color: hsl(120, 100%, 40%);">+                overhead = sizeof(*obj_mutex);</span><br><span style="color: hsl(120, 100%, 40%);">+                obj_mutex = __ast_calloc(1, overhead + data_size, file, line, func);</span><br><span>                 if (obj_mutex == NULL) {</span><br><span>                     return NULL;</span><br><span>                 }</span><br><span>@@ -685,7 +700,8 @@</span><br><span>              obj = (struct astobj2 *) &obj_mutex->priv_data;</span><br><span>               break;</span><br><span>       case AO2_ALLOC_OPT_LOCK_RWLOCK:</span><br><span style="color: hsl(0, 100%, 40%);">-         obj_rwlock = __ast_calloc(1, sizeof(*obj_rwlock) + data_size, file, line, func);</span><br><span style="color: hsl(120, 100%, 40%);">+              overhead = sizeof(*obj_rwlock);</span><br><span style="color: hsl(120, 100%, 40%);">+               obj_rwlock = __ast_calloc(1, overhead + data_size, file, line, func);</span><br><span>                if (obj_rwlock == NULL) {</span><br><span>                    return NULL;</span><br><span>                 }</span><br><span>@@ -694,7 +710,8 @@</span><br><span>              obj = (struct astobj2 *) &obj_rwlock->priv_data;</span><br><span>              break;</span><br><span>       case AO2_ALLOC_OPT_LOCK_NOLOCK:</span><br><span style="color: hsl(0, 100%, 40%);">-         obj = __ast_calloc(1, sizeof(*obj) + data_size, file, line, func);</span><br><span style="color: hsl(120, 100%, 40%);">+            overhead = sizeof(*obj);</span><br><span style="color: hsl(120, 100%, 40%);">+              obj = __ast_calloc(1, overhead + data_size, file, line, func);</span><br><span>               if (obj == NULL) {</span><br><span>                   return NULL;</span><br><span>                 }</span><br><span>@@ -706,7 +723,8 @@</span><br><span>                      return NULL;</span><br><span>                 }</span><br><span> </span><br><span style="color: hsl(0, 100%, 40%);">-           obj_lockobj = __ast_calloc(1, sizeof(*obj_lockobj) + data_size, file, line, func);</span><br><span style="color: hsl(120, 100%, 40%);">+            overhead = sizeof(*obj_lockobj);</span><br><span style="color: hsl(120, 100%, 40%);">+              obj_lockobj = __ast_calloc(1, overhead + data_size, file, line, func);</span><br><span>               if (obj_lockobj == NULL) {</span><br><span>                   ao2_t_ref(lockobj, -1, "release lockobj for failed alloc");</span><br><span>                        return NULL;</span><br><span>@@ -735,8 +753,8 @@</span><br><span> #endif</span><br><span> </span><br><span>     if (ref_log && tag) {</span><br><span style="color: hsl(0, 100%, 40%);">-           fprintf(ref_log, "%p,+1,%d,%s,%d,%s,**constructor**,%s\n",</span><br><span style="color: hsl(0, 100%, 40%);">-                    EXTERNAL_OBJ(obj), ast_get_tid(), file, line, func, tag);</span><br><span style="color: hsl(120, 100%, 40%);">+             fprintf(ref_log, "%p,+1,%d,%s,%d,%s,**constructor**%zu**%zu**,%s\n",</span><br><span style="color: hsl(120, 100%, 40%);">+                        EXTERNAL_OBJ(obj), ast_get_tid(), file, line, func, overhead, data_size, tag);</span><br><span>               fflush(ref_log);</span><br><span>     }</span><br><span> </span><br><span></span><br></pre><p>To view, visit <a href="https://gerrit.asterisk.org/10477">change 10477</a>. To unsubscribe, or for help writing mail filters, visit <a href="https://gerrit.asterisk.org/settings">settings</a>.</p><div itemscope itemtype="http://schema.org/EmailMessage"><div itemscope itemprop="action" itemtype="http://schema.org/ViewAction"><link itemprop="url" href="https://gerrit.asterisk.org/10477"/><meta itemprop="name" content="View Change"/></div></div>

<div style="display:none"> Gerrit-Project: asterisk </div>
<div style="display:none"> Gerrit-Branch: master </div>
<div style="display:none"> Gerrit-MessageType: merged </div>
<div style="display:none"> Gerrit-Change-Id: I84e9d679cc86d772cc97c888d9d856a17e0d3a4a </div>
<div style="display:none"> Gerrit-Change-Number: 10477 </div>
<div style="display:none"> Gerrit-PatchSet: 2 </div>
<div style="display:none"> Gerrit-Owner: Corey Farrell <git@cfware.com> </div>
<div style="display:none"> Gerrit-Reviewer: George Joseph <gjoseph@digium.com> </div>
<div style="display:none"> Gerrit-Reviewer: Jenkins2 (1000185) </div>
<div style="display:none"> Gerrit-Reviewer: Richard Mudgett <rmudgett@digium.com> </div>