[asterisk-dev] [Code Review] 3377: ref count logs: Redo structure of log file, provide a python debugging tool

[Corey Farrell]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/3377/#review11290
-----------------------------------------------------------


We should remove utils/refcounter from all branches.  The format is changing, so the existing utils/refcounter will be unusable with 1.8.27.0 / 11.9.0 / 12.2.0.  I think providing utils/refcounter in those versions will only cause confusion.  Maybe we could enable installation of refcounter.py to ${ASTSBINDIR}/refcounter?


/branches/1.8/build_tools/cflags-devmode.xml
<https://reviewboard.asterisk.org/r/3377/#comment20911>

    I think cflags-devmode should not be used for options that bug reporters will be asked to use, so this probably belongs in cflags.xml just like MALLOC_DEBUG.  I think your reasoning for applying this to 1.8 is still valid.  This option is not intended for everyday production use and only effects code compiled with the define/new option.
    
    A comment exists in chan_sip.c to tell us where to put REF_DEBUG.  Comments like that should probably be changed or removed to reflect the preference for system-wide REF_DEBUG.



/branches/1.8/contrib/scripts/refcounter.py
<https://reviewboard.asterisk.org/r/3377/#comment20919>

    We should pass maxsplit = 6 to prevent splitting a tag that includes ','.



/branches/1.8/contrib/scripts/refcounter.py
<https://reviewboard.asterisk.org/r/3377/#comment20926>

    This should check parsed_line['state'] == '**constructor**', produce a warning and add obj to a list of errors if it's not.  The previous refcounter would inform us of objects that were never created.



/branches/1.8/contrib/scripts/refcounter.py
<https://reviewboard.asterisk.org/r/3377/#comment20924>

    Would be nice to say "==== %s Object %s history ====" with "Finalized" or "Leaked" as the first %s.  The single line section header would be difficult to find with more than a couple leaked objects.



/branches/1.8/contrib/scripts/refcounter.py
<https://reviewboard.asterisk.org/r/3377/#comment20912>

    Can we get an option for output filename?  I'd prefer we have --input and --output rather than --file.
    
    Would be nice if both options supported '-' for STDIO.



/branches/1.8/main/astobj2.c
<https://reviewboard.asterisk.org/r/3377/#comment20914>

    Can we deduplicate the ref_log writes into a macro?  I'm suggesting a macro so we can pass "%d" or "%s" for the format of the 'state' field, and let the compiler join the const strings.
    
    This would make it easier to append additional fields if desired, and ensure consistent output.  For example it is sometimes useful to add the current thread id to all ref_log messages.



/branches/1.8/main/astobj2.c
<https://reviewboard.asterisk.org/r/3377/#comment20925>

    We need to reorder the output so tag is last.  A comma will never be included in file, line or funcname.  If tag is not last and includes a comma it will shift all fields that follow.



/branches/1.8/main/astobj2.c
<https://reviewboard.asterisk.org/r/3377/#comment20921>

    I think we should not check for destructor_fn != NULL.  Consider the following:
    ao2_ref(ao2_alloc(50, NULL), -1);
    
    This would not print "**call destructor**", causing refcounter.py to report a leak.  
    
    Also I think we should change this to "**destructor**", since we will print the line for all objects to reach 0 refs.  Since we're already making changes to the format we might as well make all the desired tweaks now.



/branches/1.8/main/astobj2.c
<https://reviewboard.asterisk.org/r/3377/#comment20923>

    Should this be "%p,+1,..." to be consistent with the way ao2_ref increments?



/branches/1.8/main/astobj2.c
<https://reviewboard.asterisk.org/r/3377/#comment20910>

    What about ao2 objects created before astobj2_init?  I know that storing refs in astlogdir was my idea/request, but I'm concerned this will cause an unfixable issue with initialization order.  If we open the file too early, ast_config_AST_LOG_DIR will be the compiled value instead of the value in asterisk.conf.  If we open too late, startup messages are lost.  Further it's impossible to read astlogdir from asterisk.conf before __attribute__((constructor)).
    
    What if we allow an environmental variable to override /tmp/refs?  My goal with using astlogdir was to allow multiple instances of asterisk to run without causing conflicts in /tmp/refs.  This would be essential for the testsuite to check refs after tests.
    
    I don't think we can (safely) avoid opening/closing the refs file for every output.  Doing so would require us to open ref_log before any other __attribute__((constructor)) and closing it after all other __attribute__((destructor)), I think this would be impossible without starting to use constructor/destructor priorities.


- Corey Farrell


On March 19, 2014, 2:22 p.m., Matt Jordan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/3377/
> -----------------------------------------------------------
> 
> (Updated March 19, 2014, 2:22 p.m.)
> 
> 
> Review request for Asterisk Developers, Corey Farrell and wdoekes.
> 
> 
> Repository: Asterisk
> 
> 
> Description
> -------
> 
> Note: while an improvement to Asterisk, this patch only affects Asterisk when compiled in -dev-mode. Since it has benefit only for developers looking to fix bugs, I'm proposing it for Asterisk 1.8+. The removal of refcounter should be done in trunk only.
> 
> Asterisk uses reference counted objects. A lot. As their use has spread, the bugs related to reference counting errors has grown as well. Central to resolving reference counting issues is the usage of REF_DEBUG; unfortunately, REF_DEBUG has had several problems:
> (1) It could not be enabled through menuselect
> (2) When not enabled globally, updates to objects were often lost, resulting in insufficient data to resolve problems
> (3) The format of the ref debug log file was not suitable for parsing
> 
> This patch changes REF_DEBUG in the following ways:
> (1) It makes REF_DEBUG a meneselect item when Asterisk is compiled with --enable-dev-mode
> (2) The ref debug log file is now created in the AST_LOG_DIR directory. Every run will now blow away the previous run (as large ref files sometimes caused issues). We now also no longer open/close the file on each write, instead relying on fflush to make sure data gets written to the file (in case the ao2 call being performed is about to cause a crash)
> (3) It goes with a comma delineated format for the ref debug file. This makes parsing much easier.
> (4) It throws out the refcounter utility and goes with a python script instead.
> 
> 
> Diffs
> -----
> 
>   /branches/1.8/utils/refcounter.c 410891 
>   /branches/1.8/utils/Makefile 410891 
>   /branches/1.8/main/astobj2.c 410891 
>   /branches/1.8/contrib/scripts/refcounter.py PRE-CREATION 
>   /branches/1.8/build_tools/cflags-devmode.xml 410891 
> 
> Diff: https://reviewboard.asterisk.org/r/3377/diff/
> 
> 
> Testing
> -------
> 
> Things spit out ref logs and the script parses them. Example below:
> 
> ==== Object 0x21ed9b8 history ====
> features.c[5427]:create_parkinglot 1  - [**constructor**]
> features.c[5707]:build_parkinglot +1  - [1]
> features.c[5392]:parkinglot_unref -1  - [2]
> features.c[6358]:build_dialplan_useage_map +1  - [1]
> features.c[6358]:build_dialplan_useage_map -1  - [2]
> features.c[4985]:find_parkinglot +1  - [1]
> features.c[5392]:parkinglot_unref -1  - [2]
> features.c[6358]:build_dialplan_useage_map +1  - [1]
> features.c[6358]:build_dialplan_useage_map -1  - [2]
> features.c[4955]:do_parking_thread +1  - [1]
> features.c[4957]:do_parking_thread -1  - [2]
> astobj2.c[955]:cd_cb_debug -1 deref object via container destroy - [1]
> astobj2.c[955]:cd_cb_debug -1 deref object via container destroy - [**call destructor**]
> 
> 
> Thanks,
> 
> Matt Jordan
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20140320/d8c1975e/attachment-0001.html>