[asterisk-dev] [Code Review] 3027: Valgrind support in TestSuite

[Scott Griepentrog]

> On Nov. 27, 2013, 11:37 a.m., Matt Jordan wrote:
> > /asterisk/team/sgriepentrog/testsuite-valgrind/runtests.py, lines 84-93
> > <https://reviewboard.asterisk.org/r/3027/diff/2/?file=48827#file48827line84>
> >
> >     It's very strange for us to call __parse_run_output twice here. We should only need to call it once - ideally, we would have the valgrind output that we want to display on stdout already ready for it.

The purpose was to reassign self.output to self.failure_message after parse_valgrind_xml() had added to it (self.output).  This has been reworked.


> On Nov. 27, 2013, 11:37 a.m., Matt Jordan wrote:
> > /asterisk/team/sgriepentrog/testsuite-valgrind/runtests.py, lines 297-298
> > <https://reviewboard.asterisk.org/r/3027/diff/2/?file=48827#file48827line297>
> >
> >     Don't direct the output of errors to both stdout and print them directly. In general, we shouldn't call print - that overrides the user's choices in terms of logging and what is displayed.

When manually running a large group of tests, stdout is delayed until all tests complete, whereas print is shown immediately.  There are times when this output is comforting to observe progress on the test, but I'll reduce it to a total # of errors and exceptions.


- Scott


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/3027/#review10291
-----------------------------------------------------------


On Dec. 10, 2013, 11:51 a.m., Scott Griepentrog wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/3027/
> -----------------------------------------------------------
> 
> (Updated Dec. 10, 2013, 11:51 a.m.)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Repository: testsuite
> 
> 
> Description
> -------
> 
> This patch adds support for running Asterisk under Valgrind (say: Val-Grinned) to check for all sorts of nasty runtime bugs.  This started off with a post to asterisk-dev list by nitesh.bansal at gmail.com, and he wrote and contributed the initial version.  So if you find this useful, be sure to thank him.  Then I made extensive changes and additions, so if the code stinks, blame me.
> 
> The following has been done:
> 
> - Check runtests.py arguments for --valgrind and --valgrind-gensupp flags, pass via environ to TestCase.py
> - Note previously existing instances of ast# logs to insure we only process new log/xml files
> - Increase reactor timeout by x5 when valgrind enabled
> - Patch to reactor_stop() to insure it does even when exceptions occur in deferred stack
> - Add valgrind with correct arguments into executable path
> - After run, check valgrind.xml for errors, parse and condense them into something more managable
> - If -gensup mode enabled, write suppressions to logs/(test)/ast#/valgrind.supp that can be added (manually)
> - A default valgrind.supp "suppressions" file is in configs/ to prevent complaints about known unfixables
> 
> Notes:
> 
> - valgrind can be triggered by argument to runtests.py, export VALGRIND=true, or in test-config.yaml
> - configs/valgrind.supp will be used if found, but tests/(test)/configs/valgrind.supp will take precedence
> - valgrind-gensupp mode will create example suppressions file, during which no suppressions occur
> 
> 
> Diffs
> -----
> 
>   /asterisk/trunk/runtests.py 4350 
>   /asterisk/trunk/lib/python/asterisk/asterisk.py 4350 
>   /asterisk/trunk/lib/python/asterisk/Valgrind.py PRE-CREATION 
>   /asterisk/trunk/lib/python/asterisk/TestConfig.py 4350 
>   /asterisk/trunk/lib/python/asterisk/TestCase.py 4350 
>   /asterisk/trunk/configs/valgrind.supp PRE-CREATION 
>   /asterisk/trunk/README.txt 4350 
> 
> Diff: https://reviewboard.asterisk.org/r/3027/diff/
> 
> 
> Testing
> -------
> 
> Tested on 64bit and 32bit CentOS, including low cpu power conditions.  Some sporadic timing issues affecting AMI/twisted operation have been seen and will be corrected.
> 
> 
> Thanks,
> 
> Scott Griepentrog
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20131210/6c25da1a/attachment-0001.html>