[asterisk-dev] [Code Review] 3363: Testsuite: Pluggable module for testing realtime

[Mark Michelson]

> On April 8, 2014, 7:24 p.m., Corey Farrell wrote:
> > /asterisk/trunk/lib/python/asterisk/realtime_test_module.py, line 489
> > <https://reviewboard.asterisk.org/r/3363/diff/1/?file=56114#file56114line489>
> >
> >     Port 8000 is commonly used as the alternate/unprivileged port 80, so this increases the chance of port conflicts.  I often use 127.0.0.1:8000 for SSH tunnels to remote HTTP from my desktop (test system).  I believe the correct fix would be for realtime_test_module to listen to a dynamic port (port 0), generate extconfig.conf from the result.  I recognize that doing so is probably out of scope for this change.  Using a higher port or a different localhost IP would reduce the chance of this happening.
> >     
> >     As a workaround can this test detect if the HTTP server fails to start due to port already in use and skip/abort the test?  We would want to avoid hitting an unknown daemon listening to port 127.0.0.1:8000, especially since that daemon might trust our requests due to localhost source address.

I have altered this to bind to a higher-numbered port.

Regarding your second suggestion, right now, twisted will throw a twisted.internet.error.CannotListenError if it fails to bind to the requested port. This will cause the test to abort currently due to an unhandled exception.

The more graceful way of handling this would be to catch the exception, throw another exception that the testsuite understands, and have the test runner return a failure result for the test. However, that requires modification of the core test_runner.py code that I think is outside the scope of this changeset.


- Mark


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/3363/#review11516
-----------------------------------------------------------


On March 15, 2014, 6:34 p.m., Mark Michelson wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/3363/
> -----------------------------------------------------------
> 
> (Updated March 15, 2014, 6:34 p.m.)
> 
> 
> Review request for Asterisk Developers.
> 
> 
> Repository: testsuite
> 
> 
> Description
> -------
> 
> This introduces a new pluggable module to the Asterisk testsuite intended to assist in testing using realtime storage. The test module sets up an HTTP server on port 8000 and services requests Asterisk makes using its res_config_curl realtime backend. The actual data is stored in memory in python using simple dictionaries and lists. The test module configuration allows for data to be preloaded into python before Asterisk is started so that Asterisk may retrieve realtime data during startup. Once AMI is connected to Asterisk, the realtime test module hands over control to a python module so that the test may be further controlled by the test writer.
> 
> Along with the module are six realtime tests, each designed to test an operation that Asterisk can attempt. The "require" operation is not tested since we currently always claim to have the proper items stored in the expected way.
> 
> There are potential improvements that could be made, such as:
> * Memoization of HTTP resources served by the test module.
> * yaml-driven test development beyond the initial population of realtime data.
> * Potentially allow for the realtime test module to be run stand-alone so that developers can have an easy realtime store to use for testing.
> 
> None of these are enough to prevent inclusion into the test suite though.
> 
> 
> Diffs
> -----
> 
>   /asterisk/trunk/tests/tests.yaml 4836 
>   /asterisk/trunk/tests/realtime/update/update.py PRE-CREATION 
>   /asterisk/trunk/tests/realtime/update/test-config.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/update/configs/ast1/sorcery.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/update/configs/ast1/extconfig.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/tests.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/store/test-config.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/store/store.py PRE-CREATION 
>   /asterisk/trunk/tests/realtime/store/configs/ast1/sorcery.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/store/configs/ast1/extconfig.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/static/test-config.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/static/static.py PRE-CREATION 
>   /asterisk/trunk/tests/realtime/static/configs/ast1/modules.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/static/configs/ast1/extconfig.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/single/test-config.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/single/single.py PRE-CREATION 
>   /asterisk/trunk/tests/realtime/single/configs/ast1/sorcery.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/single/configs/ast1/extconfig.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/multi/test-config.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/multi/multi.py PRE-CREATION 
>   /asterisk/trunk/tests/realtime/multi/configs/ast1/sorcery.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/multi/configs/ast1/extconfig.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/destroy/test-config.yaml PRE-CREATION 
>   /asterisk/trunk/tests/realtime/destroy/destroy.py PRE-CREATION 
>   /asterisk/trunk/tests/realtime/destroy/configs/ast1/sorcery.conf PRE-CREATION 
>   /asterisk/trunk/tests/realtime/destroy/configs/ast1/extconfig.conf PRE-CREATION 
>   /asterisk/trunk/lib/python/asterisk/realtime_test_module.py PRE-CREATION 
> 
> Diff: https://reviewboard.asterisk.org/r/3363/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Mark Michelson
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20140615/31f43027/attachment.html>