No subject
Fri Sep 2 03:59:05 CDT 2011
ms like a decent idea would be to get tests that aren't currently derived f=
rom TestCase converted so they are. It will take some of the burden of star=
ting and stopping Asterisk from them.
A more python-oriented person might want to take a look through this regard=
ing style, etc.
/asterisk/trunk/tests/udptl_v6/run-test
<https://reviewboard.asterisk.org/r/1821/#comment10696>
Something tells me this isn't supposed to be here.
- Mark
On March 20, 2012, 11:47 a.m., Matt Jordan wrote:
> =
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/1821/
> -----------------------------------------------------------
> =
> (Updated March 20, 2012, 11:47 a.m.)
> =
> =
> Review request for Asterisk Developers.
> =
> =
> Summary
> -------
> =
> As part of the Lightweight NAT test development (https://reviewboard.aste=
risk.org/r/1759/), Josh discovered that the Python subprocess module and tw=
isted do not play well together. At all. Both twisted and the subprocess =
attempt to handle SIGCHLD signals, with the result that one or the other (t=
ypically twisted, however) fail in some fashion or another. =
> =
> More information on this can be found at the links below:
> =
> http://stackoverflow.com/questions/1948641/twisted-threading-with-subproc=
ess-popen
> http://armyofevilrobots.com/node/422
> http://twistedmatrix.com/trac/ticket/2535
> =
> Note that in the case of the Lightweight NAT tests, the behavior was foun=
d without even spawning additional processes; the reactor would block indef=
initely when told to listen for UDP packets.
> =
> To solve this, there were a few approaches that we could take:
> 1. Abandon twisted. This would also imply that we would have to abandon =
our usage of starpy for FastAGI/AMI integration. This would not be good, b=
ut we could write a non-twisted dependent interface for those two libraries=
. However, twisted is a robust library that greatly speeds development of =
communication interfaces beyond FastAGI/AMI. For that reason (and a whole =
host of others), we decided not to go with this.
> 2. Instruct twisted to not use sign handlers by passing installSignalHand=
lers=3DFalse to the reactor. While this may make subprocess and twisted pl=
ay nice with each other, it limits the effectiveness of twisted, and could =
cause issues elsewhere when third party libraries using twisted make the as=
sumption that the reactor will have signal handlers. This seemed like a wo=
rkaround, as opposed to a solution.
> 3. Stop using subprocess and use twisted to manage processes.
> =
> We went with option 3. This has the impact of making a lot of previously=
blocking operations non-blocking (CLI commands, for example) - which while=
good, does increase the complexity of those operations. At the same time,=
it has the added benefit of removing a number of polling loops and blockin=
g operations that didn't need to be blocking, with the end result that the =
tests run much, much faster. (For example: some tests which previously too=
k 15 - 30 seconds now take 2 - 5)
> =
> This patch has the following impacts on test development:
> 1. The Asterisk class now uses non-blocking operations to start/stop itse=
lf, and instead returns a deferred object that can be used to be notified w=
hen Asterisk has started/stopped. As a result, tests that derive from Test=
Case now no longer need to call start_asterisk and stop_asterisk, as the st=
arting/stopping of Asterisk is handled by the TestCase class itself. Tests=
that do not use the TestCase object will need to add callbacks to the defe=
rred object to ensure that Asterisk is fully started/stopped.
> 2. The Asterisk class's CLI command interfaces are all non-blocking, and =
return a deferred object that signals when the command has completed.
> 3. SIPpScenario now returns a deferred when run, which signals when the s=
cenario has ended and its success/failure.
> 4. A new class, SIPpScenarioSequence has been added, which will run a ser=
ies of SIPpScenario objects in a synchronous fashion. This eases the burde=
n on people who were using SIPpScenario to run successive tests, as opposed=
to tests in parallel.
> 5. The utils module has been renamed to TestSuiteUtils, to prevent a name=
conflict with twisted's utils modules.
> =
> Note that we still have a number of tests that use pjsua and the subproce=
ss module; as we currently have no library wrapping this usage I've deferre=
d (pun intended) that work to another patch. The Lightweight NAT tests do =
not use pjsua, and hence are not dependent on that work being finished to b=
e included in the Test Suite.
> =
> =
> Diffs
> -----
> =
> /asterisk/trunk/tests/udptl_v6/configs/ast1/manager.conf 3104 =
> /asterisk/trunk/tests/udptl_v6/configs/ast2/manager.conf 3104 =
> /asterisk/trunk/tests/udptl_v6/run-test 3104 =
> /asterisk/trunk/tests/mixmonitor/run-test 3104 =
> /asterisk/trunk/tests/mixmonitor_audiohook_inherit/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-date/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-datetime/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-digits/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-number/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-phonetic/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-time/run-test 3104 =
> /asterisk/trunk/tests/feature_attended_transfer/run-test 3104 =
> /asterisk/trunk/tests/feature_blonde_transfer/run-test 3104 =
> /asterisk/trunk/tests/iax2/basic-call/configs/ast1/extensions.conf 3104 =
> /asterisk/trunk/tests/iax2/basic-call/configs/ast1/manager.conf 3104 =
> /asterisk/trunk/tests/iax2/basic-call/configs/ast2/extensions.conf 3104 =
> /asterisk/trunk/tests/iax2/basic-call/configs/ast2/manager.conf 3104 =
> /asterisk/trunk/tests/iax2/basic-call/run-test 3104 =
> /asterisk/trunk/tests/fastagi/say-alpha/run-test 3104 =
> /asterisk/trunk/tests/dynamic-modules/test-config.yaml 3104 =
> /asterisk/trunk/tests/fastagi/get-data/run-test 3104 =
> /asterisk/trunk/tests/fastagi/hangup/run-test 3104 =
> /asterisk/trunk/tests/dynamic-modules/run-test 3104 =
> /asterisk/trunk/tests/chanspy/chanspy_w_mixmonitor/run-test 3104 =
> /asterisk/trunk/tests/chanspy/chanspy_barge/configs/ast1/manager.conf 3=
104 =
> /asterisk/trunk/tests/chanspy/chanspy_barge/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/sip_tls_call/configs/ast1/extensions=
.conf 3104 =
> /asterisk/trunk/tests/channels/SIP/sip_tls_call/configs/ast2/extensions=
.conf 3104 =
> /asterisk/trunk/tests/channels/SIP/sip_tls_register/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/use_contact_from_200/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/sip_register_domain_acl/run-test 310=
4 =
> /asterisk/trunk/tests/channels/SIP/sip_hold/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/nat_supertest/test-config.yaml 3104 =
> /asterisk/trunk/tests/channels/SIP/realtime_nosipregs/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/realtime_sipregs/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/nat_supertest/sipp/inject.csv 3104 =
> /asterisk/trunk/tests/channels/SIP/nat_supertest/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/info_dtmf/run-test 3104 =
> /asterisk/trunk/tests/channels/SIP/handle_response_address_incomplete/r=
un-test 3104 =
> /asterisk/trunk/tests/apps/incomplete/sip_incomplete/run-test 3104 =
> /asterisk/trunk/tests/apps/voicemail/func_vmcount/run-test 3104 =
> /asterisk/trunk/lib/python/asterisk/utils.py 3104 =
> /asterisk/trunk/lib/python/asterisk/version.py 3104 =
> /asterisk/trunk/runtests.py 3104 =
> /asterisk/trunk/lib/python/asterisk/sippversion.py 3104 =
> /asterisk/trunk/lib/python/asterisk/sipp.py 3104 =
> /asterisk/trunk/lib/python/asterisk/cdr.py 3104 =
> /asterisk/trunk/lib/python/asterisk/asterisk.py 3104 =
> /asterisk/trunk/lib/python/asterisk/TestConfig.py 3104 =
> /asterisk/trunk/lib/python/asterisk/TestCase.py 3104 =
> /asterisk/trunk/lib/python/asterisk/CDRTestCase.py 3104 =
> =
> Diff: https://reviewboard.asterisk.org/r/1821/diff
> =
> =
> Testing
> -------
> =
> =
> Thanks,
> =
> Matt
> =
>
--===============8336183712139067814==
Content-Type: text/html; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
<html>
<body>
<div style=3D"font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor=3D"#f9f3c9" width=3D"100%" cellpadding=3D"8" style=3D"bor=
der: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href=3D"https://reviewboard.asterisk.org/r/1821/">https://reviewbo=
ard.asterisk.org/r/1821/</a>
</td>
</tr>
</table>
<br />
<p>Ship it!</p>
<pre style=3D"white-space: pre-wrap; white-space: -moz-pre-wrap; white-spa=
ce: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">From my lo=
ok through, things look good. With these sorts of changes, it seems like a =
decent idea would be to get tests that aren't currently derived from Te=
stCase converted so they are. It will take some of the burden of starting a=
nd stopping Asterisk from them.
A more python-oriented person might want to take a look through this regard=
ing style, etc.</pre>
<br />
<div>
<table width=3D"100%" border=3D"0" bgcolor=3D"white" style=3D"border: 1px s=
olid #C0C0C0; border-collapse: collapse; margin: 2px padding: 2px;">
<thead>
<tr>
<th colspan=3D"4" bgcolor=3D"#F0F0F0" style=3D"border-bottom: 1px solid =
#C0C0C0; font-size: 9pt; padding: 4px 8px; text-align: left;">
<a href=3D"https://reviewboard.asterisk.org/r/1821/diff/2/?file=3D26468=
#file26468line126" style=3D"color: black; font-weight: bold; text-decoratio=
n: underline;">/asterisk/trunk/tests/udptl_v6/run-test</a>
<span style=3D"font-weight: normal;">
(Diff revision 2)
</span>
</th>
</tr>
</thead>
<tbody style=3D"background-color: #e4d9cb; padding: 4px 8px; text-align: c=
enter;">
<tr>
<td colspan=3D"4"><pre style=3D"font-size: 8pt; line-height: 140%; margi=
n: 0; "></pre></td>
</tr>
</tbody>
=
=
<tbody>
<tr>
<th bgcolor=3D"#b1ebb0" style=3D"border-right: 1px solid #C0C0C0;" alig=
n=3D"right"><font size=3D"2"></font></th>
<td bgcolor=3D"#c5ffc4" width=3D"50%"><pre style=3D"font-size: 8pt; lin=
e-height: 140%; margin: 0; "></pre></td>
<th bgcolor=3D"#b1ebb0" style=3D"border-left: 1px solid #C0C0C0; border=
-right: 1px solid #C0C0C0;" align=3D"right"><font size=3D"2">70</font></th>
<td bgcolor=3D"#c5ffc4" width=3D"50%"><pre style=3D"font-size: 8pt; lin=
e-height: 140%; margin: 0; "># vim:sw=3D4:ts=3D4:expandtab:textwidth=3D79</=
pre></td>
</tr>
</tbody>
</table>
<pre style=3D"margin-left: 2em; white-space: pre-wrap; white-space: -moz-pr=
e-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-=
word;">Something tells me this isn't supposed to be here.</pre>
</div>
<br />
<p>- Mark</p>
<br />
<p>On March 20th, 2012, 11:47 a.m., Matt Jordan wrote:</p>
<table bgcolor=3D"#fefadf" width=3D"100%" cellspacing=3D"0" cellpadding=3D"=
8" style=3D"background-image: url('https://reviewboard.asterisk.org/media/r=
b/images/review_request_box_top_bg.png'); background-position: left top; ba=
ckground-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers.</div>
<div>By Matt Jordan.</div>
<p style=3D"color: grey;"><i>Updated March 20, 2012, 11:47 a.m.</i></p>
<h1 style=3D"color: #575012; font-size: 10pt; margin-top: 1.5em;">Descripti=
on </h1>
<table width=3D"100%" bgcolor=3D"#ffffff" cellspacing=3D"0" cellpadding=3D"=
10" style=3D"border: 1px solid #b8b5a0">
<tr>
<td>
<pre style=3D"margin: 0; padding: 0; white-space: pre-wrap; white-space:=
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap=
: break-word;">As part of the Lightweight NAT test development (https://rev=
iewboard.asterisk.org/r/1759/), Josh discovered that the Python subprocess =
module and twisted do not play well together. At all. Both twisted and th=
e subprocess attempt to handle SIGCHLD signals, with the result that one or=
the other (typically twisted, however) fail in some fashion or another. =
More information on this can be found at the links below:
http://stackoverflow.com/questions/1948641/twisted-threading-with-subproces=
s-popen
http://armyofevilrobots.com/node/422
http://twistedmatrix.com/trac/ticket/2535
Note that in the case of the Lightweight NAT tests, the behavior was found =
without even spawning additional processes; the reactor would block indefin=
itely when told to listen for UDP packets.
To solve this, there were a few approaches that we could take:
1. Abandon twisted. This would also imply that we would have to abandon ou=
r usage of starpy for FastAGI/AMI integration. This would not be good, but=
we could write a non-twisted dependent interface for those two libraries. =
However, twisted is a robust library that greatly speeds development of co=
mmunication interfaces beyond FastAGI/AMI. For that reason (and a whole ho=
st of others), we decided not to go with this.
2. Instruct twisted to not use sign handlers by passing installSignalHandle=
rs=3DFalse to the reactor. While this may make subprocess and twisted play=
nice with each other, it limits the effectiveness of twisted, and could ca=
use issues elsewhere when third party libraries using twisted make the assu=
mption that the reactor will have signal handlers. This seemed like a work=
around, as opposed to a solution.
3. Stop using subprocess and use twisted to manage processes.
We went with option 3. This has the impact of making a lot of previously b=
locking operations non-blocking (CLI commands, for example) - which while g=
ood, does increase the complexity of those operations. At the same time, i=
t has the added benefit of removing a number of polling loops and blocking =
operations that didn't need to be blocking, with the end result that th=
e tests run much, much faster. (For example: some tests which previously t=
ook 15 - 30 seconds now take 2 - 5)
This patch has the following impacts on test development:
1. The Asterisk class now uses non-blocking operations to start/stop itself=
, and instead returns a deferred object that can be used to be notified whe=
n Asterisk has started/stopped. As a result, tests that derive from TestCa=
se now no longer need to call start_asterisk and stop_asterisk, as the star=
ting/stopping of Asterisk is handled by the TestCase class itself. Tests t=
hat do not use the TestCase object will need to add callbacks to the deferr=
ed object to ensure that Asterisk is fully started/stopped.
2. The Asterisk class's CLI command interfaces are all non-blocking, an=
d return a deferred object that signals when the command has completed.
3. SIPpScenario now returns a deferred when run, which signals when the sce=
nario has ended and its success/failure.
4. A new class, SIPpScenarioSequence has been added, which will run a serie=
s of SIPpScenario objects in a synchronous fashion. This eases the burden =
on people who were using SIPpScenario to run successive tests, as opposed t=
o tests in parallel.
5. The utils module has been renamed to TestSuiteUtils, to prevent a name c=
onflict with twisted's utils modules.
Note that we still have a number of tests that use pjsua and the subprocess=
module; as we currently have no library wrapping this usage I've defer=
red (pun intended) that work to another patch. The Lightweight NAT tests d=
o not use pjsua, and hence are not dependent on that work being finished to=
be included in the Test Suite.</pre>
</td>
</tr>
</table>
<h1 style=3D"color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b>=
</h1>
<ul style=3D"margin-left: 3em; padding-left: 0;">
<li>/asterisk/trunk/tests/udptl_v6/configs/ast1/manager.conf <span style=
=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/udptl_v6/configs/ast2/manager.conf <span style=
=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/udptl_v6/run-test <span style=3D"color: grey">(3=
104)</span></li>
<li>/asterisk/trunk/tests/mixmonitor/run-test <span style=3D"color: grey">=
(3104)</span></li>
<li>/asterisk/trunk/tests/mixmonitor_audiohook_inherit/run-test <span styl=
e=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-date/run-test <span style=3D"color: =
grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-datetime/run-test <span style=3D"col=
or: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-digits/run-test <span style=3D"color=
: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-number/run-test <span style=3D"color=
: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-phonetic/run-test <span style=3D"col=
or: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-time/run-test <span style=3D"color: =
grey">(3104)</span></li>
<li>/asterisk/trunk/tests/feature_attended_transfer/run-test <span style=
=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/feature_blonde_transfer/run-test <span style=3D"=
color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/iax2/basic-call/configs/ast1/extensions.conf <sp=
an style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/iax2/basic-call/configs/ast1/manager.conf <span =
style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/iax2/basic-call/configs/ast2/extensions.conf <sp=
an style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/iax2/basic-call/configs/ast2/manager.conf <span =
style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/iax2/basic-call/run-test <span style=3D"color: g=
rey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-alpha/run-test <span style=3D"color:=
grey">(3104)</span></li>
<li>/asterisk/trunk/tests/dynamic-modules/test-config.yaml <span style=3D"=
color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/get-data/run-test <span style=3D"color: =
grey">(3104)</span></li>
<li>/asterisk/trunk/tests/fastagi/hangup/run-test <span style=3D"color: gr=
ey">(3104)</span></li>
<li>/asterisk/trunk/tests/dynamic-modules/run-test <span style=3D"color: g=
rey">(3104)</span></li>
<li>/asterisk/trunk/tests/chanspy/chanspy_w_mixmonitor/run-test <span styl=
e=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/chanspy/chanspy_barge/configs/ast1/manager.conf =
<span style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/chanspy/chanspy_barge/run-test <span style=3D"co=
lor: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_tls_call/configs/ast1/extension=
s.conf <span style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_tls_call/configs/ast2/extension=
s.conf <span style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_tls_register/run-test <span sty=
le=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/use_contact_from_200/run-test <span=
style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_register_domain_acl/run-test <s=
pan style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_hold/run-test <span style=3D"co=
lor: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/nat_supertest/test-config.yaml <spa=
n style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/realtime_nosipregs/run-test <span s=
tyle=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/realtime_sipregs/run-test <span sty=
le=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/nat_supertest/sipp/inject.csv <span=
style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/nat_supertest/run-test <span style=
=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/info_dtmf/run-test <span style=3D"c=
olor: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/handle_response_address_incomplete/=
run-test <span style=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/apps/incomplete/sip_incomplete/run-test <span st=
yle=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/func_vmcount/run-test <span style=
=3D"color: grey">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/utils.py <span style=3D"color: gre=
y">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/version.py <span style=3D"color: g=
rey">(3104)</span></li>
<li>/asterisk/trunk/runtests.py <span style=3D"color: grey">(3104)</span><=
/li>
<li>/asterisk/trunk/lib/python/asterisk/sippversion.py <span style=3D"colo=
r: grey">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/sipp.py <span style=3D"color: grey=
">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/cdr.py <span style=3D"color: grey"=
>(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/asterisk.py <span style=3D"color: =
grey">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestConfig.py <span style=3D"color=
: grey">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/TestCase.py <span style=3D"color: =
grey">(3104)</span></li>
<li>/asterisk/trunk/lib/python/asterisk/CDRTestCase.py <span style=3D"colo=
r: grey">(3104)</span></li>
</ul>
<p><a href=3D"https://reviewboard.asterisk.org/r/1821/diff/" style=3D"margi=
n-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>
--===============8336183712139067814==--
More information about the asterisk-dev
mailing list