<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="8" style="border: 1px #c9c399 solid;">
<tr>
<td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://reviewboard.asterisk.org/r/1791/">https://reviewboard.asterisk.org/r/1791/</a>
</td>
</tr>
</table>
<br />
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="8" style="background-image: url('https://reviewboard.asterisk.org/media/rb/images/review_request_box_top_bg.png'); background-position: left top; background-repeat: repeat-x; border: 1px black solid;">
<tr>
<td>
<div>Review request for Asterisk Developers, Paul Belanger, wdoekes, and opticron.</div>
<div>By Matt Jordan.</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: 1px solid #b8b5a0">
<tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: -moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word;">In review 1771 (https://reviewboard.asterisk.org/r/1771), wdoekes modified the -t option in the Test Suite to only run a single test, as opposed to all tests that matched the parameter value. While this was the correct behavior for that option, it did prevent running sets of tests (for example, by specifying "tests/channels/SIP"). Fortunately, the Test Suite has the ability to run sets of tests using the "tags" option, although most of the tests (with the exception of the fax tests) were not taking advantage of this capability.
The tags feature allows each test to have zero or more tags associated with it, where if tags are associated with a test and specified as parameters to runtests.py, determine if it can run. All tags specified on the command line must be matched in order for the test to run.
As an example, if Test A specifies:
tags:
- SIP
- fax
Then executing runtests.py with "-g SIP" will run the test, as will "-g SIP -g fax"; however, "-g SIP -g CDR" would not.
This patch does the following:
1) It tags the vast majority of tests with a default set of tags. Tests that didn't really belong in any group with any other test were left alone (func_srv, for example), as having tests that can be run standalone using the -t option exist in a group with one member didn't make a lot of sense.
2) It adds an option to runtests.py to list all available tags (-L). The current output of this option is:
Available test tags:
AGI AMI apps
CDR chan_local chanspy
confbridge dial dialplan
dialplan_lua directory DTMF
fax fax_gateway fax_passthrough
features IAX incomplete
mixmonitor parking pickup
queues SIP transfer
voicemail
3) It displays the value of a test configuration's tags field in a number of places, including the -l (list tests) option, as well as when a test is skipped
Additional tags could certainly be specified and added for many of the tests - this patch is merely a first effort at applying basic, useful tags across all tests. It is expected that when new tests are added, part of the criteria for reviewing that test should be to check whether or not they are tagged appropriately.</pre>
</td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>/asterisk/trunk/lib/python/asterisk/TestConfig.py <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/runtests.py <span style="color: grey">(3076)</span></li>
<li>/asterisk/trunk/tests/agi/exit_status/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/confbridge/confbridge_nominal/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/directory_attendant_exit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/directory_context_operator_exit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/directory_operator_exit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/incomplete/sip_incomplete/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/authenticate_extensions/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/authenticate_invalid_mailbox/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/authenticate_invalid_password/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/authenticate_nominal/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_callback/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_delete/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_dialout/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_envelope/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_forward/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_hangup/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_with_prepend/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user_hangup/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_nominal/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_options_change_password/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_busy/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_name/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_temp/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_unavail/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_reply/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/check_voicemail_while_leaving_msg/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/func_vmcount/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_contexts/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_external_notification/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_forwarding/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_forwarding_auto_urgent/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_nominal/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/apps/voicemail/leave_voicemail_priority/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/blind-transfer-parkingtimeout/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/callparking/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/callparking_retrieval/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/app_dial_G_flag/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/app_queue/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/blind-transfer-accountcode/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/cdr_originate_sip_congestion_log/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/cdr_unanswered_yes/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/cdr_userfield/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/console_dial_sip_answer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/console_dial_sip_busy/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/console_dial_sip_congestion/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/console_dial_sip_transfer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/console_fork_after_busy_forward/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/console_fork_before_dial/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/nocdr/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/cdr/originate-cdr-disposition/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/codec_negotiation/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/handle_response_address_incomplete/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/handle_response_refer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/header_parsing/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/info_dtmf/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/message_auth/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/message_auth_cust_hdr/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/message_disabled/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/message_from_call/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/message_unauth/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/message_unauth_from/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/nat_supertest/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/noload_res_srtp/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/noload_res_srtp_attempt_srtp/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/options/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/pcap_demo/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/realtime_nosipregs/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/realtime_sipregs/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/refer_replaces_to_self/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/secure_bridge_media/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_attended_transfer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_attended_transfer_tcp/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_attended_transfer_v6/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_blind_transfer/callee_refer_only/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_blind_transfer/callee_with_reinvite/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_blind_transfer/caller_refer_only/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_blind_transfer/caller_with_reinvite/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_channel_params/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_hold/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_one_legged_transfer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_one_legged_transfer_v6/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_outbound_address/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_register/test-config.yaml <span style="color: grey">(3070)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_register_domain_acl/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_srtp/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_tls_call/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/sip_tls_register/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/tcpauthlimit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/tcpauthtimeout/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/channels/SIP/use_contact_from_200/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/chanspy/chanspy_barge/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/chanspy/chanspy_w_mixmonitor/test-config.yaml <span style="color: grey">(3076)</span></li>
<li>/asterisk/trunk/tests/dialplan/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/directed_pickup/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/channel-status/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/connect/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/control-stream-file/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/database/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/execute/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/get-data/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/hangup/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/record-file/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-alpha/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-date/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-datetime/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-digits/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-number/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-phonetic/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/say-time/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/stream-file/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/fastagi/wait-for-digit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/feature_attended_transfer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/feature_blonde_transfer/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/iax2/basic-call/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/manager/action-events-response/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/manager/authlimit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/manager/authtimeout/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/manager/login/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/manager/response-time/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/mixmonitor/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/mixmonitor_audiohook_inherit/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/one-step-parking/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/pbx/call-files/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/pbx/call-files2/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/pbx/merge_contexts/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/pbx/pbx_lua_background/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/pbx/pbx_lua_goto/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/queues/macro_gosub_test/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/queues/position_priority_maxlen/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/queues/queue_baseline/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/queues/ringinuse_and_pause/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/queues/set_penalty/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/queues/wrapup_time/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/regressions/M18882/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/rfc2833_dtmf_detect/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/udptl/test-config.yaml <span style="color: grey">(3064)</span></li>
<li>/asterisk/trunk/tests/udptl_v6/test-config.yaml <span style="color: grey">(3064)</span></li>
</ul>
<p><a href="https://reviewboard.asterisk.org/r/1791/diff/" style="margin-left: 3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>