[asterisk-dev] [Code Review] Rework the Asterisk Test Suite's version handling

opticron reviewboard at asterisk.org
Tue Jun 19 16:41:43 CDT 2012 

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/1982/#review6512
-----------------------------------------------------------

Ship it!


Given that we're not branching the testsuite right now, this looks good to go.

- opticron


On June 8, 2012, 3:56 p.m., Matt Jordan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviewboard.asterisk.org/r/1982/
> -----------------------------------------------------------
> 
> (Updated June 8, 2012, 3:56 p.m.)
> 
> 
> Review request for Asterisk Developers, Russell Bryant, Paul Belanger, and wdoekes.
> 
> 
> Summary
> -------
> 
> This patch updates the Asterisk Test Suite's handling of Asterisk versions.  Specifically, it does the following:
> 
> * Removes support for Asterisk versions 1.4 and 1.6.2.  While this is some what controversial, it makes version parsing slightly saner, as it doesn't have to handle the 1.6.x scheme of versions.  (1.4 was just a freebie in the removal)
> * Changes the internal representation of versions to essentially ignore the 'concept' number in the comparisons.  This is, after all, one of the reasons why we dropped the '1' in '1.10' in the first place - it adds no meaningful value.
> * Adds the concepts of 'features' to a version.  Features are not used in terms of comparisons, but instead are explicitly checked by things that want to determine if a particular test can run.  They are, however, parsed out of the version tag - so 10.6.0-digiumphones-rc1 is understood as being the first release candidate of 10.6.0 with a feature of digiumphones; 1.8.11-cert3 is understood as being the third release of 1.8.11 with a feature of cert, etc.
> * Completely re-done version string parsing.  I'm not sure if I made it uniquely better - our versions are pretty tough to parse in a sane fashion, what with team branch names, multiple numbering schemes, features, modifiers, and other iteration numbers sprinkled liberally throughout - but I gave it a shot.  It passes the unit tests (which were tweaked up slightly as well to test some things that I'm not sure we tested previously), and seems to work well.
> * Updated the minversion in all test-config.yaml to be a tag.  A minversion in a test is a tag.  It is not a branch.  (Unless you actually put branch-10, but that would be silly because then it would only ever be run for a checkout of a branch, and not for any tags - and why would that make sense?)  While this would still treat a minversion: 1.8 as a minimum version of tag 1.8.0.0, that nomenclature is unclear.  This updates it to be consistent and clear as to what a minimum version is.
> * Added a new option to test-config.yaml 'feature'.  If a test can only be run against digiumphones, you add 'digiumphones' as the feature.  If it can only be run against 'cert',  you add 'cert'.  If its run for both, just put 'cert', since that implies 'digiumphones'.  And there's an override (for now) to make Asterisk 11 run everything.
> 
> 
> This addresses bug ASTERISK-19818.
>     https://issues.asterisk.org/jira/browse/ASTERISK-19818
> 
> 
> Diffs
> -----
> 
>   /asterisk/trunk/README.txt 3259 
>   /asterisk/trunk/lib/python/asterisk/TestConfig.py 3259 
>   /asterisk/trunk/lib/python/asterisk/version.py 3259 
>   /asterisk/trunk/runtests.py 3259 
>   /asterisk/trunk/tests/agi/exit_status/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/bridge/bridge_baseline/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/bridge/bridge_transfer_callee/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/confbridge/confbridge_nominal/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/confbridge/confbridge_recording/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/directory_attendant_exit/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/directory_context_operator_exit/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/directory_operator_exit/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/incomplete/sip_incomplete/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/page/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/authenticate_extensions/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/authenticate_invalid_mailbox/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/authenticate_invalid_password/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/authenticate_nominal/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_callback/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_delete/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_dialout/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_envelope/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_hangup/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_with_prepend/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user_hangup/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_nominal/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_change_password/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_busy/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_name/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_temp/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_unavail/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_reply/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/check_voicemail_while_leaving_msg/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/func_vmcount/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/leave_voicemail_external_notification/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/leave_voicemail_nominal/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/leave_voicemail_priority/test-config.yaml 3259 
>   /asterisk/trunk/tests/apps/voicemail/play_message/test-config.yaml 3259 
>   /asterisk/trunk/tests/blind-transfer-parkingtimeout/test-config.yaml 3259 
>   /asterisk/trunk/tests/callparking/test-config.yaml 3259 
>   /asterisk/trunk/tests/callparking_retrieval/test-config.yaml 3259 
>   /asterisk/trunk/tests/cause_answered_elsewhere/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/app_dial_G_flag/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/app_queue/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/blind-transfer-accountcode/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/cdr_accountcode/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/cdr_unanswered_yes/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/cdr_userfield/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/console_dial_sip_answer/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/console_dial_sip_busy/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/console_dial_sip_congestion/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/console_dial_sip_transfer/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/console_fork_after_busy_forward/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/console_fork_before_dial/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/nocdr/test-config.yaml 3259 
>   /asterisk/trunk/tests/cdr/originate-cdr-disposition/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/custom_info/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/generic_ccss/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/handle_response_address_incomplete/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/handle_response_refer/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/hangupcause/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/info_dtmf/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/invite_no_totag/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_auth/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_auth_cust_hdr/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_disabled/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_from_call/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_mark_all_outbound/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_send_ami/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_unauth/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/message_unauth_from/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/nat_supertest/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/noload_res_srtp/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/noload_res_srtp_attempt_srtp/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/options/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/pcap_demo/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/realtime_nosipregs/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/realtime_sipregs/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/refer_replaces_to_self/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/secure_bridge_media/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_attended_transfer/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_attended_transfer_tcp/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_attended_transfer_v6/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_cause/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_channel_params/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_custom_presence/multiple_state_change/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_custom_presence/nominal_state_change/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_custom_presence/non_digium_state_change/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_custom_presence/resubscribe/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_one_legged_transfer/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_one_legged_transfer_v6/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_outbound_address/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_register/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/sip_srtp/test-config.yaml 3259 
>   /asterisk/trunk/tests/channels/SIP/use_contact_from_200/test-config.yaml 3259 
>   /asterisk/trunk/tests/chanspy/chanspy_barge/test-config.yaml 3259 
>   /asterisk/trunk/tests/chanspy/chanspy_w_mixmonitor/test-config.yaml 3259 
>   /asterisk/trunk/tests/connected_line/macro/test-config.yaml 3259 
>   /asterisk/trunk/tests/connected_line/subroutine/test-config.yaml 3259 
>   /asterisk/trunk/tests/dialplan/test-config.yaml 3259 
>   /asterisk/trunk/tests/directed_pickup/test-config.yaml 3259 
>   /asterisk/trunk/tests/dynamic-modules/test-config.yaml 3259 
>   /asterisk/trunk/tests/example/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/channel-status/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/connect/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/control-stream-file/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/database/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/execute/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/get-data/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/hangup/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/record-file/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-alpha/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-date/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-datetime/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-digits/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-number/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-phonetic/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/say-time/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/stream-file/test-config.yaml 3259 
>   /asterisk/trunk/tests/fastagi/wait-for-digit/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_g711_t38/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_mix1/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_mix2/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_mix3/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_mix4/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_native_t38/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_native_t38_ced/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_no_t38/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_t38_g711/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_timeout1/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_timeout2/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_timeout3/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_timeout4/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/gateway_timeout5/test-config.yaml 3259 
>   /asterisk/trunk/tests/fax/local_channel_t38_queryoption/test-config.yaml 3259 
>   /asterisk/trunk/tests/feature_attended_transfer/test-config.yaml 3259 
>   /asterisk/trunk/tests/func_presencestate/test-config.yaml 3259 
>   /asterisk/trunk/tests/func_srv/test-config.yaml 3259 
>   /asterisk/trunk/tests/iax2/basic-call/test-config.yaml 3259 
>   /asterisk/trunk/tests/manager/action-events-response/test-config.yaml 3259 
>   /asterisk/trunk/tests/manager/authlimit/test-config.yaml 3259 
>   /asterisk/trunk/tests/manager/authtimeout/test-config.yaml 3259 
>   /asterisk/trunk/tests/manager/login/test-config.yaml 3259 
>   /asterisk/trunk/tests/manager/response-time/test-config.yaml 3259 
>   /asterisk/trunk/tests/masquerade/test-config.yaml 3260 
>   /asterisk/trunk/tests/mixmonitor/test-config.yaml 3259 
>   /asterisk/trunk/tests/mixmonitor_audiohook_inherit/test-config.yaml 3259 
>   /asterisk/trunk/tests/one-step-parking/test-config.yaml 3259 
>   /asterisk/trunk/tests/pbx/call-files/test-config.yaml 3259 
>   /asterisk/trunk/tests/pbx/call-files2/test-config.yaml 3259 
>   /asterisk/trunk/tests/pbx/merge_contexts/test-config.yaml 3259 
>   /asterisk/trunk/tests/pbx/pbx_lua_background/test-config.yaml 3259 
>   /asterisk/trunk/tests/pbx/pbx_lua_goto/test-config.yaml 3259 
>   /asterisk/trunk/tests/queues/position_priority_maxlen/test-config.yaml 3259 
>   /asterisk/trunk/tests/queues/queue_baseline/test-config.yaml 3259 
>   /asterisk/trunk/tests/queues/queue_transfer_callee/test-config.yaml UNKNOWN 
>   /asterisk/trunk/tests/queues/ringinuse_and_pause/test-config.yaml 3259 
>   /asterisk/trunk/tests/queues/set_penalty/test-config.yaml 3259 
>   /asterisk/trunk/tests/redirecting/macro/test-config.yaml 3259 
>   /asterisk/trunk/tests/redirecting/subroutine/test-config.yaml 3259 
>   /asterisk/trunk/tests/regressions/M18882/test-config.yaml 3259 
>   /asterisk/trunk/tests/rfc2833_dtmf_detect/test-config.yaml 3259 
>   /asterisk/trunk/tests/skeleton_test/test-config.yaml 3259 
>   /asterisk/trunk/tests/udptl/test-config.yaml 3259 
>   /asterisk/trunk/tests/udptl_v6/test-config.yaml 3259 
> 
> Diff: https://reviewboard.asterisk.org/r/1982/diff
> 
> 
> Testing
> -------
> 
> Ran with:
> 1.8 branch
> 10 branch
> 10-digiumphones branch
> 1.8.11-cert3 tag
> trunk
> some team branch of mine
> 
> The tests that were supposed to run against a version did; those that weren't, didn't.
> 
> 
> Thanks,
> 
> Matt
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20120619/944eee3a/attachment-0001.htm>

More information about the asterisk-dev mailing list