[asterisk-dev] [Code Review] Rework the Asterisk Test Suite's version handling

Matt Jordan reviewboard at asterisk.org
Fri Jun 8 15:56:04 CDT 2012 

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviewboard.asterisk.org/r/1982/
-----------------------------------------------------------

Review request for Asterisk Developers, Russell Bryant, Paul Belanger, and wdoekes.


Summary
-------

This patch updates the Asterisk Test Suite's handling of Asterisk versions.  Specifically, it does the following:

* Removes support for Asterisk versions 1.4 and 1.6.2.  While this is some what controversial, it makes version parsing slightly saner, as it doesn't have to handle the 1.6.x scheme of versions.  (1.4 was just a freebie in the removal)
* Changes the internal representation of versions to essentially ignore the 'concept' number in the comparisons.  This is, after all, one of the reasons why we dropped the '1' in '1.10' in the first place - it adds no meaningful value.
* Adds the concepts of 'features' to a version.  Features are not used in terms of comparisons, but instead are explicitly checked by things that want to determine if a particular test can run.  They are, however, parsed out of the version tag - so 10.6.0-digiumphones-rc1 is understood as being the first release candidate of 10.6.0 with a feature of digiumphones; 1.8.11-cert3 is understood as being the third release of 1.8.11 with a feature of cert, etc.
* Completely re-done version string parsing.  I'm not sure if I made it uniquely better - our versions are pretty tough to parse in a sane fashion, what with team branch names, multiple numbering schemes, features, modifiers, and other iteration numbers sprinkled liberally throughout - but I gave it a shot.  It passes the unit tests (which were tweaked up slightly as well to test some things that I'm not sure we tested previously), and seems to work well.
* Updated the minversion in all test-config.yaml to be a tag.  A minversion in a test is a tag.  It is not a branch.  (Unless you actually put branch-10, but that would be silly because then it would only ever be run for a checkout of a branch, and not for any tags - and why would that make sense?)  While this would still treat a minversion: 1.8 as a minimum version of tag 1.8.0.0, that nomenclature is unclear.  This updates it to be consistent and clear as to what a minimum version is.
* Added a new option to test-config.yaml 'feature'.  If a test can only be run against digiumphones, you add 'digiumphones' as the feature.  If it can only be run against 'cert',  you add 'cert'.  If its run for both, just put 'cert', since that implies 'digiumphones'.  And there's an override (for now) to make Asterisk 11 run everything.


This addresses bug ASTERISK-19818.
    https://issues.asterisk.org/jira/browse/ASTERISK-19818


Diffs
-----

  /asterisk/trunk/README.txt 3259 
  /asterisk/trunk/lib/python/asterisk/TestConfig.py 3259 
  /asterisk/trunk/lib/python/asterisk/version.py 3259 
  /asterisk/trunk/runtests.py 3259 
  /asterisk/trunk/tests/agi/exit_status/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/bridge/bridge_baseline/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/bridge/bridge_transfer_callee/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/confbridge/confbridge_nominal/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/confbridge/confbridge_recording/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/directory_attendant_exit/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/directory_context_operator_exit/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/directory_operator_exit/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/incomplete/sip_incomplete/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/page/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/authenticate_extensions/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/authenticate_invalid_mailbox/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/authenticate_invalid_password/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/authenticate_nominal/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_callback/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_delete/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_dialout/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_envelope/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_hangup/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_forward_with_prepend/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_new_user_hangup/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_nominal/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_change_password/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_busy/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_name/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_temp/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_options_record_unavail/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_reply/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/check_voicemail_while_leaving_msg/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/func_vmcount/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_external_notification/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_nominal/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/leave_voicemail_priority/test-config.yaml 3259 
  /asterisk/trunk/tests/apps/voicemail/play_message/test-config.yaml 3259 
  /asterisk/trunk/tests/blind-transfer-parkingtimeout/test-config.yaml 3259 
  /asterisk/trunk/tests/callparking/test-config.yaml 3259 
  /asterisk/trunk/tests/callparking_retrieval/test-config.yaml 3259 
  /asterisk/trunk/tests/cause_answered_elsewhere/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/app_dial_G_flag/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/app_queue/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/blind-transfer-accountcode/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/cdr_accountcode/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/cdr_unanswered_yes/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/cdr_userfield/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/console_dial_sip_answer/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/console_dial_sip_busy/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/console_dial_sip_congestion/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/console_dial_sip_transfer/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/console_fork_after_busy_forward/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/console_fork_before_dial/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/nocdr/test-config.yaml 3259 
  /asterisk/trunk/tests/cdr/originate-cdr-disposition/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/custom_info/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/generic_ccss/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/handle_response_address_incomplete/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/handle_response_refer/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/hangupcause/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/info_dtmf/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/invite_no_totag/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_auth/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_auth_cust_hdr/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_disabled/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_from_call/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_mark_all_outbound/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_send_ami/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_unauth/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/message_unauth_from/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/nat_supertest/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/noload_res_srtp/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/noload_res_srtp_attempt_srtp/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/options/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/pcap_demo/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/realtime_nosipregs/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/realtime_sipregs/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/refer_replaces_to_self/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/secure_bridge_media/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_attended_transfer/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_attended_transfer_tcp/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_attended_transfer_v6/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_cause/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_channel_params/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_custom_presence/multiple_state_change/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_custom_presence/nominal_state_change/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_custom_presence/non_digium_state_change/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_custom_presence/resubscribe/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_one_legged_transfer/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_one_legged_transfer_v6/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_outbound_address/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_register/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/sip_srtp/test-config.yaml 3259 
  /asterisk/trunk/tests/channels/SIP/use_contact_from_200/test-config.yaml 3259 
  /asterisk/trunk/tests/chanspy/chanspy_barge/test-config.yaml 3259 
  /asterisk/trunk/tests/chanspy/chanspy_w_mixmonitor/test-config.yaml 3259 
  /asterisk/trunk/tests/connected_line/macro/test-config.yaml 3259 
  /asterisk/trunk/tests/connected_line/subroutine/test-config.yaml 3259 
  /asterisk/trunk/tests/dialplan/test-config.yaml 3259 
  /asterisk/trunk/tests/directed_pickup/test-config.yaml 3259 
  /asterisk/trunk/tests/dynamic-modules/test-config.yaml 3259 
  /asterisk/trunk/tests/example/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/channel-status/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/connect/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/control-stream-file/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/database/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/execute/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/get-data/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/hangup/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/record-file/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-alpha/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-date/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-datetime/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-digits/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-number/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-phonetic/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/say-time/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/stream-file/test-config.yaml 3259 
  /asterisk/trunk/tests/fastagi/wait-for-digit/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_g711_t38/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_mix1/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_mix2/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_mix3/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_mix4/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_native_t38/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_native_t38_ced/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_no_t38/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_t38_g711/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_timeout1/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_timeout2/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_timeout3/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_timeout4/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/gateway_timeout5/test-config.yaml 3259 
  /asterisk/trunk/tests/fax/local_channel_t38_queryoption/test-config.yaml 3259 
  /asterisk/trunk/tests/feature_attended_transfer/test-config.yaml 3259 
  /asterisk/trunk/tests/func_presencestate/test-config.yaml 3259 
  /asterisk/trunk/tests/func_srv/test-config.yaml 3259 
  /asterisk/trunk/tests/iax2/basic-call/test-config.yaml 3259 
  /asterisk/trunk/tests/manager/action-events-response/test-config.yaml 3259 
  /asterisk/trunk/tests/manager/authlimit/test-config.yaml 3259 
  /asterisk/trunk/tests/manager/authtimeout/test-config.yaml 3259 
  /asterisk/trunk/tests/manager/login/test-config.yaml 3259 
  /asterisk/trunk/tests/manager/response-time/test-config.yaml 3259 
  /asterisk/trunk/tests/masquerade/test-config.yaml 3260 
  /asterisk/trunk/tests/mixmonitor/test-config.yaml 3259 
  /asterisk/trunk/tests/mixmonitor_audiohook_inherit/test-config.yaml 3259 
  /asterisk/trunk/tests/one-step-parking/test-config.yaml 3259 
  /asterisk/trunk/tests/pbx/call-files/test-config.yaml 3259 
  /asterisk/trunk/tests/pbx/call-files2/test-config.yaml 3259 
  /asterisk/trunk/tests/pbx/merge_contexts/test-config.yaml 3259 
  /asterisk/trunk/tests/pbx/pbx_lua_background/test-config.yaml 3259 
  /asterisk/trunk/tests/pbx/pbx_lua_goto/test-config.yaml 3259 
  /asterisk/trunk/tests/queues/position_priority_maxlen/test-config.yaml 3259 
  /asterisk/trunk/tests/queues/queue_baseline/test-config.yaml 3259 
  /asterisk/trunk/tests/queues/queue_transfer_callee/test-config.yaml UNKNOWN 
  /asterisk/trunk/tests/queues/ringinuse_and_pause/test-config.yaml 3259 
  /asterisk/trunk/tests/queues/set_penalty/test-config.yaml 3259 
  /asterisk/trunk/tests/redirecting/macro/test-config.yaml 3259 
  /asterisk/trunk/tests/redirecting/subroutine/test-config.yaml 3259 
  /asterisk/trunk/tests/regressions/M18882/test-config.yaml 3259 
  /asterisk/trunk/tests/rfc2833_dtmf_detect/test-config.yaml 3259 
  /asterisk/trunk/tests/skeleton_test/test-config.yaml 3259 
  /asterisk/trunk/tests/udptl/test-config.yaml 3259 
  /asterisk/trunk/tests/udptl_v6/test-config.yaml 3259 

Diff: https://reviewboard.asterisk.org/r/1982/diff


Testing
-------

Ran with:
1.8 branch
10 branch
10-digiumphones branch
1.8.11-cert3 tag
trunk
some team branch of mine

The tests that were supposed to run against a version did; those that weren't, didn't.


Thanks,

Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-dev/attachments/20120608/8b245c7a/attachment-0001.htm>

More information about the asterisk-dev mailing list