[Asterisk-code-review] lib/python/asterisk/pcap/py Replace construct_legacy.protocols (testsuite[13])
Friendly Automation
asteriskteam at digium.com
Mon Dec 9 13:34:46 CST 2019
Friendly Automation has submitted this change. ( https://gerrit.asterisk.org/c/testsuite/+/13222 )
Change subject: lib/python/asterisk/pcap/py Replace construct_legacy.protocols
......................................................................
lib/python/asterisk/pcap/py Replace construct_legacy.protocols
The "protocols" module provides parsing for PCAP packets
layers 2-4 (Ethernet, IPv4/IPv6, UDP/TCP). The
Testsuite uses it to parse packets for the tests that
need that functionality.
It was originally part of the "construct" module but was removed
after v2.5.5. Unfortunately, it was and is the only
standalone/low overhead packet parsing module available.
This python package is an extract from the construct package v2.5.5.
Since construct itself underwent major API changes since v2.5.5,
the protocols code needed significant work to make it compatible
with the current construct version (2.9 at the time of this writing).
Since no functional changes were made, only API compatibility
changes, the original construct (license)[LICENSE.md] still
applies and is included here.
* Ported construct.protocols to construct 2.9 and added it
to lib/python.
* Updated pcap.py...
** Using /lib/python/PcapListener directly instead of through
test_case to make it a bit less complicated and to facilitate
standalone unit tests.
** Refactored for construct 2.9
** Removed references to construct.protcols
** Added a unit test.
cd <testsuite_dir>/lib/python/asterisk
PYTHONPATH=<testsuite_dir>/lib/python python2 pcap.py --help
* Updated tests that used construct_legacy to work with
construct 2.9
Change-Id: Id38d01a2cd073b240fde909a38c95d69313bbbe7
---
M lib/python/asterisk/pcap.py
M lib/python/asterisk/test_case.py
A lib/python/protocols/LICENSE.md
A lib/python/protocols/README.md
A lib/python/protocols/__init__.py
A lib/python/protocols/application/__init__.py
A lib/python/protocols/ipstack.py
A lib/python/protocols/layer2/__init__.py
A lib/python/protocols/layer2/arp.py
A lib/python/protocols/layer2/ethernet.py
A lib/python/protocols/layer2/mtp2.py
A lib/python/protocols/layer3/__init__.py
A lib/python/protocols/layer3/ipv4.py
A lib/python/protocols/layer3/ipv6.py
A lib/python/protocols/layer4/__init__.py
A lib/python/protocols/layer4/tcp.py
A lib/python/protocols/layer4/udp.py
A lib/python/protocols/unconverted/application/dns.py
A lib/python/protocols/unconverted/layer3/dhcpv4.py
A lib/python/protocols/unconverted/layer3/dhcpv6.py
A lib/python/protocols/unconverted/layer3/icmpv4.py
A lib/python/protocols/unconverted/layer3/igmpv2.py
A lib/python/protocols/unconverted/layer3/mtp3.py
A lib/python/protocols/unconverted/layer4/isup.py
M tests/channels/SIP/pcap_demo/run-test
M tests/channels/SIP/pcap_demo/test-config.yaml
M tests/channels/pjsip/rtp/rtp_keepalive/base/rtp.py
M tests/hep/hep_capture_node.py
28 files changed, 1,354 insertions(+), 144 deletions(-)
Approvals:
Kevin Harwell: Looks good to me, but someone else must approve
Joshua Colp: Looks good to me, but someone else must approve
George Joseph: Looks good to me, approved
Friendly Automation: Approved for Submit
diff --git a/lib/python/asterisk/pcap.py b/lib/python/asterisk/pcap.py
index cb5fa5c..536a7ca 100644
--- a/lib/python/asterisk/pcap.py
+++ b/lib/python/asterisk/pcap.py
@@ -13,18 +13,16 @@
import sys
import logging
+import signal
+import argparse
import binascii
sys.path.append('lib/python')
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
-try:
- from construct_legacy import *
- from construct_legacy.protocols.ipstack import ip_stack
-except ImportError:
- from construct import *
- from construct.protocols.ipstack import ip_stack
+from construct import *
+from construct.core import *
try:
from yappcap import PcapOffline
PCAP_AVAILABLE = True
@@ -33,6 +31,9 @@
import rlmi
+from protocols.ipstack import ip_stack
+from pcap_listener import PcapListener as PacketCapturer
+
LOGGER = logging.getLogger(__name__)
@@ -59,18 +60,13 @@
filename = module_config.get('filename')
snaplen = module_config.get('snaplen')
buffer_size = module_config.get('buffer-size')
- if (module_config.get('register-observer')):
- test_object.register_pcap_observer(self.__pcap_callback)
self.debug_packets = module_config.get('debug-packets', False)
# Let exceptions propagate - if we can't create the pcap, this should
# throw the exception to the pluggable module creation routines
- test_object.create_pcap_listener(
- device=device,
- bpf_filter=bpf_filter,
- dumpfile=filename,
- snaplen=snaplen,
- buffer_size=buffer_size)
+
+ PacketCapturer(device, bpf_filter, filename, self.__pcap_callback,
+ snaplen, buffer_size)
def __pcap_callback(self, packet):
"""Private callback that logs packets if the configuration supports it
@@ -108,7 +104,6 @@
packet_type A text string describing what type of packet this is
raw_packet The bytes comprising the packet
"""
-
self.packet_type = packet_type
self.raw_packet = raw_packet
if isinstance(self.raw_packet, str):
@@ -154,43 +149,45 @@
self.__parse_raw_data(self.transport_layer.next)
def __parse_raw_data(self, binary_blob):
- header_def = Struct('rtcp_header',
- BitStruct('header',
- BitField('version', 2),
+ header_def = 'rtcp_header' / BitStruct(
+ 'header' / BitStruct(
+ 'version' / BitsInteger(2),
Padding(1),
- BitField('reception_report_count', 5),
+ 'reception_report_count' / BitsInteger(5),
),
- UBInt8('packet_type'),
- UBInt16('length'),
- UBInt32('ssrc'))
+ 'packet_type' / Bytewise(Int8ub),
+ 'length' / Bytewise(Int16ub),
+ 'ssrc' / Bytewise(Int32ub)
+ )
self.rtcp_header = header_def.parse(binary_blob)
+
report_block_def = GreedyRange(
- Struct(
- 'report_block',
- UBInt32('ssrc'),
- BitStruct(
- 'lost_counts',
- BitField('fraction_lost', 8),
- BitField('packets_lost', 24)),
- UBInt32('sequence_number_received'),
- UBInt32('interarrival_jitter'),
- UBInt32('last_sr'),
- UBInt32('delay_last_sr')
+ 'report_block' / BitStruct(
+ 'ssrc' / Bytewise(Int32ub),
+ 'lost_counts' / Struct(
+ 'fraction_lost' / BitsInteger(8),
+ 'packets_lost' / BitsInteger(24)
+ ),
+ 'sequence_number_received' / Bytewise(Int32ub),
+ 'interarrival_jitter' / Bytewise(Int32ub),
+ 'last_sr' / Bytewise(Int32ub),
+ 'delay_last_sr' / Bytewise(Int32ub)
)
)
if self.rtcp_header.packet_type == 200:
sender_def = Struct('sr',
- Struct('sender_info',
- UBInt32('ntp_msw'),
- UBInt32('ntp_lsw'),
- UBInt32('rtp_timestamp'),
- UBInt32('sender_packet_count'),
- UBInt32('sender_octet_count')),
+ 'sender_info' / Struct(
+ 'ntp_msw' / Int32ub,
+ 'ntp_lsw' / Int32ub,
+ 'rtp_timestamp' / Int32ub,
+ 'sender_packet_count' / Int32ub,
+ 'sender_octet_count' / Int32ub
+ ),
report_block_def)
self.sender_report = sender_def.parse(binary_blob[8:])
elif self.rtcp_header.packet_type == 201:
- receiver_def = Struct('rr',
+ receiver_def = 'rr' / Struct(
report_block_def)
self.receiver_report = receiver_def.parse(binary_blob[8:])
@@ -205,19 +202,16 @@
"""An RTP Packet
"""
- rtp_header = Struct(
- "rtp_header",
- EmbeddedBitStruct(
- BitField("version", 2),
- Bit("padding"),
- Bit("extension"),
- Nibble("csrc_count"),
- Bit("marker"),
- BitField("payload_type", 7),
- ),
- UBInt16("sequence_number"),
- UBInt32("timestamp"),
- UBInt32("ssrc"),
+ rtp_header = "rtp_header" / BitStruct(
+ "version" / BitsInteger(2),
+ "padding" / Bit,
+ "extension" / Bit,
+ "csrc_count" / Nibble,
+ "marker" / Bit,
+ "payload_type" / BitsInteger(7),
+ "sequence_number" / Bytewise(Int16ub),
+ "timestamp" / Bytewise(Int32ub),
+ "ssrc" / Bytewise(Int32ub),
# 'csrc' can be added later when needed
)
@@ -748,6 +742,7 @@
"""
(host, port) = addr
packet = self.packet_factory.interpret_packet(packet)
+
if packet is None:
return
@@ -760,10 +755,12 @@
if host not in self.traces:
self.traces[host] = []
self.traces[host].append(packet)
- if packet.packet_type not in self.callbacks:
- return
- for callback in self.callbacks[packet.packet_type]:
- callback(packet)
+ if packet.packet_type in self.callbacks:
+ for callback in self.callbacks[packet.packet_type]:
+ callback(packet)
+ if "*" in self.callbacks:
+ for callback in self.callbacks["*"]:
+ callback(packet)
def add_callback(self, packet_type, callback):
"""Add a callback function for received packets of a particular type
@@ -872,11 +869,16 @@
module_config The module configuration for this pluggable module
test_object The object we will attach to
"""
- if not 'register-observer' in module_config:
- raise Exception('VOIPListener needs register-observer to be set')
- VOIPSniffer.__init__(self, module_config, test_object)
- PcapListener.__init__(self, module_config, test_object)
+ VOIPSniffer.__init__(self, module_config, test_object)
+
+ packet_type = module_config.get("packet-type")
+ bpf = module_config.get("bpf-filter")
+
+ if packet_type:
+ self.add_callback(packet_type, module_config.get("callback"))
+
+ PcapListener.__init__(self, module_config, test_object)
def pcap_callback(self, packet):
"""Packet capture callback function
@@ -888,5 +890,49 @@
Keyword Arguments:
packet A received packet from the pcap listener
"""
+
self.process_packet(packet, (None, None))
+
+
+# This is a unit test for capture and parsing.
+# By default it listens on the loopback interface
+# UDP port 5060.
+
+if __name__ == "__main__":
+
+ def callback(packet):
+ print(packet)
+
+ def signal_handler(sig, frame):
+ print('You pressed Ctrl+C!')
+ reactor.stop()
+
+ signal.signal(signal.SIGINT, signal_handler)
+
+ parser = argparse.ArgumentParser(description="pcap unit test")
+ parser.add_argument("-i", "--interface", metavar="i", action="store",
+ type=str, dest="interface", default="lo",
+ help="Interface to listen on")
+ parser.add_argument("-f", "--filter", metavar="f", action="store",
+ type=str, dest="filter", default="udp port 5060",
+ help="BPF Filter")
+ parser.add_argument("-o", "--output", metavar="o", action="store",
+ type=str, dest="output", default="/tmp/pcap_unit_test.pcap",
+ help="Output file")
+ options = parser.parse_args()
+ print('Listening on "%s", using filter "%s", capturing to "%s"'
+ % (options.interface, options.filter, options.output))
+
+ module_config = {
+ "device": options.interface,
+ "bpf-filter": options.filter,
+ "filename": options.output,
+ "snaplen": 65535,
+ "buffer-size": 0,
+ "callback": callback,
+ "debug-packets": True,
+ "packet-type": "*"
+ }
+ pcap = VOIPListener(module_config, None)
+ reactor.run()
diff --git a/lib/python/asterisk/test_case.py b/lib/python/asterisk/test_case.py
index 7bf6d16..1c7edfa 100644
--- a/lib/python/asterisk/test_case.py
+++ b/lib/python/asterisk/test_case.py
@@ -138,7 +138,6 @@
self._stop_callbacks = []
self._ami_connect_callbacks = []
self._ami_reconnect_callbacks = []
- self._pcap_callbacks = []
self._stop_deferred = None
log_full = True
log_messages = True
@@ -396,7 +395,7 @@
# tests can create their own. Tests may only want to watch a specific
# port, while a general logger will want to watch more general traffic
# which can be filtered later.
- return PcapListener(device, bpf_filter, dumpfile, self._pcap_callback,
+ return PcapListener(device, bpf_filter, dumpfile, self.pcap_callback,
snaplen, buffer_size)
def start_asterisk(self):
@@ -622,12 +621,6 @@
"""Virtual method used to receive captured packets."""
pass
- def _pcap_callback(self, packet):
- """Packet capture callback"""
- self.pcap_callback(packet)
- for callback in self._pcap_callbacks:
- callback(packet)
-
def handle_originate_failure(self, reason):
"""Fail the test on an Originate failure
@@ -676,17 +669,6 @@
return self.passed
- def register_pcap_observer(self, callback):
- """Register an observer that will be called when a packet is received
- from a created pcap listener
-
- Keyword Arguments:
- callback The callback to receive the packet. The callback function
- should take in a single parameter, which will be the packet
- received
- """
- self._pcap_callbacks.append(callback)
-
def register_start_observer(self, callback):
"""Register an observer that will be called when all Asterisk instances
have started
diff --git a/lib/python/protocols/LICENSE.md b/lib/python/protocols/LICENSE.md
new file mode 100644
index 0000000..366ed6e
--- /dev/null
+++ b/lib/python/protocols/LICENSE.md
@@ -0,0 +1,24 @@
+```
+Copyright (C) 2006-2018
+ Arkadiusz Bulski (arek.bulski at gmail.com)
+ Tomer Filiba (tomerfiliba at gmail.com)
+ Corbin Simpson (MostAwesomeDude at gmail.com)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```
\ No newline at end of file
diff --git a/lib/python/protocols/README.md b/lib/python/protocols/README.md
new file mode 100644
index 0000000..171a5e1
--- /dev/null
+++ b/lib/python/protocols/README.md
@@ -0,0 +1,47 @@
+## protocols
+
+The "protocols" module provides parsing for PCAP packets
+layers 2-4 (Ethernet, IPv4/IPv6, UDP/TCP). The
+Testsuite uses it to parse packets for the tests that
+need that functionality.
+
+It was originally part of the
+[construct](https://github.com/construct/construct)
+module but was removed after v2.5.5. Unfortunately,
+it was and is the only standalone/low overhead packet parsing
+module available.
+
+This python package is an extract from the
+[construct](https://github.com/construct/construct)
+package v2.5.5. Since construct itself underwent
+major API changes since v2.5.5, the protocols
+code needed significant work to make it compatible
+with the current construct version (2.9 at the
+time of this writing). Since no functional changes
+were made, only API compatibility changes, the
+original construct (license)[LICENSE.md] still
+applies and is included here.
+
+This module is compatible with both python2 and
+python3. It also contains unit tests for every
+parser including the top-level ipstack wrapper.
+To run the unit tests, you must add
+`<testsuite_path>/lib/python` to the `PYTHONPATH`
+environment variable.
+
+Example
+```
+# cd /usr/src/asterisk/testsuite/lib/python
+# PYTHONPATH=/usr/src/asterisk/testsuite/lib/python python3
+./ipstack.py
+```
+Or...
+```
+# cd /usr/src/asterisk/testsuite/lib/python
+# PYTHONPATH=../ python3 ./ipstack.py
+```
+
+There is also an "unconverted" directory in this
+module which contains parsers we don't use and
+therefore haven't been updated wo work with
+the latest construct.
diff --git a/lib/python/protocols/__init__.py b/lib/python/protocols/__init__.py
new file mode 100644
index 0000000..0ec215e
--- /dev/null
+++ b/lib/python/protocols/__init__.py
@@ -0,0 +1,4 @@
+"""
+protocols - a collection of network protocols
+unlike the formats package, protocols convey information between two sides
+"""
diff --git a/lib/python/protocols/application/__init__.py b/lib/python/protocols/application/__init__.py
new file mode 100644
index 0000000..7ea61f7
--- /dev/null
+++ b/lib/python/protocols/application/__init__.py
@@ -0,0 +1,4 @@
+"""
+application layer (various) protocols
+"""
+
diff --git a/lib/python/protocols/ipstack.py b/lib/python/protocols/ipstack.py
new file mode 100644
index 0000000..cd63fce
--- /dev/null
+++ b/lib/python/protocols/ipstack.py
@@ -0,0 +1,146 @@
+"""
+TCP/IP Protocol Stack
+Note: before parsing the application layer over a TCP stream, you must
+first combine all the TCP frames into a stream. See utils.tcpip for
+some solutions
+"""
+from binascii import unhexlify
+#import six
+from construct import Struct, HexDump, Switch, Pass, Computed, Hex, Bytes, setGlobalPrintFullStrings
+from protocols.layer2.ethernet import ethernet_header
+from protocols.layer3.ipv4 import ipv4_header
+from protocols.layer3.ipv6 import ipv6_header
+from protocols.layer4.tcp import tcp_header
+from protocols.layer4.udp import udp_header
+
+setGlobalPrintFullStrings(True)
+
+layer4_tcp = "layer4_tcp" / Struct(
+ "layer" / Computed(4),
+ "packet_type" / Computed("TCP"),
+ "header" / tcp_header,
+ "next" / Bytes(lambda ctx: ctx["_"]["header"].payload_length - ctx["header"].header_length)
+)
+
+layer4_udp = "layer4_udp" / Struct(
+ "layer" / Computed(4),
+ "packet_type" / Computed("UDP"),
+ "header" / udp_header,
+ "next" / Bytes(lambda ctx: ctx["header"].payload_length)
+)
+
+layer3_payload = "next" / Switch(lambda ctx: ctx["header"].protocol,
+ {
+ "TCP" : layer4_tcp,
+ "UDP" : layer4_udp,
+ },
+ default = Pass
+)
+
+layer3_ipv4 = "layer3_ipv4" / Struct(
+ "layer" / Computed(3),
+ "packet_type" / Computed("IPv4"),
+ "header" / ipv4_header,
+ layer3_payload,
+)
+
+layer3_ipv6 = "layer3_ipv6" / Struct(
+ "layer" / Computed(3),
+ "packet_type" / Computed("IPv6"),
+ "header" / ipv6_header,
+ layer3_payload,
+)
+
+layer2_ethernet = "layer2_ethernet" / Struct(
+ "layer" / Computed(2),
+ "packet_type" / Computed("ETHERNET"),
+ "header" / ethernet_header,
+ "next" / Switch(lambda ctx: ctx["header"].type,
+ {
+ "IPv4" : layer3_ipv4,
+ "IPv6" : layer3_ipv6,
+ },
+ default = Pass,
+ )
+)
+
+ip_stack = "ip_stack" / layer2_ethernet
+
+
+if __name__ == "__main__":
+ cap1 = unhexlify(
+ "0011508c283c001150886b570800450001e971474000800684e4c0a80202525eedda11"
+ "2a0050d98ec61d54fe977d501844705dcc0000474554202f20485454502f312e310d0a"
+ "486f73743a207777772e707974686f6e2e6f72670d0a557365722d4167656e743a204d"
+ "6f7a696c6c612f352e30202857696e646f77733b20553b2057696e646f7773204e5420"
+ "352e313b20656e2d55533b2072763a312e382e302e3129204765636b6f2f3230303630"
+ "3131312046697265666f782f312e352e302e310d0a4163636570743a20746578742f78"
+ "6d6c2c6170706c69636174696f6e2f786d6c2c6170706c69636174696f6e2f7868746d"
+ "6c2b786d6c2c746578742f68746d6c3b713d302e392c746578742f706c61696e3b713d"
+ "302e382c696d6167652f706e672c2a2f2a3b713d302e350d0a4163636570742d4c616e"
+ "67756167653a20656e2d75732c656e3b713d302e350d0a4163636570742d456e636f64"
+ "696e673a20677a69702c6465666c6174650d0a4163636570742d436861727365743a20"
+ "49534f2d383835392d312c7574662d383b713d302e372c2a3b713d302e370d0a4b6565"
+ "702d416c6976653a203330300d0a436f6e6e656374696f6e3a206b6565702d616c6976"
+ "650d0a507261676d613a206e6f2d63616368650d0a43616368652d436f6e74726f6c3a"
+ "206e6f2d63616368650d0a0d0a"
+ )
+
+ cap2 = unhexlify(
+ "0002e3426009001150f2c280080045900598fd22000036063291d149baeec0a8023c00"
+ "500cc33b8aa7dcc4e588065010ffffcecd0000485454502f312e3120323030204f4b0d"
+ "0a446174653a204672692c2031352044656320323030362032313a32363a323520474d"
+ "540d0a5033503a20706f6c6963797265663d22687474703a2f2f7033702e7961686f6f"
+ "2e636f6d2f7733632f7033702e786d6c222c2043503d2243414f2044535020434f5220"
+ "4355522041444d20444556205441492050534120505344204956416920495644692043"
+ "4f4e692054454c6f204f545069204f55522044454c692053414d69204f54526920554e"
+ "5269205055426920494e4420504859204f4e4c20554e49205055522046494e20434f4d"
+ "204e415620494e542044454d20434e542053544120504f4c204845412050524520474f"
+ "56220d0a43616368652d436f6e74726f6c3a20707269766174650d0a566172793a2055"
+ "7365722d4167656e740d0a5365742d436f6f6b69653a20443d5f796c683d58336f444d"
+ "54466b64476c6f5a7a567842463954417a49334d5459784e446b4563476c6b417a4578"
+ "4e6a59794d5463314e5463456447567a64414d7742485274634777446157356b5a5867"
+ "7462412d2d3b20706174683d2f3b20646f6d61696e3d2e7961686f6f2e636f6d0d0a43"
+ "6f6e6e656374696f6e3a20636c6f73650d0a5472616e736665722d456e636f64696e67"
+ "3a206368756e6b65640d0a436f6e74656e742d547970653a20746578742f68746d6c3b"
+ "20636861727365743d7574662d380d0a436f6e74656e742d456e636f64696e673a2067"
+ "7a69700d0a0d0a366263382020200d0a1f8b0800000000000003dcbd6977db38b200fa"
+ "f9fa9cf90f88326dd9b1169212b5d891739cd84ed2936d1277a7d3cbf1a1484a624c91"
+ "0c4979893bbfec7d7bbfec556121012eb29d65e6be7be7762c9240a1502854150a85c2"
+ "c37b87af9f9c7c7873449e9dbc7c41defcf2f8c5f327a4d1ee76dff79e74bb872787ec"
+ "43bfa3e9ddeed1ab06692cd234daed762f2e2e3a17bd4e18cfbb276fbb8b74e9f7bb49"
+ "1a7b76da7152a7b1bff110dfed3f5cb896030f4b37b508566dbb9f56def9a4f1240c52"
+ "3748db275791db20367b9a3452f732a5d0f688bdb0e2c44d27bf9c1cb7470830b1632f"
+ "4a490a3578c18fd6b9c5dec2f7732b2641783109dc0b7268a56e2bd527a931497b93b4"
+ "3f49cd493a98a4c3493a9aa4e349aa6bf01f7cd78d89d6b2ed49b3d9baf223f8b307b5"
+ "004a67eea627ded2dddadedb78d8656de428f856305f5973779223b0fff05ebbbde1db"
+ "67082a499289ae0f06863e1c8f4c0639eaccbdd9a3547abf798a1f0ec6c73fafd2e4f1"
+ "51ffd5f1c9e2f9e37ff74e74fbddd941b375eadb0942b3e3d5723a69f6060373a6cff4"
+ "9e6df586dac8b11c4d1f1afd81319b0df45e6fd4925a6cee6db4dbfb19e225bc1b12e5"
+ "6a098aed9309715c3b74dc5fde3e7f122ea3308061dac22f4018a4f8878367af5f4f2e"
+ "bcc001a2d187bfffbefeb2477f75026be9269165bb93d92ab0532f0cb68264fbda9b6d"
+ "dd0b92bfff867f3abe1bccd3c5f675eca6ab3820c1caf7f7be20e05363029f93c8f7d2"
+ "ad46a7b1bd475ff62614f2de2c8cb7f08537d93a35fed0fe9a4c1af44363fb91beabed"
+ "790f4f0d0e7a6f67c7dbbe3eedfd01e5bcbffe9a64bf289e00307bb1f7852371dadb13"
+ "3df0c3798efba9d93a1db44e87dbd7d8b4cf50e95c780e304be745389fbbf11ef4cddf"
+ "dcf4b162d629fa94d7defbe2fa892b3ece2c78d8fb221a84517003476a73dc3ad535d6"
+ "e22c7fbd0db8cf3a511ca6211d3e28933fed9d8ea54f381f66c0c7f2cb0e4c3898ad2b"
+ "3b0de3c9e918bf25abc88d6ddf02d65581418f94174addc9ebe94717e67ce557207b6d"
+ "45f892773ae393adc62af57c18ecd27b46e5aa2feea5b58c7c173e6d94be1d3bd5afa3"
+ "fcf571d409ded9b1eb06ef3d275d00c36f25f4916c6ed2a911cef88b0e4c0ecfa7a5b6"
+ "27936600b3d28d9bdbe411"
+ )
+
+ obj = ip_stack.parse(cap1)
+ print (obj)
+ # Print just the payload
+ print (obj.next.next.next)
+ built = ip_stack.build(obj)
+ assert built == cap1
+
+ print ("-" * 80)
+
+ obj = ip_stack.parse(cap2)
+ print (obj.next.next.header)
+ built = ip_stack.build(obj)
+ assert built == cap2
diff --git a/lib/python/protocols/layer2/__init__.py b/lib/python/protocols/layer2/__init__.py
new file mode 100644
index 0000000..bdcdb4a
--- /dev/null
+++ b/lib/python/protocols/layer2/__init__.py
@@ -0,0 +1,4 @@
+"""
+layer 2 (data link) protocols
+"""
+
diff --git a/lib/python/protocols/layer2/arp.py b/lib/python/protocols/layer2/arp.py
new file mode 100644
index 0000000..a88acfa
--- /dev/null
+++ b/lib/python/protocols/layer2/arp.py
@@ -0,0 +1,103 @@
+"""
+Ethernet (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+
+from construct import *
+from construct.core import *
+from protocols.layer3.ipv4 import IpAddress
+from ethernet import MacAddress
+
+'''
+def HwAddress(name):
+ return IfThenElse(name, lambda ctx: ctx.hardware_type == "ETHERNET",
+ MacAddressAdapter(Field("data", lambda ctx: ctx.hwaddr_length)),
+ Field("data", lambda ctx: ctx.hwaddr_length)
+ )
+
+def ProtoAddress(name):
+ return IfThenElse(name, lambda ctx: ctx.protocol_type == "IP",
+ IpAddressAdapter(Field("data", lambda ctx: ctx.protoaddr_length)),
+ Field("data", lambda ctx: ctx.protoaddr_length)
+ )
+'''
+
+def HardwareTypeEnum(code):
+ return Enum(code,
+ ETHERNET = 1,
+ EXPERIMENTAL_ETHERNET = 2,
+ ProNET_TOKEN_RING = 4,
+ CHAOS = 5,
+ IEEE802 = 6,
+ ARCNET = 7,
+ HYPERCHANNEL = 8,
+ ULTRALINK = 13,
+ FRAME_RELAY = 15,
+ FIBRE_CHANNEL = 18,
+ IEEE1394 = 24,
+ HIPARP = 28,
+ ISO7816_3 = 29,
+ ARPSEC = 30,
+ IPSEC_TUNNEL = 31,
+ INFINIBAND = 32,
+ )
+
+def OpcodeEnum(code):
+ return Enum(code,
+ REQUEST = 1,
+ REPLY = 2,
+ REQUEST_REVERSE = 3,
+ REPLY_REVERSE = 4,
+ DRARP_REQUEST = 5,
+ DRARP_REPLY = 6,
+ DRARP_ERROR = 7,
+ InARP_REQUEST = 8,
+ InARP_REPLY = 9,
+ ARP_NAK = 10
+ )
+
+arp_header = "arp_header" / Struct(
+ "hardware_type" / HardwareTypeEnum(Int16ub),
+ "protocol_type" / Enum(Int16ub,
+ IP = 0x0800,
+ ),
+ "hwaddr_length" / Int8ub,
+ "protoaddr_length" / Int8ub,
+ "opcode" / OpcodeEnum(Int16ub),
+ "source_hwaddr" / MacAddress(),
+ "source_protoaddr" / IpAddress(),
+ "dest_hwaddr" / MacAddress(),
+ "dest_protoaddr" / IpAddress(),
+)
+
+rarp_header = "rarp_header" / arp_header
+
+if __name__ == "__main__":
+ cap1 = unhexlify(b"00010800060400010002e3426009c0a80204000000000000c0a80201")
+ obj = arp_header.parse(cap1)
+ print (obj)
+ built = arp_header.build(obj)
+ print(repr(built))
+ assert built == cap1
+
+ print ("-" * 80)
+
+ cap2 = unhexlify(b"00010800060400020011508c283cc0a802010002e3426009c0a80204")
+ obj = arp_header.parse(cap2)
+ print (obj)
+ built = arp_header.build(obj)
+ print(repr(built))
+ assert built == cap2
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/lib/python/protocols/layer2/ethernet.py b/lib/python/protocols/layer2/ethernet.py
new file mode 100644
index 0000000..15a52aa
--- /dev/null
+++ b/lib/python/protocols/layer2/ethernet.py
@@ -0,0 +1,40 @@
+"""
+Ethernet (TCP/IP protocol stack)
+"""
+from binascii import hexlify, unhexlify
+
+from construct import *
+from construct.core import *
+
+
+class MacAddressAdapter(Adapter):
+ def _encode(self, obj, context, path):
+ return unhexlify(obj.replace("-", ""))
+ def _decode(self, obj, context, path):
+ return "-".join(hexlify(b) for b in obj)
+
+def MacAddress():
+ return MacAddressAdapter(Bytes(6))
+
+ethernet_header = "ethernet_header" / Struct(
+ "destination" / MacAddress(),
+ "source" / MacAddress(),
+ "type" / Enum(Int16ub,
+ IPv4=0x0800,
+ ARP=0x0806,
+ RARP=0x8035,
+ X25=0x0805,
+ IPX=0x8137,
+ IPv6=0x86DD
+ ),
+)
+
+
+if __name__ == "__main__":
+ cap = unhexlify(b"0011508c283c0002e34260090800")
+ obj = ethernet_header.parse(cap)
+ print (obj)
+ built = ethernet_header.build(obj)
+ print(repr(built))
+ assert built == cap
+
diff --git a/lib/python/protocols/layer2/mtp2.py b/lib/python/protocols/layer2/mtp2.py
new file mode 100644
index 0000000..c73ade9
--- /dev/null
+++ b/lib/python/protocols/layer2/mtp2.py
@@ -0,0 +1,30 @@
+"""
+Message Transport Part 2 (SS7 protocol stack)
+(untested)
+"""
+from construct import *
+from binascii import unhexlify
+
+
+mtp2_header = "mtp2_header" / BitStruct(
+ "flag1" / Octet,
+ "bsn" / BitsInteger(7),
+ "bib" / Bit,
+ "fsn" / BitsInteger(7),
+ "sib" / Bit,
+ "length" / Octet,
+ "service_info" / Octet,
+ "signalling_info" / Octet,
+ "crc" / BitsInteger(16),
+ "flag2" / Octet,
+)
+
+if __name__ == "__main__":
+ cap = unhexlify(b"0bcc003500280689aa")
+ obj = mtp2_header.parse(cap)
+ print (obj)
+ built = mtp2_header.build(obj)
+ print(repr(built))
+ assert cap == built
+
+
diff --git a/lib/python/protocols/layer3/__init__.py b/lib/python/protocols/layer3/__init__.py
new file mode 100644
index 0000000..4477713
--- /dev/null
+++ b/lib/python/protocols/layer3/__init__.py
@@ -0,0 +1,4 @@
+"""
+layer 3 (network) protocols
+"""
+
diff --git a/lib/python/protocols/layer3/ipv4.py b/lib/python/protocols/layer3/ipv4.py
new file mode 100644
index 0000000..2ea0ad9
--- /dev/null
+++ b/lib/python/protocols/layer3/ipv4.py
@@ -0,0 +1,73 @@
+"""
+Internet Protocol version 4 (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+
+from construct import *
+from construct.core import *
+
+
+class IpAddressAdapter(Adapter):
+ def _encode(self, obj, context, path):
+ return "".join(chr(int(b)) for b in obj.split("."))
+
+ def _decode(self, obj, context, path):
+ return ".".join(str(ord(b)) for b in obj)
+
+def IpAddress():
+ return IpAddressAdapter(Bytes(4))
+
+def ProtocolEnum(code):
+ return Enum(code,
+ ICMP = 1,
+ TCP = 6,
+ UDP = 17,
+ )
+
+ipv4_header = "ip_header" / BitStruct(
+ "version" / Nibble,
+ "header_length" / ExprAdapter(Nibble,
+ lambda obj, ctx: obj * 4,
+ lambda obj, ctx: obj / 4
+ ),
+ "tos" / Struct(
+ "precedence" / BitsInteger(3),
+ "minimize_delay" / Flag,
+ "high_throuput" / Flag,
+ "high_reliability" / Flag,
+ "minimize_cost" / Flag,
+ Padding(1),
+ ),
+ "total_length" / Bytewise(Int16ub),
+ "payload_length" / Computed(lambda ctx: ctx.total_length - ctx.header_length),
+ "identification" / Bytewise(Int16ub),
+ "flags" / Struct(
+ Padding(1),
+ "dont_fragment" / Flag,
+ "more_fragments" / Flag,
+ ),
+ "frame_offset" / BitsInteger(13),
+ "ttl" / Bytewise(Int8ub),
+ "protocol" / Bytewise(ProtocolEnum(Int8ub)),
+ "checksum" / Bytewise(Int16ub),
+ "source" / Bytewise(IpAddress()),
+ "destination" / Bytewise(IpAddress()),
+ "options" / Computed(lambda ctx: ctx.header_length - 20),
+)
+
+
+if __name__ == "__main__":
+ cap = unhexlify(b"4600003ca0e3000080116185c0a80205d474a126")
+ obj = ipv4_header.parse(cap)
+ print (obj)
+ built = ipv4_header.build(obj)
+ print(repr(built))
+ assert built == cap
+
+
+
+
+
+
+
+
diff --git a/lib/python/protocols/layer3/ipv6.py b/lib/python/protocols/layer3/ipv6.py
new file mode 100644
index 0000000..cfc345b
--- /dev/null
+++ b/lib/python/protocols/layer3/ipv6.py
@@ -0,0 +1,39 @@
+"""
+Internet Protocol version 6 (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+from construct import *
+from construct.core import *
+from protocols.layer3.ipv4 import ProtocolEnum
+
+
+class Ipv6AddressAdapter(Adapter):
+ def _encode(self, obj, context, path):
+ return "".join(part.decode("hex") for part in obj.split(":"))
+ def _decode(self, obj, context, path):
+ return ":".join(b.encode("hex") for b in obj)
+
+def Ipv6Address():
+ return Ipv6AddressAdapter(Bytes(16))
+
+
+ipv6_header = "ip_header" / BitStruct(
+ "version" / OneOf(BitsInteger(4), [6]),
+ "traffic_class" / BitsInteger(8),
+ "flow_label" / BitsInteger(20),
+ "payload_length" / Bytewise(Int16ub),
+ "protocol" / Bytewise(ProtocolEnum(Int8ub)),
+ "ttl" / Bytewise(Int8ub),
+ "source" / Bytewise(Ipv6Address()),
+ "destination" / Bytewise(Ipv6Address()),
+)
+
+
+if __name__ == "__main__":
+ cap = unhexlify(b"6ff0000001020680" b"0123456789ABCDEF" b"FEDCBA9876543210" b"FEDCBA9876543210" b"0123456789ABCDEF")
+ obj = ipv6_header.parse(cap)
+ print (obj)
+ built = ipv6_header.build(obj)
+ print(repr(built))
+ assert built == cap
+
diff --git a/lib/python/protocols/layer4/__init__.py b/lib/python/protocols/layer4/__init__.py
new file mode 100644
index 0000000..38693c6
--- /dev/null
+++ b/lib/python/protocols/layer4/__init__.py
@@ -0,0 +1,4 @@
+"""
+layer 4 (transporation) protocols
+"""
+
diff --git a/lib/python/protocols/layer4/tcp.py b/lib/python/protocols/layer4/tcp.py
new file mode 100644
index 0000000..b2d3162
--- /dev/null
+++ b/lib/python/protocols/layer4/tcp.py
@@ -0,0 +1,63 @@
+"""
+Transmission Control Protocol (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+
+from construct import *
+from construct.core import *
+
+setGlobalPrintFullStrings(True)
+
+tcp_header = "tcp_header" / BitStruct(
+ "source" / Bytewise(Int16ub),
+ "destination" / Bytewise(Int16ub),
+ "seq" / Bytewise(Int32ub),
+ "ack" / Bytewise(Int32ub),
+ "header_length" / ExprAdapter(Nibble,
+ encoder = lambda obj, ctx: obj / 4,
+ decoder = lambda obj, ctx: obj * 4,
+ ),
+ Padding(3),
+ "flags" / Struct(
+ "ns" / Flag,
+ "cwr" / Flag,
+ "ece" / Flag,
+ "urg" / Flag,
+ "ack" / Flag,
+ "psh" / Flag,
+ "rst" / Flag,
+ "syn" / Flag,
+ "fin" / Flag,
+ ),
+ "window" / Bytewise(Int16ub),
+ "checksum" / Bytewise(Int16ub),
+ "urgent" / Bytewise(Int16ub),
+ "options" / Computed(lambda ctx: ctx.header_length - 20),
+# "_payload" / Bytewise(GreedyBytes),
+# "payload_length" / Computed(lambda ctx: len(ctx._payload)),
+
+)
+
+if __name__ == "__main__":
+ cap = unhexlify(b"0db5005062303fb21836e9e650184470c9bc000031323334")
+ obj = tcp_header.parse(cap)
+ print (obj)
+ built = tcp_header.build(obj)
+ print(repr(built))
+ assert cap == built
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/lib/python/protocols/layer4/udp.py b/lib/python/protocols/layer4/udp.py
new file mode 100644
index 0000000..6b4bcb1
--- /dev/null
+++ b/lib/python/protocols/layer4/udp.py
@@ -0,0 +1,29 @@
+"""
+User Datagram Protocol (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+
+from construct import *
+from construct.core import *
+
+
+udp_header = "udp_header" / Struct(
+ "header_length" / Computed(lambda ctx: 8),
+ "source" / Int16ub,
+ "destination" / Int16ub,
+ "payload_length" / ExprAdapter(Int16ub,
+ encoder = lambda obj, ctx: obj + 8,
+ decoder = lambda obj, ctx: obj - 8,
+ ),
+ "checksum" / Int16ub,
+)
+
+if __name__ == "__main__":
+ cap = unhexlify(b"0bcc003500280689")
+ obj = udp_header.parse(cap)
+ print (obj)
+ built = udp_header.build(obj)
+ print(repr(built))
+ assert cap == built
+
+
diff --git a/lib/python/protocols/unconverted/application/dns.py b/lib/python/protocols/unconverted/application/dns.py
new file mode 100644
index 0000000..d157227
--- /dev/null
+++ b/lib/python/protocols/unconverted/application/dns.py
@@ -0,0 +1,143 @@
+"""
+Domain Name System (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+
+from construct import *
+from construct.protocols.layer3.ipv4 import IpAddressAdapter
+
+
+class DnsStringAdapter(Adapter):
+ def _encode(self, obj, context):
+ parts = obj.split(".")
+ parts.append("")
+ return parts
+ def _decode(self, obj, context):
+ return ".".join(obj[:-1])
+
+dns_record_class = Enum(UBInt16("class"),
+ RESERVED = 0,
+ INTERNET = 1,
+ CHAOS = 3,
+ HESIOD = 4,
+ NONE = 254,
+ ANY = 255,
+)
+
+dns_record_type = Enum(UBInt16("type"),
+ IPv4 = 1,
+ AUTHORITIVE_NAME_SERVER = 2,
+ CANONICAL_NAME = 5,
+ NULL = 10,
+ MAIL_EXCHANGE = 15,
+ TEXT = 16,
+ X25 = 19,
+ ISDN = 20,
+ IPv6 = 28,
+ UNSPECIFIED = 103,
+ ALL = 255,
+)
+
+query_record = Struct("query_record",
+ DnsStringAdapter(
+ RepeatUntil(lambda obj, ctx: obj == "",
+ PascalString("name")
+ )
+ ),
+ dns_record_type,
+ dns_record_class,
+)
+
+rdata = Field("rdata", lambda ctx: ctx.rdata_length)
+
+resource_record = Struct("resource_record",
+ CString("name", terminators = b"\xc0\x00"),
+ Padding(1),
+ dns_record_type,
+ dns_record_class,
+ UBInt32("ttl"),
+ UBInt16("rdata_length"),
+ IfThenElse("data", lambda ctx: ctx.type == "IPv4",
+ IpAddressAdapter(rdata),
+ rdata
+ )
+)
+
+dns = Struct("dns",
+ UBInt16("id"),
+ BitStruct("flags",
+ Enum(Bit("type"),
+ QUERY = 0,
+ RESPONSE = 1,
+ ),
+ Enum(Nibble("opcode"),
+ STANDARD_QUERY = 0,
+ INVERSE_QUERY = 1,
+ SERVER_STATUS_REQUEST = 2,
+ NOTIFY = 4,
+ UPDATE = 5,
+ ),
+ Flag("authoritive_answer"),
+ Flag("truncation"),
+ Flag("recurssion_desired"),
+ Flag("recursion_available"),
+ Padding(1),
+ Flag("authenticated_data"),
+ Flag("checking_disabled"),
+ Enum(Nibble("response_code"),
+ SUCCESS = 0,
+ FORMAT_ERROR = 1,
+ SERVER_FAILURE = 2,
+ NAME_DOES_NOT_EXIST = 3,
+ NOT_IMPLEMENTED = 4,
+ REFUSED = 5,
+ NAME_SHOULD_NOT_EXIST = 6,
+ RR_SHOULD_NOT_EXIST = 7,
+ RR_SHOULD_EXIST = 8,
+ NOT_AUTHORITIVE = 9,
+ NOT_ZONE = 10,
+ ),
+ ),
+ UBInt16("question_count"),
+ UBInt16("answer_count"),
+ UBInt16("authority_count"),
+ UBInt16("additional_count"),
+ Array(lambda ctx: ctx.question_count,
+ Rename("questions", query_record),
+ ),
+ Rename("answers",
+ Array(lambda ctx: ctx.answer_count, resource_record)
+ ),
+ Rename("authorities",
+ Array(lambda ctx: ctx.authority_count, resource_record)
+ ),
+ Array(lambda ctx: ctx.additional_count,
+ Rename("additionals", resource_record),
+ ),
+)
+
+
+if __name__ == "__main__":
+ cap1 = unhexlify(b"2624010000010000000000000377777706676f6f676c6503636f6d0000010001")
+
+ cap2 = unhexlify(six.b(
+ "2624818000010005000600060377777706676f6f676c6503636f6d0000010001c00c00"
+ "05000100089065000803777777016cc010c02c0001000100000004000440e9b768c02c"
+ "0001000100000004000440e9b793c02c0001000100000004000440e9b763c02c000100"
+ "0100000004000440e9b767c030000200010000a88600040163c030c030000200010000"
+ "a88600040164c030c030000200010000a88600040165c030c030000200010000a88600"
+ "040167c030c030000200010000a88600040161c030c030000200010000a88600040162"
+ "c030c0c00001000100011d0c0004d8ef3509c0d0000100010000ca7c000440e9b309c0"
+ "80000100010000c4c5000440e9a109c0900001000100004391000440e9b709c0a00001"
+ "00010000ca7c000442660b09c0b00001000100000266000440e9a709"
+ ))
+
+ obj = dns.parse(cap1)
+ print (obj)
+ print (repr(dns.build(obj)))
+
+ print ("-" * 80)
+
+ obj = dns.parse(cap2)
+ print (obj)
+ print (repr(dns.build(obj)))
diff --git a/lib/python/protocols/unconverted/layer3/dhcpv4.py b/lib/python/protocols/unconverted/layer3/dhcpv4.py
new file mode 100644
index 0000000..d0e5a07
--- /dev/null
+++ b/lib/python/protocols/unconverted/layer3/dhcpv4.py
@@ -0,0 +1,195 @@
+"""
+Dynamic Host Configuration Protocol for IPv4
+
+http://www.networksorcery.com/enp/protocol/dhcp.htm
+http://www.networksorcery.com/enp/protocol/bootp/options.htm
+"""
+from binascii import unhexlify
+
+from construct import *
+from ipv4 import IpAddress
+
+
+dhcp_option = Struct("dhcp_option",
+ Enum(Byte("code"),
+ Pad = 0,
+ Subnet_Mask = 1,
+ Time_Offset = 2,
+ Router = 3,
+ Time_Server = 4,
+ Name_Server = 5,
+ Domain_Name_Server = 6,
+ Log_Server = 7,
+ Quote_Server = 8,
+ LPR_Server = 9,
+ Impress_Server = 10,
+ Resource_Location_Server = 11,
+ Host_Name = 12,
+ Boot_File_Size = 13,
+ Merit_Dump_File = 14,
+ Domain_Name = 15,
+ Swap_Server = 16,
+ Root_Path = 17,
+ Extensions_Path = 18,
+ IP_Forwarding_enabledisable = 19,
+ Nonlocal_Source_Routing_enabledisable = 20,
+ Policy_Filter = 21,
+ Maximum_Datagram_Reassembly_Size = 22,
+ Default_IP_TTL = 23,
+ Path_MTU_Aging_Timeout = 24,
+ Path_MTU_Plateau_Table = 25,
+ Interface_MTU = 26,
+ All_Subnets_are_Local = 27,
+ Broadcast_Address = 28,
+ Perform_Mask_Discovery = 29,
+ Mask_supplier = 30,
+ Perform_router_discovery = 31,
+ Router_solicitation_address = 32,
+ Static_routing_table = 33,
+ Trailer_encapsulation = 34,
+ ARP_cache_timeout = 35,
+ Ethernet_encapsulation = 36,
+ Default_TCP_TTL = 37,
+ TCP_keepalive_interval = 38,
+ TCP_keepalive_garbage = 39,
+ Network_Information_Service_domain = 40,
+ Network_Information_Servers = 41,
+ NTP_servers = 42,
+ Vendor_specific_information = 43,
+ NetBIOS_over_TCPIP_name_server = 44,
+ NetBIOS_over_TCPIP_Datagram_Distribution_Server = 45,
+ NetBIOS_over_TCPIP_Node_Type = 46,
+ NetBIOS_over_TCPIP_Scope = 47,
+ X_Window_System_Font_Server = 48,
+ X_Window_System_Display_Manager = 49,
+ Requested_IP_Address = 50,
+ IP_address_lease_time = 51,
+ Option_overload = 52,
+ DHCP_message_type = 53,
+ Server_identifier = 54,
+ Parameter_request_list = 55,
+ Message = 56,
+ Maximum_DHCP_message_size = 57,
+ Renew_time_value = 58,
+ Rebinding_time_value = 59,
+ Class_identifier = 60,
+ Client_identifier = 61,
+ NetWareIP_Domain_Name = 62,
+ NetWareIP_information = 63,
+ Network_Information_Service_Domain = 64,
+ Network_Information_Service_Servers = 65,
+ TFTP_server_name = 66,
+ Bootfile_name = 67,
+ Mobile_IP_Home_Agent = 68,
+ Simple_Mail_Transport_Protocol_Server = 69,
+ Post_Office_Protocol_Server = 70,
+ Network_News_Transport_Protocol_Server = 71,
+ Default_World_Wide_Web_Server = 72,
+ Default_Finger_Server = 73,
+ Default_Internet_Relay_Chat_Server = 74,
+ StreetTalk_Server = 75,
+ StreetTalk_Directory_Assistance_Server = 76,
+ User_Class_Information = 77,
+ SLP_Directory_Agent = 78,
+ SLP_Service_Scope = 79,
+ Rapid_Commit = 80,
+ Fully_Qualified_Domain_Name = 81,
+ Relay_Agent_Information = 82,
+ Internet_Storage_Name_Service = 83,
+ NDS_servers = 85,
+ NDS_tree_name = 86,
+ NDS_context = 87,
+ BCMCS_Controller_Domain_Name_list = 88,
+ BCMCS_Controller_IPv4_address_list = 89,
+ Authentication = 90,
+ Client_last_transaction_time = 91,
+ Associated_ip = 92,
+ Client_System_Architecture_Type = 93,
+ Client_Network_Interface_Identifier = 94,
+ Lightweight_Directory_Access_Protocol = 95,
+ Client_Machine_Identifier = 97,
+ Open_Group_User_Authentication = 98,
+ Autonomous_System_Number = 109,
+ NetInfo_Parent_Server_Address = 112,
+ NetInfo_Parent_Server_Tag = 113,
+ URL = 114,
+ Auto_Configure = 116,
+ Name_Service_Search = 117,
+ Subnet_Selection = 118,
+ DNS_domain_search_list = 119,
+ SIP_Servers_DHCP_Option = 120,
+ Classless_Static_Route_Option = 121,
+ CableLabs_Client_Configuration = 122,
+ GeoConf = 123,
+ ),
+ Switch("value", lambda ctx: ctx.code,
+ {
+ # codes without any value
+ "Pad" : Pass,
+ },
+ # codes followed by length and value fields
+ default = Struct("value",
+ Byte("length"),
+ Field("data", lambda ctx: ctx.length),
+ )
+ )
+)
+
+dhcp_header = Struct("dhcp_header",
+ Enum(Byte("opcode"),
+ BootRequest = 1,
+ BootReply = 2,
+ ),
+ Enum(Byte("hardware_type"),
+ Ethernet = 1,
+ Experimental_Ethernet = 2,
+ ProNET_Token_Ring = 4,
+ Chaos = 5,
+ IEEE_802 = 6,
+ ARCNET = 7,
+ Hyperchannel = 8,
+ Lanstar = 9,
+ ),
+ Byte("hardware_address_length"),
+ Byte("hop_count"),
+ UBInt32("transaction_id"),
+ UBInt16("elapsed_time"),
+ BitStruct("flags",
+ Flag("boardcast"),
+ Padding(15),
+ ),
+ IpAddress("client_addr"),
+ IpAddress("your_addr"),
+ IpAddress("server_addr"),
+ IpAddress("relay_addr"),
+ Bytes("client_hardware_addr", 16),
+ Bytes("server_host_name", 64),
+ Bytes("boot_filename", 128),
+ # BOOTP/DHCP options
+ # "The first four bytes contain the (decimal) values 99, 130, 83 and 99"
+ Const(Bytes("magic", 4), b"\x63\x82\x53\x63"),
+ Rename("options", OptionalGreedyRange(dhcp_option)),
+)
+
+
+if __name__ == "__main__":
+ test = unhexlify(six.b(
+ "0101060167c05f5a00000000"
+ "0102030405060708090a0b0c"
+ "0d0e0f10"
+ "DEADBEEFBEEF"
+ "000000000000000000000000000000000000000000000000000000"
+ "000000000000000000000000000000000000000000000000000000"
+ "000000000000000000000000000000000000000000000000000000"
+ "000000000000000000000000000000000000000000000000000000"
+ "000000000000000000000000000000000000000000000000000000"
+ "000000000000000000000000000000000000000000000000000000"
+ "000000000000000000000000000000000000000000000000000000"
+ "00000000000000000000000000"
+ "63825363"
+ "3501083d0701DEADBEEFBEEF0c04417375733c084d53465420352e"
+ "30370d010f03062c2e2f1f2179f92bfc52210117566c616e333338"
+ "382b45746865726e6574312f302f32340206f8f0827348f9ff"
+ ))
+
+ print (dhcp_header.parse(test))
diff --git a/lib/python/protocols/unconverted/layer3/dhcpv6.py b/lib/python/protocols/unconverted/layer3/dhcpv6.py
new file mode 100644
index 0000000..b9ff172
--- /dev/null
+++ b/lib/python/protocols/unconverted/layer3/dhcpv6.py
@@ -0,0 +1,111 @@
+"""
+the Dynamic Host Configuration Protocol (DHCP) for IPv6
+
+http://www.networksorcery.com/enp/rfc/rfc3315.txt
+"""
+from construct import *
+from ipv6 import Ipv6Address
+
+
+dhcp_option = Struct("dhcp_option",
+ Enum(UBInt16("code"),
+ OPTION_CLIENTID = 1,
+ OPTION_SERVERID = 2,
+ OPTION_IA_NA = 3,
+ OPTION_IA_TA = 4,
+ OPTION_IAADDR = 5,
+ OPTION_ORO = 6,
+ OPTION_PREFERENCE = 7,
+ OPTION_ELAPSED_TIME = 8,
+ OPTION_RELAY_MSG = 9,
+ OPTION_AUTH = 11,
+ OPTION_UNICAST = 12,
+ OPTION_STATUS_CODE = 13,
+ OPTION_RAPID_COMMIT = 14,
+ OPTION_USER_CLASS = 15,
+ OPTION_VENDOR_CLASS = 16,
+ OPTION_VENDOR_OPTS = 17,
+ OPTION_INTERFACE_ID = 18,
+ OPTION_RECONF_MSG = 19,
+ OPTION_RECONF_ACCEPT = 20,
+ SIP_SERVERS_DOMAIN_NAME_LIST = 21,
+ SIP_SERVERS_IPV6_ADDRESS_LIST = 22,
+ DNS_RECURSIVE_NAME_SERVER = 23,
+ DOMAIN_SEARCH_LIST = 24,
+ OPTION_IA_PD = 25,
+ OPTION_IAPREFIX = 26,
+ OPTION_NIS_SERVERS = 27,
+ OPTION_NISP_SERVERS = 28,
+ OPTION_NIS_DOMAIN_NAME = 29,
+ OPTION_NISP_DOMAIN_NAME = 30,
+ SNTP_SERVER_LIST = 31,
+ INFORMATION_REFRESH_TIME = 32,
+ BCMCS_CONTROLLER_DOMAIN_NAME_LIST = 33,
+ BCMCS_CONTROLLER_IPV6_ADDRESS_LIST = 34,
+ OPTION_GEOCONF_CIVIC = 36,
+ OPTION_REMOTE_ID = 37,
+ RELAY_AGENT_SUBSCRIBER_ID = 38,
+ OPTION_CLIENT_FQDN = 39,
+ ),
+ UBInt16("length"),
+ Field("data", lambda ctx: ctx.length),
+)
+
+client_message = Struct("client_message",
+ Bitwise(BitField("transaction_id", 24)),
+)
+
+relay_message = Struct("relay_message",
+ Byte("hop_count"),
+ Ipv6Address("linkaddr"),
+ Ipv6Address("peeraddr"),
+)
+
+dhcp_message = Struct("dhcp_message",
+ Enum(Byte("msgtype"),
+ # these are client-server messages
+ SOLICIT = 1,
+ ADVERTISE = 2,
+ REQUEST = 3,
+ CONFIRM = 4,
+ RENEW = 5,
+ REBIND = 6,
+ REPLY = 7,
+ RELEASE_ = 8,
+ DECLINE_ = 9,
+ RECONFIGURE = 10,
+ INFORMATION_REQUEST = 11,
+ # these two are relay messages
+ RELAY_FORW = 12,
+ RELAY_REPL = 13,
+ ),
+ # relay messages have a different structure from client-server messages
+ Switch("params", lambda ctx: ctx.msgtype,
+ {
+ "RELAY_FORW" : relay_message,
+ "RELAY_REPL" : relay_message,
+ },
+ default = client_message,
+ ),
+ Rename("options", GreedyRange(dhcp_option)),
+)
+
+
+if __name__ == "__main__":
+ test1 = b"\x03\x11\x22\x33\x00\x17\x00\x03ABC\x00\x05\x00\x05HELLO"
+ test2 = b"\x0c\x040123456789abcdef0123456789abcdef\x00\x09\x00\x0bhello world\x00\x01\x00\x00"
+ print (dhcp_message.parse(test1))
+ print (dhcp_message.parse(test2))
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/lib/python/protocols/unconverted/layer3/icmpv4.py b/lib/python/protocols/unconverted/layer3/icmpv4.py
new file mode 100644
index 0000000..7a554d1
--- /dev/null
+++ b/lib/python/protocols/unconverted/layer3/icmpv4.py
@@ -0,0 +1,87 @@
+"""
+Internet Control Message Protocol for IPv4 (TCP/IP protocol stack)
+"""
+from binascii import unhexlify
+
+from construct import *
+from ipv4 import IpAddress
+
+
+echo_payload = Struct("echo_payload",
+ UBInt16("identifier"),
+ UBInt16("sequence"),
+ Bytes("data", 32), # length is implementation dependent...
+ # is anyone using more than 32 bytes?
+)
+
+dest_unreachable_payload = Struct("dest_unreachable_payload",
+ Padding(2),
+ UBInt16("next_hop_mtu"),
+ IpAddress("host"),
+ Bytes("echo", 8),
+)
+
+dest_unreachable_code = Enum(Byte("code"),
+ Network_unreachable_error = 0,
+ Host_unreachable_error = 1,
+ Protocol_unreachable_error = 2,
+ Port_unreachable_error = 3,
+ The_datagram_is_too_big = 4,
+ Source_route_failed_error = 5,
+ Destination_network_unknown_error = 6,
+ Destination_host_unknown_error = 7,
+ Source_host_isolated_error = 8,
+ Desination_administratively_prohibited = 9,
+ Host_administratively_prohibited2 = 10,
+ Network_TOS_unreachable = 11,
+ Host_TOS_unreachable = 12,
+)
+
+icmp_header = Struct("icmp_header",
+ Enum(Byte("type"),
+ Echo_reply = 0,
+ Destination_unreachable = 3,
+ Source_quench = 4,
+ Redirect = 5,
+ Alternate_host_address = 6,
+ Echo_request = 8,
+ Router_advertisement = 9,
+ Router_solicitation = 10,
+ Time_exceeded = 11,
+ Parameter_problem = 12,
+ Timestamp_request = 13,
+ Timestamp_reply = 14,
+ Information_request = 15,
+ Information_reply = 16,
+ Address_mask_request = 17,
+ Address_mask_reply = 18,
+ _default_ = Pass,
+ ),
+ Switch("code", lambda ctx: ctx.type,
+ {
+ "Destination_unreachable" : dest_unreachable_code,
+ },
+ default = Byte("code"),
+ ),
+ UBInt16("crc"),
+ Switch("payload", lambda ctx: ctx.type,
+ {
+ "Echo_reply" : echo_payload,
+ "Echo_request" : echo_payload,
+ "Destination_unreachable" : dest_unreachable_payload,
+ },
+ default = Pass
+ )
+)
+
+
+if __name__ == "__main__":
+ cap1 = unhexlify(six.b("0800305c02001b006162636465666768696a6b6c6d6e6f70717273747576776162"
+ "63646566676869"))
+ cap2 = unhexlify(six.b("0000385c02001b006162636465666768696a6b6c6d6e6f70717273747576776162"
+ "63646566676869"))
+ cap3 = unhexlify(b"0301000000001122aabbccdd0102030405060708")
+
+ print (icmp_header.parse(cap1))
+ print (icmp_header.parse(cap2))
+ print (icmp_header.parse(cap3))
diff --git a/lib/python/protocols/unconverted/layer3/igmpv2.py b/lib/python/protocols/unconverted/layer3/igmpv2.py
new file mode 100644
index 0000000..99ab6db
--- /dev/null
+++ b/lib/python/protocols/unconverted/layer3/igmpv2.py
@@ -0,0 +1,29 @@
+"""
+What : Internet Group Management Protocol, Version 2
+ How : http://www.ietf.org/rfc/rfc2236.txt
+ Who : jesse @ housejunkie . ca
+"""
+
+from binascii import unhexlify
+
+from construct import Byte, Enum,Struct, UBInt16
+from construct.protocols.layer3.ipv4 import IpAddress
+
+
+igmp_type = Enum(Byte("igmp_type"),
+ MEMBERSHIP_QUERY = 0x11,
+ MEMBERSHIP_REPORT_V1 = 0x12,
+ MEMBERSHIP_REPORT_V2 = 0x16,
+ LEAVE_GROUP = 0x17,
+)
+
+igmpv2_header = Struct("igmpv2_header",
+ igmp_type,
+ Byte("max_resp_time"),
+ UBInt16("checksum"),
+ IpAddress("group_address"),
+)
+
+if __name__ == '__main__':
+ capture = unhexlify(b"1600FA01EFFFFFFD")
+ print (igmpv2_header.parse(capture))
diff --git a/lib/python/protocols/unconverted/layer3/mtp3.py b/lib/python/protocols/unconverted/layer3/mtp3.py
new file mode 100644
index 0000000..7f712f2
--- /dev/null
+++ b/lib/python/protocols/unconverted/layer3/mtp3.py
@@ -0,0 +1,12 @@
+"""
+Message Transport Part 3 (SS7 protocol stack)
+(untested)
+"""
+from construct import *
+
+
+mtp3_header = BitStruct("mtp3_header",
+ Nibble("service_indicator"),
+ Nibble("subservice"),
+)
+
diff --git a/lib/python/protocols/unconverted/layer4/isup.py b/lib/python/protocols/unconverted/layer4/isup.py
new file mode 100644
index 0000000..8111b60
--- /dev/null
+++ b/lib/python/protocols/unconverted/layer4/isup.py
@@ -0,0 +1,15 @@
+"""
+ISDN User Part (SS7 protocol stack)
+"""
+from construct import *
+
+
+isup_header = Struct("isup_header",
+ Bytes("routing_label", 5),
+ UBInt16("cic"),
+ UBInt8("message_type"),
+ # mandatory fixed parameters
+ # mandatory variable parameters
+ # optional parameters
+)
+
diff --git a/tests/channels/SIP/pcap_demo/run-test b/tests/channels/SIP/pcap_demo/run-test
index 8400a85..ecb2bb0 100755
--- a/tests/channels/SIP/pcap_demo/run-test
+++ b/tests/channels/SIP/pcap_demo/run-test
@@ -9,10 +9,7 @@
from sip_message import SIPMessage, SIPMessageTest
from twisted.internet import reactor
-try:
- from construct_legacy.protocols.ipstack import ip_stack
-except ImportError:
- from construct.protocols.ipstack import ip_stack
+from protocols.ipstack import ip_stack
logger = logging.getLogger(__name__)
test1 = [
diff --git a/tests/channels/SIP/pcap_demo/test-config.yaml b/tests/channels/SIP/pcap_demo/test-config.yaml
index 6a5111a..ec151be 100644
--- a/tests/channels/SIP/pcap_demo/test-config.yaml
+++ b/tests/channels/SIP/pcap_demo/test-config.yaml
@@ -16,7 +16,6 @@
properties:
dependencies:
- python : 'twisted'
- - python : 'construct'
- 'pcap'
- asterisk : 'chan_sip'
tags:
diff --git a/tests/channels/pjsip/rtp/rtp_keepalive/base/rtp.py b/tests/channels/pjsip/rtp/rtp_keepalive/base/rtp.py
index 2e325f5..633971d 100644
--- a/tests/channels/pjsip/rtp/rtp_keepalive/base/rtp.py
+++ b/tests/channels/pjsip/rtp/rtp_keepalive/base/rtp.py
@@ -12,11 +12,7 @@
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
-
-try:
- from construct_legacy import *
-except ImportError:
- from construct import *
+from asterisk.pcap import RTPPacket
sys.path.append('lib/python')
@@ -30,22 +26,9 @@
self.test_object.register_stop_observer(self.asterisk_stopped)
def datagramReceived(self, data, (host, port)):
- header = Struct('rtp_packet',
- BitStruct('header',
- BitField('version', 2),
- Bit('padding'),
- Bit('extension'),
- Nibble('csrc_count'),
- Bit('marker'),
- BitField('payload', 7)
- ),
- UBInt16('sequence_number'),
- UBInt32('timestamp'),
- UBInt32('ssrc')
- )
- rtp_header = header.parse(data)
+ rtp_header = RTPPacket.rtp_header.parse(data)
LOGGER.debug("Parsed RTP packet is {0}".format(rtp_header))
- if rtp_header.header.payload == 13:
+ if rtp_header.payload_type == 13:
current_time = time.time()
# Don't compare intervals on the first received CNG
if self.last_rx_time:
diff --git a/tests/hep/hep_capture_node.py b/tests/hep/hep_capture_node.py
index 0fb081e..010756a 100644
--- a/tests/hep/hep_capture_node.py
+++ b/tests/hep/hep_capture_node.py
@@ -20,10 +20,8 @@
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
-try:
- from construct_legacy import *
-except ImportError:
- from construct import *
+from construct import *
+from construct.core import *
LOGGER = logging.getLogger(__name__)
@@ -107,40 +105,40 @@
self.module = module
- self.hep_chunk = Struct('hep_chunk',
- UBInt16('vendor_id'),
- UBInt16('type_id'),
- UBInt16('length'))
- hep_ctrl = Struct('hep_ctrl',
- Array(4, UBInt8('id')),
- UBInt16('length'))
- hep_ip_family = Struct('hep_ip_family',
+ self.hep_chunk = 'hep_chunk' / Struct(
+ 'vendor_id' / Int16ub,
+ 'type_id' / Int16ub,
+ 'length' / Int16ub)
+ hep_ctrl = 'hep_ctrl' / Struct(
+ 'id' / Array(4, Int8ub),
+ 'length' / Int16ub)
+ hep_ip_family = 'hep_ip_family' / Struct(
self.hep_chunk,
- UBInt8('ip_family'));
- hep_ip_id = Struct('hep_ip_id',
+ 'ip_family' / Int8ub);
+ hep_ip_id = 'hep_ip_id' / Struct(
self.hep_chunk,
- UBInt8('ip_id'))
- hep_port = Struct('hep_port',
+ 'ip_id' / Int8ub)
+ hep_port = 'hep_port' / Struct(
self.hep_chunk,
- UBInt16('port'))
- hep_timestamp_sec = Struct('hep_timestamp_sec',
+ 'port' / Int16ub)
+ hep_timestamp_sec = 'hep_timestamp_sec' / Struct(
self.hep_chunk,
- UBInt32('timestamp_sec'))
- hep_timestamp_usec = Struct('hep_timestamp_usec',
+ 'timestamp_sec' / Int32ub)
+ hep_timestamp_usec = 'hep_timestamp_usec' / Struct(
self.hep_chunk,
- UBInt32('timestamp_usec'))
- hep_protocol_type = Struct('hep_protocol_type',
+ 'timestamp_usec' / Int32ub)
+ hep_protocol_type = 'hep_protocol_type' / Struct(
self.hep_chunk,
- UBInt8('protocol_type'))
- hep_capture_agent_id = Struct('hep_capture_agent_id',
+ 'protocol_type' / Int8ub)
+ hep_capture_agent_id = 'hep_capture_agent_id' / Struct(
self.hep_chunk,
- UBInt32('capture_agent_id'))
- self.hep_generic_msg = Struct('hep_generic',
+ 'capture_agent_id' / Int32ub)
+ self.hep_generic_msg = 'hep_generic' / Struct(
hep_ctrl,
hep_ip_family,
hep_ip_id,
- Struct('src_port', hep_port),
- Struct('dst_port', hep_port),
+ 'src_port' / Struct(hep_port),
+ 'dst_port' / Struct(hep_port),
hep_timestamp_sec,
hep_timestamp_usec,
hep_protocol_type,
@@ -161,18 +159,18 @@
dst_addr = None
if parsed_hdr.hep_ip_family.ip_family == IP_FAMILY.v4:
# IPv4
- hep_ipv4_addr = Struct('hep_ipv4_addr',
+ hep_ipv4_addr = 'hep_ipv4_addr' / Struct(
self.hep_chunk,
- String('ipv4_addr', 4))
+ 'ipv4_addr' / PaddedString(4, "ascii"))
src_addr = hep_ipv4_addr.parse(data[length:])
length += hep_ipv4_addr.sizeof()
dst_addr = hep_ipv4_addr.parse(data[length:])
length += hep_ipv4_addr.sizeof()
elif parsed_hdr.hep_ip_family.ip_family == IP_FAMILY.v6:
# IPv6
- hep_ipv6_addr = Struct('hep_ipv6_addr',
+ hep_ipv6_addr = 'hep_ipv6_addr' / Struct(
self.hep_chunk,
- String('ipv6_addr', 16))
+ 'ipv6_addr' / PaddedString(16, "ascii"))
src_addr = hep_ipv6_addr.parse(data[length:])
length += hep_ipv6_addr.sizeof()
dst_addr = hep_ipv6_addr.parse(data[length:])
@@ -185,20 +183,20 @@
hdr = self.hep_chunk.parse(data[length:])
length += self.hep_chunk.sizeof()
if hdr.type_id == HEP_VARIABLE_TYPES.auth_key:
- hep_auth_key = String('hep_auth_key',
- hdr.length - self.hep_chunk.sizeof())
+ hep_auth_key = 'hep_auth_key' / PaddedString(
+ hdr.length - self.hep_chunk.sizeof(), "ascii")
packet.auth_key = hep_auth_key.parse(data[length:])
length += hep_auth_key.sizeof() - self.hep_chunk.sizeof()
elif hdr.type_id == HEP_VARIABLE_TYPES.payload:
- hep_payload = String('hep_payload',
- hdr.length - self.hep_chunk.sizeof())
+ hep_payload = 'hep_payload' / PaddedString(
+ hdr.length - self.hep_chunk.sizeof(), "ascii")
packet.payload = hep_payload.parse(data[length:])
length += hep_payload.sizeof() - self.hep_chunk.sizeof()
LOGGER.debug('Packet payload: %s' % packet.payload)
elif hdr.type_id == HEP_VARIABLE_TYPES.uuid:
- hep_uuid = String('hep_uuid',
- hdr.length - self.hep_chunk.sizeof())
+ hep_uuid = 'hep_uuid' / PaddedString(
+ hdr.length - self.hep_chunk.sizeof(), "ascii")
packet.uuid = hep_uuid.parse(data[length:])
length += hep_uuid.sizeof() - self.hep_chunk.sizeof()
self.module.verify_packet(packet)
@@ -379,4 +377,3 @@
LOGGER.error('Failed to match packet %d: %s' %
(self.current_packet, str(packet.__dict__)))
self.test_object.set_passed(False)
-
--
To view, visit https://gerrit.asterisk.org/c/testsuite/+/13222
To unsubscribe, or for help writing mail filters, visit https://gerrit.asterisk.org/settings
Gerrit-Project: testsuite
Gerrit-Branch: 13
Gerrit-Change-Id: Id38d01a2cd073b240fde909a38c95d69313bbbe7
Gerrit-Change-Number: 13222
Gerrit-PatchSet: 3
Gerrit-Owner: George Joseph <gjoseph at digium.com>
Gerrit-Reviewer: Friendly Automation
Gerrit-Reviewer: George Joseph <gjoseph at digium.com>
Gerrit-Reviewer: Joshua Colp <jcolp at digium.com>
Gerrit-Reviewer: Kevin Harwell <kharwell at digium.com>
Gerrit-MessageType: merged
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.digium.com/pipermail/asterisk-code-review/attachments/20191209/520ff7da/attachment-0001.html>
More information about the asterisk-code-review
mailing list