[asterisk-users] Big time system

[Stuart Elvish]

Hi All,

I am not sure if my comments will be helpful but here goes.

Recently completed a roll out of Asterisk to provide a FTTH (fibre to
the home) based telephone exchange. It uses Asterisk real time and a
MySQL database. The design is for a somewhat smaller audience (~4000
extensions) but should provide some insight into a successful larger
build which is basically a server scaling exercise.

The setup works like this:
2x MySQL servers, one primary and one backup with a shared IP address.
2x Asterisk "gateway" servers, one primary and one secondary, a shared
IP address.
2x Asterisk "SIP" servers, one primary and one secondary with a shared
IP address.

The Asterisk servers are joined by IAX which allows the SIP server
(only serving internal connections) to connect to ISDN or via SIP
through one of the two gateway servers whilst providing redundancy and
isolation of the different parts of the network. Each house has a
termination box that supplies cable TV, an ATA and internet access via
a single QoS managed fibre link. The equipment at the end of each
fibre is proprietary but is the same as an ATA for the purposes of
voice (therefore each house is presented with a two-wire "exchange"
connection). Each termination box provides two copper lines.

Use of MySQL for provisioning has been fantastic - it has been stable
and allows us to provide third parties with access to provision every
aspect of the server (voicemail, DID's, extensions for the ATA's, call
type restrictions and control line level settings such as voicemail
and ring delay). We have some code in the dial plan which checks these
settings using ODBC integration. For reliability we have a local copy
of extensions.conf for each server.

In terms of system design, there have been three things that we needed
to know inside out:
* How to make MySQL bullet proof. In your case you are probably
considering a cluster with multiple physical locations for redundancy.
* What changes to the default kernel settings need to be made to
facilitate large network groups and large buffers. This really depends
on how many individual registrations and calls each server is going to
handle.
* Setup of Asterisk to work well for high availability and ease of
configuration.

One interesting design project I have also worked on (which may help
here) is for emergency telephones on a motorway. Every telephone is
fed from an alternating supply cabinet (phone 1 - cabinet 1, phone 2 -
cabinet 2, phone 3 - cabinet 1, phone 4 - cabinet 2) so if one cabinet
fails (due to a switch, an ATA or power supply failure) only every
second phone is "taken out".

There are other issues - faxing and data calls don't work well in most
setups. On the FTTH project we use an override code for faxes and send
them via ISDN rather than VoIP. Unfortunately one of the contractors
chose VoIP as the backbone of the network rather than ISDN despite our
advice to the contrary. Your choice of ATA will also be important to
faxing.

>>
>> Asterisk may carry you a way down this road, but in the end, it's not,
>> and was never designed to be a class 5 telecom switch. There are people
>> working on a carrier grade implementation that may or may not be fully
>> class 5, but I don't know what the status is on that. I haven't gotten
>> an answer from Digium on that lately.
>>
I disagree with this statement for a purely voice network. A few years
ago this statement would be true. With the correct hardware and
engineering, Asterisk can competently handle class 5 and higher (for
example class 4) switching. They key however is good design and
separating servers to run core functions as the network gets larger. A
large network like this means that you would most likely have specific
servers running highly specialized routing rather than having one
server route all sorts of different calls.

>>
>> You'll want some type of Multiservice Access Platform (MSAP). Zhone
>> makes the MALC and their newer MXK box. Adtran has the TA-5000 shelf.
>> Neither are what you'd call cheap. Both will provide T1 access, DSL,
>> SDSL, VDSL, bonded, and even ethernet access to the customer over a
>> variety of transport options, including copper pairs.
>>
There are a few other vendors that have this, CoreCess is one and I
believe Motorola also have been looking at manufacturing this type of
unit. (The Motorola conversation wasn't exactly specific, it was
mentioned they were interested in a job which was similar in
requirements to what the Zhone equipment was recommended for.) But, I
think you were looking for exchange equipment.

Your suggestion of using a server at each local exchange seems to be
most logical. I am assuming you will be using existing copper which
you can then put into a channel bank. I unfortunately don't have too
many brand recommendations but this is similar in style to what I have
done on some closed networks.

One thing that I can't emphasize enough is to test the system
thoroughly. You will need to make sure you comply with E911 and other
emergency services requirements as well as making sure that the system
works 24/7 especially for emergency calls. If you do your testing it
really helps in knowing when someone is making a false tech report and
helps you find the problem when there are real reports. The user base
you will be dealing with is not just a few users any more, it will be
thousands.

I hope this helps.

Let me know if you need anymore pointers. Also happy to consult but
you would need to contact me off list for that...

Stuart Elvish