[Asterisk-Users] update - 512 Simultaneous Calls with Digital Recording

[Matt Florell]

> Matthew, thanks for your feedback and advice.
> > what I actually experienced was the complete breakdown of Asterisk at
> > around 60 concurrent recordings without it (the reality).
>
> The drive for saving your voice recordings is the same as your OS
> (Asterisk)? What do you think that save the voice recordings to a
> dedicated drive rather than the one which Asterisk program (OS) locates?

That is what we do actually. One drive for Linux/Asterisk and a SCSI
RAID for /var/spool/asterisk/monitor

> I also think about using GSM format (Monitor(gsm,${CALLFILENAME}, mb))
> rather than WAV, PCM. In this case, it will use more CPU, but I/O of
> hard disk is reduced dramatically as you mentioned that it is I/O
> bottleneck issue, not CPU (In my case, I want to use P4 Dual core CPU or
> extreme edition). In order to reduce the CPU usage, we can have two leg
> files mixed after peak time.

Yes, you will have to wait until off-hours to mix the recordings, but
while switching to GSM does help reduce the amount of data written to
the drives, the gains are mostly cancelled out by the compression that
is needed to convert each stream to GSM. In the case of 60 calls you
would be converting 120 streams to GSM.

> Matt mentioned about fragmented free space. I googled about Linux
> defragment topic. People always talk about that Linux doesn't need to
> defragment, it can handle it by itself very well. Not sure how true it
> is.

It is something we ran into before we switched to SCSI RAID setups. It
happened quicker with IDE drives than with SATA(and so far not at all
with SCSI). The problem does not show up until you have deleted files
several times from the drives and it gets worse over time until you 
wipe and reformat the recording drive.

The problem that happens with a large amount of concurrent recordings
is audio skips, from a quarter second to two seconds. There isn't much
of a recording buffer built-in to Asterisk so if anything causes a
delay in packet delivery there will be an audio skip. No matter what
kind of setup we used we have not been able to guarantee skip-free
recording files in any case above 60 concurrent recordings over the
course of a week. It is very hard to test for, but if you have all of
your recordings listened to you will be able to know if there is a
problem. For us in one case(70 concurrent recordings) the skips only
occured very infrequently, maybe 0.00005% of the time(10 skips in 55
hours of recordings) which doesn't sound bad, but if it happens during
the wrong part of a critical sales confirmation that half second skip
could cost my company hundreds of dollars.

> I am looking a solution to record expanding simultaneous calls in the
> future in a call centre which accepts calls from our global branches. If
> I find the good solution, I definitely post it to the community.

The best reliable solution that I have been able to depend on is to
limit the number to 60 concurrent recordings on a single P4 3.4GHz
server with a LSILogic MegaRAID 320-1 and four 15k-RPM SCSI drives in
a RAID 10. With this we experience usually no audio skips across the
course of a week(and thousands of recordings), and it has worked this
way reliably for the last nine months.

If you have less strict quality standards you could probably push that
number up, but you will get more and more audio skips the higher you
go.

MATT---