[asterisk-users] Benchmarking AGI performance in C, PHP, and Perl

[Steve Edwards]

Many times, I've made the statement that you can execute hundreds of AGIs 
written in C in the time it takes to load an interpreter and parse a 
script written in PHP or Perl.

Recently, a Doubting Thomas asked me to substantiate my claim.

I suspect nobody has made the effort to implement an AGI of any reasonable 
size and function in multiple languages.

I'm guessing it may not really be all that important and the results would 
be too task specific to be relevant.

I suspect once an AGI is executing, the choice of source language is 
unimportant. I'll go out on a limb and say executing:

 	select prompt_path from foo where bar;
 	stream file prompt_path "1234*"
 	stream file you_entered ""
 	say digits selected ""

will execute in effectively the same time regardless of source language. 
Waiting for Asterisk to play a file or for your database to return a row 
is beyond the scope of your AGI.

It's what you do between your AGI and database calls that will determine 
how much your choice of source language will impact the total execution 
time.

Unless you're doing a lot of stuff in between these API calls, the only 
place you can make an impact is getting your code into memory and ready to 
execute.

I 'wrote' 2 different AGIs in C, PHP, and Perl.

The first AGI, 'null-agi' reads the AGI environment variables from STDIN 
and exits. To me, this is the bare minimum a program can do and call 
itself an AGI. Each AGI was less than 10 lines.

The second AGI, 'neutered-agi' is an AGI of 'production length' (around 
1,600 lines) and supporting access to a MySQL database. The AGI is of 
'production length' but still exits after reading the AGI environment 
variables because we are measuring program startup time.

For both AGIs, the C implementation used an AGI library I developed way 
too many years ago. The PHP implementation used PHPAGI. The Perl 
implementation used Asterisk::AGI.

The C version of neutered-agi was based on a 'voicemail-like' AGI I wrote 
many years ago that stored the user credentials and messages in MySQL.

The PHP version of neutered-agi was based on dialparties.agi (nicked from 
PIAF). dialparties.agi is only about 800 lines long, so I 'doubled it' by 
copying and pasting it into the same source file. While dialparties.agi 
does not use MySQL, MySQL is available in PHP without including additional 
header or class files.

The Perl version of neutered-agi was based on agi-VDAD_ALL_outbound.agi 
(nicked from Vicidial).

I ran the tests in 3 different environments:

|-------------------+--------+--------+----------|
| CPU               | RAM    | CentOS | Asterisk |
|-------------------+--------+--------+----------|
| Geode 500MHz      | 256 MB |    4.9 |   1.2.37 |
| Atom D525 1.80GHz | 4 GB   |    5.6 |  1.8.4.1 |
| Xeon 3.40GHz      | 2 GB   |    4.8 |   1.2.40 |
|-------------------+--------+--------+----------|

No swapping occurred during the tests.

My dialplan executed each AGI 1,000 times to make the cumulative execution 
time more measurable. I wrote the dialplan in both 'inline dialplan' and 
an AEL 'for' loop. The initial execution times were the same so the test 
runs were made with the AEL version because it is more manageable. (5 
lines versus 1,000 lines.)

Here's the results for executing each AGI 1,000 times on each host in 
seconds:

Geode:

|----------+----------+--------------|
| language | null-agi | neutered-agi |
|----------+----------+--------------|
| C        |        6 |            6 |
| PHP      |      116 |          160 |
| Perl     |       99 |          639 |
|----------+----------+--------------|

Atom:

|----------+----------+--------------|
| language | null-agi | neutered-agi |
|----------+----------+--------------|
| C        |        6 |            6 |
| PHP      |       52 |           65 |
| Perl     |       38 |          197 |
|----------+----------+--------------|

Xeon:

|----------+----------+--------------|
| language | null-agi | neutered-agi |
|----------+----------+--------------|
| C        |        2 |            2 |
| PHP      |       40 |           47 |
| Perl     |       10 |          107 |
|----------+----------+--------------|

Summary:

Geode - Perl / C: 106
Atom - Perl / C: 33
Xeon - Perl / C: 54

The C null-agi AGI was statically linked. I didn't have all the libraries 
needed to statically link neutered-agi on these boxes, but the dynamically 
linked versions of null-agi and neutered-agi took the same time to execute 
(16 seconds on the Geode) so I'm assuming statically linked versions of 
null-agi and neutered-agi would also take the same time to execute. It 
also helps support my original statement :)

I guessing the Perl version of neutered-agi took a big hit from having to 
load the database code as well as the AGI framework (use DBI; use 
Asterisk::AGI;) while PHP only had to load the AGI framework (require_once 
"phpagi.php";).

I'll let you decide if the methodology is meaningful to your environment. 
I don't consider myself to be a PHP or Perl expert, so if I've made some 
colossal blunder in my methodology, please let me know.

I'm guessing you'd have to resurrect a Soekris net4801 from the 
way-back-machine to substantiate my orignal claim of 'hundreds.' I'll have 
to remind myself to say 'dozens' from now on.

(I don't have a 'thing' against Soekris -- I just bought 2 off Ebay to 
play with.)

-- 
Thanks in advance,
-------------------------------------------------------------------------
Steve Edwards       sedwards at sedwards.com      Voice: +1-760-468-3867 PST
Newline                                              Fax: +1-760-731-3000