[Asterisk-Dev] Performance bottleneck in voicemail2 on nfsed huge mailboxes

[John Todd]

>On Sun, 2003-09-28 at 05:17, asterisk at billheckel.com wrote:
>>  There has been a long standing design flaw in voicemail that I 
>>have mentioned repeatedly.  If you
>>  are listening to your voicemail and another voicemail comes in it 
>>ends up orphaned until you get at
>>  least that many voicemails in your box.  I had looked into a 
>>simple patch to fix this before but it
>>  requires a fairly serious redesign of the numbering scheme.  All 
>>voicemail should have a unique ID
>>  that is globally unique and time based ( like unix milliseconds ) 
>>so that incoming voicemail is
>>  always in order.
>>
>>  If each mail is named using this and the voicemail listening 
>>section checks for the total number of
>>  existant files it will work but there is a bit of work to be done. 
>>I had started to rewrite this
>>  but when voicemail2 was announced I assumed it would be corrected.
>
>I know. I have been looking into the code a lot yesterday. Just
>"changing the filenames" isn't as easy as I thought it to be. As you
>say, it would result in an almost complete rewrite of voicemail. The msg
>number is being used in almost any corner of the voicemail application
>in a call to make_file(). So anytime a message is accessed, it uses the
>message's number to generate a file name which it accesses. The message
>number is also used for "navigating" the voicemails.
>
>As far as I can see, this behaviour is both in voicemail and voicemail2.
>
>So, the first step, would be to redesign that beast. First and the most
>important thing: Storage (re-)design. A database based approach would
>certainly be interesting,  but I (personally ;) want * to remain
>operational without mysql. I thought of some sort of front-/back-end
>design like CDR. If we could abstract the voicemail storage, we could
>have different backends (mysql, file, etc.) for storing the mails and
>the voicemail apps (frontends) just access those storages.
>
>So, defining the storage interface is up first, IMHO. What do you people
>think?
>
>BTW, my first idea of storing the voicemail in a temporary directory and
>copying the message to the box afterwards would certainly work to
>overcome my "two seconds loss"-problem, but there is an unpleasant side
>effect: the copy would stress the * box by maxing out I/O. When *
>records voicemail, it distributes it to n formats simultaneously which
>results in minimal i/o at a time.  So copying the voicemails afterwards
>isn't a solution. For now, I have patched voicemail2 to emit a beep when
>it is really ready to start recording (after it has chosen the
>filename), not before.  But this is a very dirty solution.
>
>Bye,
>  Lev.
>
[snip]

I would agree that voicemail2 needs to be re-written with an 
abstraction layer for databases for the storage of voicemail as well 
as the user data.  The ability to use a remote storage repository for 
voicemail files, as well as the ability for multiple * front end 
servers on the same VM data is a very appealing idea.  (Don't forget 
locking!)  Postgres, MySQL, or just plain old ODBC might be the best 
way to do it (I don't know about how difficult it would be to use 
ODBC without specific tweaks for each system - I'm not a programmer, 
just an integrator.)  In any case, it should be a generic framework 
that could support multiple database types to match what the site was 
using.

However, backwards compatibility should remain intact, so that flat 
file storage can continue to work for the "simple" systems that will 
continue to exist.

The current method of storing VM messages in multiple formats seems 
to be a bit aggressive; I trim it down to just gsm, and that works 
for me.  In other words, I don't think that any future system should 
store the VM in the multiple formats - it should translate the 
message on the way out, from some "universal" format.  Why do work 
that you don't know you need to?  Plus, processors are fast enough 
these days that doing real-time translation on demand is certainly 
reasonable.  Therefore, I don't see the problem with your temp file 
creation scheme in the long run.

JT