[asterisk-dev] How to improve astdb? Any ideas?

[Mark Murawski]

  Here is one fix for the fsync problem.

Spawn an astdb put worker thread. (Same goes for using any database 
backend) Writers will dump their changes to a linked list and return 
immediately.  The writer will write whenever it gets a chance.  if there 
are big or small slowdowns in fsync it wont affect any running threads.  
And if you lose power, well... you would have lost the unwritten data 
anyway because you would have been waiting on it anyway.

I think one goal should be to make asterisk as disk io agnostic as 
possible.  I've already ran into major problems using the Monitor() app 
due to disk io.  The disk writing is done in the same thread as audio 
bridging, so if you have high disk io contention (which I had), audio 
bridging suffers while you're recording a call.  MixMonitor() spawns a 
new thread and avoids this problem.  (Ideally Monitor() should also be 
rewritten to spawn a new thread, but that's another issue entirely).



On 09/06/10 04:38, Stefan Schmidt wrote:
> Am 06.09.2010 05:17, schrieb Mark Murawski:
>>    I remember talking to some main people in #asterisk-dev about a
>> possible plan to ditch astdb and go with sqlite under the hood.  sqlite
>> is really easy to incorporate into an app and is in public domain.  It
>> can also be used completely in-memory, which will avoid the disk
>> bottlenecks you're hitting.
>>
>>
>>
> sqlite use fsync itself so there would be the same problem atleast. only
> if you use it in-memory only but i think the berkley db (AFAIK astdb is
> a berkley) is faster than sqlite.
>
> if you run sqlite as a standalone process you wouldnt loose data if
> asterisk crash but if your system goes down (power loss) you will still
> loose the data.
>
> the other problem i see with this its just moving the problem down one
> level, even its possible to run sqlite on all system where asterisk is
> available (i dont know) you have to make sure the user which runs
> asterisk could also runs sqlite.
>
> if you just buildin sqlite into asterisk there is still the data loss
> problem if asterisk crashes.
>
> the bottleneck is only there if you wait for the write buffers to be
> written to the disk (thats what fsync does)
>
> i think if you want to use sqlite there should also be the possibility
> to use another DB. Think of sharing astdb through different servers
> would be a cool solution for fallback ;)
>
> overall this would need a change to astdb to split it up into a frontend
> (astdb_put, del and deltree) which are the functions which are called
> from other asterisk parts and a backend which transmit these data into
> the DB structure.
>
>
> best regards
>
> steve
>