[asterisk-dev] How to improve astdb? Any ideas?

[Ron Arts]

Op 06-09-10 16:16, Mark Murawski schreef:
>
> On 09/06/10 10:05, Ron Arts wrote:
>> Op 06-09-10 15:46, Mark Murawski schreef:
>>>    Don't fear the fsync
>>> http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/don%E2%80%99t-fear-fsync
>>>
>>>
>>> Why would running fsync in a loop be any different in performance
>>> than running fsync in the thread waiting to hear back from astdb put?
>>> The loop approach would also be beneficial in *reducing* fsyncs,
>>> since you could batch say 10 writes at a time
>>> (assuming the underlying storage supports it).. with a 60 second
>>> timeout where if you didn't fill up the write bucket it would write
>>> them out anyway. Otherwise the thread would be idle and waiting for
>>> put requests.
>>>
>>>
>>> On 09/06/10 09:37, Ron Arts wrote:
>>>> Op 06-09-10 15:10, Mark Murawski schreef:
>>>>> Here is one fix for the fsync problem.
>>>>>
>>>>> Spawn an astdb put worker thread. (Same goes for using any database
>>>>> backend) Writers will dump their changes to a linked list and return
>>>>> immediately. The writer will write whenever it gets a chance. if there
>>>>> are big or small slowdowns in fsync it wont affect any running
>>>>> threads.
>>>>> And if you lose power, well... you would have lost the unwritten data
>>>>> anyway because you would have been waiting on it anyway.
>>>>>
>>>>
>>>> Please note: if any process or thread on a system is doing fsync()
>>>> in a loop, the entire system will slow to a crawl.
>>>>
>>>> Ron
>>>>
>>
>>
>> fsync writes all buffers to disk, it blocks until this is done. This
>> means that C library
>> will push out all its buffers to the OS, and the OS will push out
>> everything to disk.
>> This will result in hard disk head movements. If you call fsync in a
>> tight loop,
>> and there are modified buffers each iteration, then the other
>> processes in the system
>> will have a hard time getting the head to move in their direction.
>>
>> Ron
>>
>>
>
> Tight loop, or each thread doing it's own thing.  What difference does
> it make?
>
> Example A:
> after 1 second: Thread 1 does an astdb put (fsync)
> after 2 seconds: Thread 2 does an astdb read
> after 3 seconds: Thread 3 does an astdb put (fsync)
> after 4 seconds: Thread 4 does an astdb put (fsync)
>
> You're now blocking on three fsyncs.
>
> Example B:
> after 1 second: Thread 1 does a delayed astdb put
> after 2 seconds: Thread 2 does a read (which should start with the async
> put list first to ensure up-to-date data)
> after 3 seconds: Thread 3 does a delayed astdb put
> after 4 seconds: Thread 4 does a delayed astdb put
> after 56 seconds: Astdb worker writes three records and does *one* fsync
>
> There is no tight loop.  You're writing data out in 60 second intervals
> (or each 10 database updates, or whatever the interval).  Which is way
> less frequent than example A where each write waits for an fsync because
> astdb put does a astdb->sync for each put.
>
>
>

I agree, but I was looking at the difference between each thread doing it's own
fsync, or moving all those fsyncs to a separate thread. That was what I read from
your original mail. I had it happen to me in a similar situation that the separate
thread made the system slow to a crawl, because it was doing an fsync() after
each write(). If you save up a couple of writes, that would indeed improve
the situation, that's what I did.

Ron


-- 
Netland Internet Services BV
bedrijfsmatige internetoplossingen

http://www.netland.nl  Science Park 140           1098 XG Amsterdam
info: 020-5628282      servicedesk: 020-5628282   fax: 020-5628281
KvK Amsterdam 33253286

Op dit bericht is de volgende disclaimer van toepassing:
http://www.netland.nl/maildisclaimer