[nSLUG] Load spikes (Was: Encrypted Emails)

Dop Ganger nslug at fop.ns.ca
Wed May 4 20:35:54 ADT 2016


On Wed, 4 May 2016, Mike Spencer wrote:

> Dop wrote:
>
>> tar -cf - ./source | mbuffer -r 4m -R 3m | ( cd /some/dest && tar -xvf - )
>>
>> This will set a max read rate of 4 megabytes per second and maximum
>> write rate of 3 megabytes per second when copying ./source into
>> /some/dest/.
>
> AFAICT, that's the opposite of what I need.  I seem to have maximal
> reads, hogging RAM while writes are delayed -- even to after cp exits.

Not exactly. The intention here is to make sure there is always data 
available to write, but then to ensure writes are capped at 3 megabytes 
per second; this should keep latency low enough to stop the system getting 
bogged down with big writes.

> But I get the point.  Reversing -r and -R values might do it.  Not
> clear that this deals with the problem of the system allocating
> too much buffer space to a process.

The idea is your hard drive keeps ahead of how far your system is writing. 
That way if it's a problem of data being buffered (on the hard drive or 
the system), mbuffer should slow things down enough that buffers are 
flushed in a timely fashion.

> TYVM; I'll let you know how it works out.

Lots of options to twiddle with. Have fun!

Cheers... Dop.


More information about the nSLUG mailing list