Wednesday, July 22, 2009

PostgreSQL Really Slow During Backups

These are my notes from debugging a website problem caused by backing up the underlying database. Symptom: When backing up the database, the website is very sluggish The old backup technique amounted to
pg_dump database > file.sql \
&& rysnc file.sql remote:file.sql
I removed the rsync component and the symptom persisted. Obviously that wasn't the cause. Based on a hint in the PostgreSQL mailing list, I reduced the total IO load by completely eliminating writes with
pg_dump database > /dev/null
That fixed the problem but unfortunately it makes a lousy backup strategy. By compressing the SQL before it hits the disk, I was able to reduce the total IO load and the symptom went away. The final backup incantation amounts to
pg_dump database | gzip --rsyncable > file.sql.gz \
&& rysnc file.sql.gz remote:file.sql.gz

2 comments:

JJ Hendricks said...

I don't know what most of the coding means, but I know the results. The website and database haven't had any crashes during backups since it was implemented.

Anonymous said...

I want to quote your post in my blog. It can?
And you et an account on Twitter?