spqr/backups released - a closer look

Well, writing a PHP-based backup script is not really something a developer really wants to do.

There are too many possible problems:

• Maximum execution time could be too low
• Memory limit could be too low
• Open files limit can easily be reached
• ...

Backups should always be done using the tools your web hosting service provides.

But sometimes a provider does not offer a good backup-solution or demands a lot of money for using it.

Well - so if you do not have another possibility you are invited to use my free extension spqr/backups.

What does this extension do exactly? Let's have a closer look.

The states of development

The first attempt

My first attempt was quite simple: I just looped through every directory and added the files to a zip.

The problem: Large Pagekit projects can not be saved to a zip without increasing PHP's memory_limit and max_execution_time to really high values (in my tests I had to increase memory_limit to 4096G - but not every host allows that large limits.

The second attempt

As I wanted to create an extension that works with as many providers as possible, I had to find a better solution.

My next attempt was:

• Creating an JavaScript-based method, that calls all necessary methods step by step:
  • Building an index of all files and folders
  • Save this index to database (= Queue)
  • Adding a bunch of files to the zip and remove the files from the queue
  • If all files have been added: Close the zip.
  • Upload the final zip

This already worked fine - but: On large sites the collection of the queue took too long. So I also decided to split the indexing-progress into smaller parts that are called using JavaScript.

The extension is now working this way:

• Queue all files in small steps
• Add all files to a zip archive in small steps
• Dump and archive the database
• Create a bundle
• Store the bundle
• Delete all temp files
• Delete old backups, if you would like to keep only x backups (can be configured in the extension's settings)
• Finalize the backup

As we all do not want to click the "Perform backup"-button manually every day I am currently working on CLI support, which allows to backup your Pagekit websites using a cronjob.

Saving backup details

Well - each backup can be created/removed using Pagekit's backend. But under the hood creating and removing a backup is not that simple:

At first the extension generates a random path in Pagekit's temp directory. This path is used to store all temp files. After storing the backup, all these temp files have to be removed. As the backup-creation could be possibly interrupted, these information have to be added to the backup data.

The storing-process itself returns the location of the backup, too. If you switch from local backup to ftp backup, the file rotation still works fine.

If you delete the backup files manually, you are still able to unlink the file from the backup-item in the database; in this case spqr/backups does not try to delete the files if the backup is removed using the backend.

Written by Roman on {{ "2017-09-27T20:38:41+00:00" | date "longDate" }}