Today we have released new, REST API-based versions of backup and restore tools. Executables are available for download here along with source code.
Backup
New backup tool, while maintaining the same process logic is two times faster – backing up 20 tables with 30K records and 200Mb attachments now takes 30 minutes while old backup does the same in about an hour. Also, code revision allowed us to significantly decrease memory workload on large data sets. Other changes include:
- -t (-type) switch is deprecated. We've found out many people always performing full backup thinking it is "safer". The assumption is wrong, in case of failure original data files are not modified with either method but doing full backup leads to unnecessary increase in database workload and network traffic. If you really want to backup anew, simply delete all .CSV files from the backup folder.
- -d (-delimiter) switch is added to control separator between columns' values. You can use it to override default TAB character with, say, comma or semicolon.
- You now can use configuration files to provide settings shared for all backup processes and use command line parameters to provide only missing arguments.
- Instead of storing table structure in a one-per-table .SCHEMA.CSV file, backup now uses file format understood by restore tool – you can back up one database and restore it to another in just two commands.
- First column in CSV file that store internal record id is now renamed from "id" to "@row.id" to avoid case-insensitive name clash with "Id" (default key naming)
- Dates were backed up with 00:00:00 time – now only date portion is written.
- Numeric values were backed up with 6 trailing zeroes. Now unnecessary zeroes are stripped out.
- Backup now explicitly checks for "one database – one folder" preventing re-use of the folder to hold backups from two different databases. This logic was present in a GUI version of old backup but was not checked by command line utility. Now logic matches across all tool versions.
- The tool now checks for another backup instance writing data to the same folder and, if found refuses to continue to avoid clashes.
Restore
New restore tool's performance increase is even more impressive – instead of "query if record exists – create or update the record – upload attachments – repeat" loop, new tool, thanks to API switch, packs records and attachments together in batches by 500 records/20Mb and sends whole batch to the server to let API decide what to do. As the result of this improvement as well as shared backup-restore configuration format the same sample set of 20 tables/30K records/200Mb attachments was restored in 30 minutes while with old restore tool it would take ages to first configure restoration logic and then perform the process. Also, instead of bailing out on first update failure, bad row's data is written to a separate file alongside original data file. Their structure match except for error file's last column contains error message. You can than review failures, fix bad data and feed error file in place of original data file to retry restoration process only for faulted rows.
Also, restore's command line switches were brought in par with backup options. And, for your convenience we have added GUI version of restore tool.
Enjoy.
