Large deployments: backup and recovery

This documentation applies to Piler enterprise edition 2.2.4+

Revision #1

Publication date: 2026-OCT-18

Backups are implemented with the backup system you already run. What to back up:

What	Where	Notes
MySQL / MariaDB databases	every worker and UI node	The metadata of every archived email, the users, the tenant settings. Back up consistently: a hot backup (e.g. mysqldump --single-transaction or mariabackup), or binary log shipping for point-in-time recovery.
Stored emails and attachments	every worker (/var/piler/store, /var/piler/astore), or the S3 bucket	Millions of small, write-once files: incremental file backups, filesystem snapshots, or object storage versioning / replication.
Manticore search index	every worker	Can be rebuilt from the archive, but at this scale that takes long: back it up as well.
Manticore audit log	the UI nodes	Every access to the archive. Keep it as long as your compliance rules require.
The encryption key, piler.key	the same on every node	Without it the archive can't be read. Keep a copy separately and securely, outside the regular backups too.
Configuration and license	every node	The environment files, the Docker Compose files, the license file.

Stored emails change only in recent directories. The emails are stored as /var/piler/store/<server id>/<tenant>/<xxx>/..., where <xxx> comes from the time the email was archived: a new <xxx> directory is started about every 12 days, and no new email is written to an older one. So after a full backup, only the directories of the last 12 days need to be backed up again; the older ones only change when emails are purged from them. Attachments (/var/piler/astore) are stored by their content, so new files can appear in any of their directories; as every file is written once and never changed, an incremental backup handles them efficiently.

Purged and erased emails stay in the backups until the backups themselves expire: set your backup retention with your compliance rules (e.g. GDPR erasure) in mind.

Order: back up a worker's database first, then its stored emails. Then every email in the database backup has its stored file in the backup; the files of emails archived in between are only extra. The other way round, files older than the database would leave the newest emails without their content. With a DR site, emails archived after the last backup can be recovered from it, as it received them as well.

Restore: restore a worker's database, search index and stored emails to the same point, with the same piler.key, then start its containers. While a site is being restored, a DR site can serve the users.