Concurrent backups with Barman and PostgreSQL 9.6

PostgreSQL 9.6 has extended the traditional framework available for physical backups by allowing users to take backups concurrently. Barman will transparently support this new set of functions without requiring the pgespresso extension.

The pgespresso extension, conceived by our Simon Riggs, allowed marking the start and the stop of a backup process even on a read-only standby server. Through pgespresso, Barman users could take physical backups through rsync/Ssh from a standby server, off-loading the actual copy of files from the master server.

This feature is called concurrent backup, and is already available in PostgreSQL through streaming replication protocol, via pg_basebackup.

The last version that pgespresso will support in terms of concurrent backup is PostgreSQL 9.6. Why is that?

The available PostgreSQL API for taking low-level physical backups has been extended in order to natively support concurrent backups (rather, I should use the term “non-exclusive”). I am not sure if Magnus (the author of this patch) was inspired by pgespresso or not, but what is important is that his contribution is definitely more generic and robust (after all, pgespresso was designed to interact only with Barman).

Therefore, PostgreSQL 9.6 and future versions will natively have functions that allow Barman to request a concurrent backup, making pgespresso not necessary.

For more details about this new API, please refer to the section that I have written in the PostgreSQL Wiki page “What’s new in PostgreSQL 9.6”. For now, it is important to know that:

• pg_start_backup() has been overridden and a new parameter specifies whether the backup is exclusive (default) or not;
• a new version of pg_stop_backup() has been provided for concurrent backups: from a technical point of view, this function will now return the content of the backup label and the map of available tablespaces.

What is relevant to our Barman users is that Barman will transparently handle this new API, with no impact on the user experience whatsoever.

By default, Barman will request an exclusive backup (using the traditional set of functions available since PostgreSQL 8). However, you can trigger the new behaviour by setting concurrent_backup in the backup_options global/server configuration option, as follows:

backup_options = concurrent_backup

In the future, PostgreSQL will get rid of traditional functions for exclusive backups in favour of concurrent ones, due to some known dangerous corner cases. In particular, the sudden death of a PostgreSQL server before calling pg_start_backup(), which leaves a backup_label file in the PGDATA directory and prevents the server from restarting (by the way, the barman check command can identify this corner case).

When the new backup API becomes the preferred one, Barman will switch its default accordingly. In the meantime, we invite our users to test this new feature which is currently available only on Github and will be included in Barman 2.0.