2ndQuadrant is now part of EDB

Bringing together some of the world's top PostgreSQL experts.

2ndQuadrant | PostgreSQL
Mission Critical Databases
  • Contact us
  • EN
    • FR
    • IT
    • ES
    • DE
    • PT
  • Support & Services
  • Products
  • Downloads
    • Installers
      • Postgres Installer
      • 2UDA – Unified Data Analytics
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
  • Postgres Learning Center
    • Webinars
      • Upcoming Webinars
      • Webinar Library
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Blog
    • Training
      • Course Catalogue
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
    • Books
      • PostgreSQL 11 Administration Cookbook
      • PostgreSQL 10 Administration Cookbook
      • PostgreSQL High Availability Cookbook – 2nd Edition
      • PostgreSQL 9 Administration Cookbook – 3rd Edition
      • PostgreSQL Server Programming Cookbook – 2nd Edition
      • PostgreSQL 9 Cookbook – Chinese Edition
    • Videos
    • Events
    • PostgreSQL
      • PostgreSQL – History
      • Who uses PostgreSQL?
      • PostgreSQL FAQ
      • PostgreSQL vs MySQL
      • The Business Case for PostgreSQL
      • Security Information
      • Documentation
  • About Us
    • About 2ndQuadrant
    • 2ndQuadrant’s Passion for PostgreSQL
    • News
    • Careers
    • Team Profile
  • Blog
  • Menu Menu
You are here: Home1 / Blog2 / Gabriele's PlanetPostgreSQL3 / Barman is finally out
Gabriele Bartolini

Barman is finally out

July 24, 2012/3 Comments/in Gabriele's PlanetPostgreSQL /by Gabriele Bartolini

Barman, backup and recovery manager for PostgreSQLIt took longer than expected, but we have finally managed to release Barman as open-source under GNU GPL 3.
Barman stands for “Backup and Recovery Manager” and it is an administration tool for disaster recovery of PostgreSQL servers. Currently only Linux systems are officially supported, however Python allows to plan porting and maintenance on different platforms as well.

The idea of starting Barman began about a year ago. We were tired of managing disaster recovery solutions using custom scripts, hard to maintain and to monitor (you probably know what I mean).

Also, we needed a tool specifically designed for disaster recovery – and not high availability as well. Some of the features we were eager to see:

  • remote backup
  • management of multiple servers
  • catalogue of backups
  • management of retention policies
  • incremental backup
  • compression of WAL files
  • etc.

None of the tools available as open source was able to offer all of these features at once.

Remote backup allows database architects to organise a business continuity and/or high scalability cluster (made up of several Postgres servers) so that physical backup of the database server is not invasive.

What we can do now with Barman is to simply setup the archive_command on the master. The rest is performed from a remote server. A simple/typical scenario is the one where one backup server sits in the middle of your Postgres cluster and manages all the backup activity. In case, you can manage the recovery process from that same console – even though you always hope that recovery operations are limited to regular simulations only (remember, it is very important to simulate recovery, usually every 6 months at least).

I already mentioned the centralised architecture of the backup, where you can have a backup server regularly receiving WAL information from all the configured Postgres servers. I said ‘all’, because with Barman you can manage backup (and recovery) of multiple Postgres servers.

Remote backup and management of multiple servers are really handy features. However, one of the features I love the most of Barman is the ability to keep a catalogue of backups per each server. The “barman list-backup” command on a given server, allows you to see all the available periodical backups in your Barman installation for that server.
The dual management of periodical backups (base backups) and WAL archive, allow Barman to assign a specific WAL file to the closest previous backup available. This way, you can actually see the size of a backup, with or without the associated stream of WAL files. (Neat!)

WAL files can be compressed using standard compression algorithms such as gzip and bzip2. Compression is handled by the backup server (not the server).

The catalogue of backups has made much simpler the setup and the implementation of backup policies and retention policies for a Postgres database server. Unfortunately, it is not yet possible to configure retention policies directly in the configuration file, like “recovery window of 30 days” or “redundancy 5” (last 5 periodical backups). But we will get there soon (sponsors are welcome).

Last but not least. Incremental backup. This feature is not that important in small size environments, I know. But it is crucial for large and very database servers (1TB and over). It will definitely be available in the next versions of Barman (again we are looking for sponsors here).

I must admit that we could have released Barman months ago. However, we decided to release it when it was stable enough, being the software highly critical (we are talking about disaster recovery).
We had the chance to install it and test it on some business critical environments in Italy, and the feedback has been great.

So … sorry if it took so long (we started talking about it at PGConf.EU 2011 and PGDay.IT 2011), but I really hope that all of this waiting has been worthwile.

Finally, I want to thank all the other members of the Italian team that worked on the project and supported the development: Marco (lead programmer), Carlo, Gianni and Giulio.

Tags: backup, Barman, Business Continuity, disaster recovery, open source, pitr, postgres, PostgreSQL
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
3 replies
  1. Username*
    Username* says:
    July 25, 2012 at 1:19 pm

    Sorry, it’s a hard wish, but someone using a backup tool, could be using hot-standby too…

    It could be nice if barman had an aditional configuration for hot-standby, serving as a gateway for the transaction logs.

    I love Python, I love PostgreSQL, but unfortunelly I had the idea, not the time to build a patch.

    Reply
    • gabriele.bartolini
      gabriele.bartolini says:
      July 25, 2012 at 1:48 pm

      Hi Daniel,

      I am not sure I understand completely what you mean here. The only very interesting use case I can see of hot standby is with PostgreSQL 9.1+ and replay/pause functions at recovery time (for more accurate Point-In-Time-Recovery operations).

      It is definitely something we have in our mind.

      However, I would be interested in hearing more from your idea.

      Finally: even having the idea is a very very good thing (in my opinion, much more than developing that in many cases). It is important as it allows us to understand and to get users’ feedback and desidered features.

      Thank you,
      Gabriele

      Reply
  2. Joshi MP
    Joshi MP says:
    November 24, 2017 at 2:44 am

    Hi , I started using barman with rsync and link option for my DB so that I can do an incremental backup . Everything works fine, even I am able to restore the DB from the backup.

    The problem I am facing is , the size of the backup (I enabled gzip compression for the wal files ) . The size of the backup is going more than 6 GB randomly. The size of the DB is 20 GB and the base backup also 20 GB . But intermittently the size is going very high . Am i missing anything in the configuration

    thank you.

    Joshi

    Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

Get in touch with us!

Recent Posts

  • Random Data December 3, 2020
  • Webinar: COMMIT Without Fear – The Beauty of CAMO [Follow Up] November 13, 2020
  • Full-text search since PostgreSQL 8.3 November 5, 2020
  • Random numbers November 3, 2020
  • Webinar: Best Practices for Bulk Data Loading in PostgreSQL [Follow Up] November 2, 2020

Featured External Blogs

Tomas Vondra's Blog

Our Bloggers

  • Simon Riggs
  • Alvaro Herrera
  • Andrew Dunstan
  • Craig Ringer
  • Francesco Canovai
  • Gabriele Bartolini
  • Giulio Calacoci
  • Ian Barwick
  • Marco Nenciarini
  • Mark Wong
  • Pavan Deolasee
  • Petr Jelinek
  • Shaun Thomas
  • Tomas Vondra
  • Umair Shahid

PostgreSQL Cloud

2QLovesPG 2UDA 9.6 backup Barman BDR Business Continuity community conference database DBA development devops disaster recovery greenplum Hot Standby JSON JSONB logical replication monitoring OmniDB open source Orange performance PG12 pgbarman pglogical PG Phriday postgres Postgres-BDR postgres-xl PostgreSQL PostgreSQL 9.6 PostgreSQL10 PostgreSQL11 PostgreSQL 11 PostgreSQL 11 New Features postgresql repmgr Recovery replication security sql wal webinar webinars

Support & Services

24/7 Production Support

Developer Support

Remote DBA for PostgreSQL

PostgreSQL Database Monitoring

PostgreSQL Health Check

PostgreSQL Performance Tuning

Database Security Audit

Upgrade PostgreSQL

PostgreSQL Migration Assessment

Migrate from Oracle to PostgreSQL

Products

HA Postgres Clusters

Postgres-BDR®

2ndQPostgres

pglogical

repmgr

Barman

Postgres Cloud Manager

SQL Firewall

Postgres-XL

OmniDB

Postgres Installer

2UDA

Postgres Learning Center

Introducing Postgres

Blog

Webinars

Books

Videos

Training

Case Studies

Events

About Us

About 2ndQuadrant

What does 2ndQuadrant Mean?

News

Careers 

Team Profile

© 2ndQuadrant Ltd. All rights reserved. | Privacy Policy
  • Twitter
  • LinkedIn
  • Facebook
  • Youtube
  • Mail
Greenplum CE 4.2 XML support How to test Greenplum Community Edition 4.2.1 on VirtualBox
Scroll to top
×