2ndQuadrant is now part of EDB

Bringing together some of the world's top PostgreSQL experts.

2ndQuadrant | PostgreSQL
Mission Critical Databases
  • Contact us
  • EN
    • FR
    • IT
    • ES
    • DE
    • PT
  • Support & Services
  • Products
  • Downloads
    • Installers
      • Postgres Installer
      • 2UDA – Unified Data Analytics
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
  • Postgres Learning Center
    • Webinars
      • Upcoming Webinars
      • Webinar Library
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Blog
    • Training
      • Course Catalogue
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
    • Books
      • PostgreSQL 11 Administration Cookbook
      • PostgreSQL 10 Administration Cookbook
      • PostgreSQL High Availability Cookbook – 2nd Edition
      • PostgreSQL 9 Administration Cookbook – 3rd Edition
      • PostgreSQL Server Programming Cookbook – 2nd Edition
      • PostgreSQL 9 Cookbook – Chinese Edition
    • Videos
    • Events
    • PostgreSQL
      • PostgreSQL – History
      • Who uses PostgreSQL?
      • PostgreSQL FAQ
      • PostgreSQL vs MySQL
      • The Business Case for PostgreSQL
      • Security Information
      • Documentation
  • About Us
    • About 2ndQuadrant
    • 2ndQuadrant’s Passion for PostgreSQL
    • News
    • Careers
    • Team Profile
  • Blog
  • Menu Menu
You are here: Home1 / Blog2 / Craig's PlanetPostgreSQL3 / BDR for PostgreSQL: Present and future
craig.ringer

BDR for PostgreSQL: Present and future

December 8, 2014/11 Comments/in Craig's PlanetPostgreSQL /by craig.ringer

For a couple of years now a team at 2ndQuadrant led by Andres Freund have been working on adding bi-directional asynchronous multi-master replication support for PostgreSQL. This effort has become known as the BDR project.

We’re really excited to see these efforts leading to new PostgreSQL features and have a great deal more still to come.

Incremental Development

As a large development project it is neither practical nor desirable to deliver all the changes to PostgreSQL as a single huge patch. That way lies madness and unmaintainable code. It would also be resoundingly rejected by the PostgreSQL community, as it should be.

Instead, BDR has been developed as a series of discrete changes to core PostgreSQL, plus an extension that uses those core changes to implement multi-master asynchronous logical replication.

To make this separation clearer we’ve split BDR into two separate trees since the last official release. We now manage and develop the set of patches to PostgreSQL separately from the BDR extension its self.

BDR is destined for community PostgreSQL

2ndQuadrant’s goal is to get all of BDR – the changes to the core server as well as the extension or its functionality – included in a future open source community PostgreSQL release.

When that’s achieved you’ll be able to use all the features currently provided by the BDR patches and extension with no 3rd party (open source or otherwise) patches, extensions and tools.

All of BDR is open source, released under the PostgreSQL license, and is developed on PostgreSQL’s public infrastructure. The git repository containing the authoritative current version of BDR is on git.postgresql.org.

Getting BDR into users’ hands

With a project as big as BDR and with the PostgreSQL project’s safety-first, conservative and careful development style it’ll be a long time before all of BDR is included in community PostgreSQL. As a result we’ve had to make some compromises in order to get BDR functionality out to users prior to the expected end of the 9.5/9.6 release cycles in a couple of years time.

The main compromise is that we have merged all the not-yet-committed patches to PostgreSQL core that BDR requires into a patched variant of PostgreSQL 9.4 that can be used to run BDR. The need for this patched version to exist is temporary. If all the patches required to run BDR (see below) get into PostgreSQL 9.5 it will be possible to run the BDR extension on a stock community version of PostgreSQL 9.5. We would’ve strongly preferred not to need to make a non-trivially patched version of PostgreSQL available to the public, even temporarily, so hopefully we can do away with it for 9.5.

BDR features in PostgreSQL 9.3, 9.4, 9.5 and onward

Many of the core server changes developed for or required by BDR have already been committed to community PostgreSQL. That means they’re part of released or soon-to-be-released PostgreSQL versions. These features include:

  • Background workers (9.3)
  • Event Triggers (9.3)
  • Replication slots (9.4)
  • Logical decoding (9.4)
  • REPLICA IDENTITY (9.4)

Other changes have been submitted for potential inclusion in PostgreSQL 9.5 or are due to be submitted shortly, including:

  • Commit Timestamps
  • DDL deparse for event triggers
  • Enhanced DROP event triggers
  • Replication Identifiers
  • Sequence Access Methods

BDR-related feature development has also led to other useful work by others, like pg_dump support for using snapshots.

Each of these features is/was a significant development effort in its self – and they’re just the foundations required to make the BDR extension possible.

Adoption of features developed for BDR by other projects

The BDR team is really excited at how far BDR has come and how much of the work done for BDR has already entered community PostgreSQL.

Some of those BDR building blocks are already seeing adoption and use for other purposes. For example, the Slony-I team are working on adopting the logical decoding feature developed for BDR to reduce replication overheads.

Background workers are sure to be a crucial foundation for new PostgreSQL features and capabilities such as built-in task scheduling, and in time I think that we’ll wonder how we ever did without them.

UDR – one-way BDR for unmodified community PostgreSQL 9.4

We’re also pleased to report that a reduced-functionality version of the BDR extension currently known as “UDR” will soon be available for use with PostgreSQL 9.4.

An extension running on unmodified PostgreSQL 9.4 can’t use the not-yet-in-core-PostgreSQL features that BDR relies on to implement full multi-master support. Nonetheless it provides useful path toward live major-version upgrade and an efficient one-way replication tool.

Because PostgreSQL 9.4 does not include support for DDL deparse, full DROP tracking, commit timestamps, or replication identifiers we can’t support DDL replication or multi-master in UDR.

Functionally UDR is quite similar to Londiste, but it avoids the write-amplification of trigger based replication and runs entirely within PostgreSQL, without any need for an external daemon. UDR will offer a useful live-migration path for moving stock 9.4 into BDR vice versa, and in future for online upgrades of 9.4 to later major releases.

More details on UDR will be available soon, as will RPMs.

What’s next for BDR

PostgreSQL is 9.4 due to be released any minute now, and we’re working to have a new BDR release ready to follow shortly afterwards.

Otherwise, the BDR team is focusing on getting all the functionality required to run BDR on top of unmodified PostgreSQL into PostgreSQL 9.5.

Support for replication sets was recently added to the BDR extension. This permits replication of subsets of tables in a database to some nodes, asymmetric replication, and various other configurations that are useful for workloads like data aggregation.

We’re also working on dynamic configuration, where node addition and removal is done using SQL commands instead of by modifying postgresql.conf and restarting the server. That work will lead to a smoother process for node addition and removal.

There are lots of goodies coming, both for core PostgreSQL and for users of the BDR extension.

Tags: asynchronous, BDR, community, features, logical replication, multi-master, open source, Postgres-BDR, PostgreSQL, replication, UDR
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
11 replies
  1. Rahul Yevala
    Rahul Yevala says:
    February 4, 2015 at 7:11 am

    Any update on the UDR? Would like to try it out.

    Reply
    • craig.ringer
      craig.ringer says:
      February 4, 2015 at 7:27 am

      I’ll be tagging BDR 0.8.0 in revision control later today or some time tomorrow, then building RHEL and Fedora packages. So the release is imminent. This release contains support for UDR, which will be automatically enabled if you configure it against a stock PostgreSQL; see the INSTALL.src file in the source distribution for details.

      UDR is considered somewhat experimental in this release, mainly because the user interface to it has already been rewritten in the working branch for the next release. It works fine, we just don’t intend to make a lot of noise about it when people who adopt it will just have to change how they use it soon anyway. Testing and feedback would still be very valuable.

      If you want you can compile it from source against an unmodified PostgreSQL release now. Just:

      git clone -b bdr-plugin/stable git://git.postgresql.org/git/2ndquadrant_bdr.git bdr-plugin

      then
      cd bdr-plugin
      PATH=/path/to/directory/containing/pg_config:$PATH ./configure –enable-udr
      make
      sudo make install

      UDR-specific documentation is still pending. It’s mostly the same to set up as BDR, you just only configure bdr.connections for one side. Feel free to ask questions.

      Reply
  2. Mel
    Mel says:
    February 23, 2015 at 12:17 pm

    When is UDR ready for production deployment?

    Reply
    • craig.ringer
      craig.ringer says:
      March 24, 2015 at 2:52 am

      We’ll be releasing 0.9.0 in the next couple of days, i.e. late March. That contains the first public version of UDR, so it’ll be a good time to start testing it out and seeing if it meets your production needs.

      Reply
  3. DanielF
    DanielF says:
    June 1, 2015 at 1:17 pm

    Hello,
    I tried installing it on CentOs 5 via:
    sudo yum install http://packages.2ndquadrant.com/postgresql-bdr94-2ndquadrant/yum-repo-rpms/postgresql-bdr94-2ndquadrant-redhat-1.0-2.noarch.rpm

    But I get the following errors:
    Failed dependencies:
    rpmlib(FileDigests) <= 4.6.0-1 is needed by postgresql-bdr94-2ndquadrant-redhat-1.0-2.noarch
    rpmlib(PayloadIsXz) <= 5.2-1 is needed by postgresql-bdr94-2ndquadrant-redhat-1.0-2.noarch

    It seems contrary to the documentation, that CentOs5 is not supported.
    Is this right?

    Reply
    • craig.ringer
      craig.ringer says:
      June 1, 2015 at 1:23 pm

      The RPMs are not produced for CentOS 5 anymore, as it’s now extremely obsolete and should not be the subject of any new deployments. Really you shouldn’t be deploying new servers or services with CentOS 6 either, RHEL/CentOS 7 should be the host OS for anything new you’re deploying, whether it’s BDR or anything else.

      Please point me at the documentation that still refers to that so I can correct it. I don’t see it in the http://bdr-project.org docs.

      Reply
  4. Rahul Yevala
    Rahul Yevala says:
    July 17, 2015 at 4:59 am

    Now that PostgreSQL 9.5 alpha is out, how is the work required to run BDR on top of unmodified PostgreSQL coming along?

    Reply
  5. JoshuaPK
    JoshuaPK says:
    January 11, 2016 at 4:56 pm

    I second Rahul’s question. Now that 9.5 has been officially released, what are the next steps for BDR? Was 2ndQuadrant able to get all of the patches included, so that BDR can be installed as an extension? Or, do you plan to release a custom 9.5 with BDR capability?

    Reply
    • craig.ringer
      craig.ringer says:
      January 18, 2016 at 12:08 am

      We weren’t able to get everything required for BDR into 9.5. The community process is slow and if there aren’t enough people available with enough time for patch review or there are too many open issues with a patch it won’t be accepted. That’s what preserves PostgresSQL’s quality, but it can also be frustrating. We’re limited in how much time we can put in for patch review here too so I don’t feel justified in complaining; that’s just how it is when everyone wants to get their features in.

      Despite that I think it’s technically possible to rebuild BDR around 9.5 (without global sequences). The problem is that the versions of the functionality accepted into 9.5 through the community process are very different to what was in BDR for 9.4. Often for the better, but it means it’s not a simple update for the extension. We also have to deal with pg_upgrade support from the patched 9.4 to unpatched 9.5 with different, incompatible representations of the catalogs. Or support for joining 9.5 nodes to 9.4bdr to do a piecemeal migration. There’s a great deal of work involved in getting all that working and it’s not clear there’s a compelling need.

      At this point the plan is to skip 9.5 and focus on 9.6, where we’ll have all the functionality required. We now have pglogical going well and will be able to use that to form the core of BDR on 9.6, so BDR will hopefully be able to become a much simpler extension with no core patches that builds on top of pglogical. More modular, simpler, and more robust.

      I and others will be writing more about this soon.

      Reply
      • Michael Richardson
        Michael Richardson says:
        January 23, 2016 at 10:43 pm

        Okay, to summarize, better to run patched 9.4 than to try to do things with 9.5.
        Will 9.5 run in UDR mode as well?

        Reply
        • craig.ringer
          craig.ringer says:
          January 25, 2016 at 7:56 pm

          UDR works with 9.5, but we’re really focusing on pglogical for 9.5 and newer. It has the same functionality as UDR but it’s much cleaner since we got the chance to really rethink how the user interface should work. pglogical will form the base of a future BDR release and you can think of it as much like a UDR 2.0.

          http://2ndquadrant.com/pglogical

          Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

Get in touch with us!

Recent Posts

  • Random Data December 3, 2020
  • Webinar: COMMIT Without Fear – The Beauty of CAMO [Follow Up] November 13, 2020
  • Full-text search since PostgreSQL 8.3 November 5, 2020
  • Random numbers November 3, 2020
  • Webinar: Best Practices for Bulk Data Loading in PostgreSQL [Follow Up] November 2, 2020

Featured External Blogs

Tomas Vondra's Blog

Our Bloggers

  • Simon Riggs
  • Alvaro Herrera
  • Andrew Dunstan
  • Craig Ringer
  • Francesco Canovai
  • Gabriele Bartolini
  • Giulio Calacoci
  • Ian Barwick
  • Marco Nenciarini
  • Mark Wong
  • Pavan Deolasee
  • Petr Jelinek
  • Shaun Thomas
  • Tomas Vondra
  • Umair Shahid

PostgreSQL Cloud

2QLovesPG 2UDA 9.6 backup Barman BDR Business Continuity community conference database DBA development devops disaster recovery greenplum Hot Standby JSON JSONB logical replication monitoring OmniDB open source Orange performance PG12 pgbarman pglogical PG Phriday postgres Postgres-BDR postgres-xl PostgreSQL PostgreSQL 9.6 PostgreSQL10 PostgreSQL11 PostgreSQL 11 PostgreSQL 11 New Features postgresql repmgr Recovery replication security sql wal webinar webinars

Support & Services

24/7 Production Support

Developer Support

Remote DBA for PostgreSQL

PostgreSQL Database Monitoring

PostgreSQL Health Check

PostgreSQL Performance Tuning

Database Security Audit

Upgrade PostgreSQL

PostgreSQL Migration Assessment

Migrate from Oracle to PostgreSQL

Products

HA Postgres Clusters

Postgres-BDR®

2ndQPostgres

pglogical

repmgr

Barman

Postgres Cloud Manager

SQL Firewall

Postgres-XL

OmniDB

Postgres Installer

2UDA

Postgres Learning Center

Introducing Postgres

Blog

Webinars

Books

Videos

Training

Case Studies

Events

About Us

About 2ndQuadrant

What does 2ndQuadrant Mean?

News

Careers 

Team Profile

© 2ndQuadrant Ltd. All rights reserved. | Privacy Policy
  • Twitter
  • LinkedIn
  • Facebook
  • Youtube
  • Mail
PostgreSQL 9.4 for administrators (part two) Japan PostgreSQL Conference 2014
Scroll to top
×