2ndQuadrant is now part of EDB

Bringing together some of the world's top PostgreSQL experts.

2ndQuadrant | PostgreSQL
Mission Critical Databases
  • Contact us
  • EN
    • FR
    • IT
    • ES
    • DE
    • PT
  • Support & Services
  • Products
  • Downloads
    • Installers
      • Postgres Installer
      • 2UDA – Unified Data Analytics
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
  • Postgres Learning Center
    • Webinars
      • Upcoming Webinars
      • Webinar Library
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Blog
    • Training
      • Course Catalogue
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
    • Books
      • PostgreSQL 11 Administration Cookbook
      • PostgreSQL 10 Administration Cookbook
      • PostgreSQL High Availability Cookbook – 2nd Edition
      • PostgreSQL 9 Administration Cookbook – 3rd Edition
      • PostgreSQL Server Programming Cookbook – 2nd Edition
      • PostgreSQL 9 Cookbook – Chinese Edition
    • Videos
    • Events
    • PostgreSQL
      • PostgreSQL – History
      • Who uses PostgreSQL?
      • PostgreSQL FAQ
      • PostgreSQL vs MySQL
      • The Business Case for PostgreSQL
      • Security Information
      • Documentation
  • About Us
    • About 2ndQuadrant
    • 2ndQuadrant’s Passion for PostgreSQL
    • News
    • Careers
    • Team Profile
  • Blog
  • Menu Menu
You are here: Home1 / Blog2 / Craig's PlanetPostgreSQL3 / BDR History and future
craig.ringer

BDR History and future

October 18, 2016/0 Comments/in Craig's PlanetPostgreSQL /by craig.ringer

BDR is both a patch to PostgreSQL core and an extension on top of PostgreSQL core. How did that come about, and what’s it’s future?

Development of BDR was initiated around the time PostgreSQL 9.2 was in development. Arguably earlier if you count things like the extension mechanism. The goal of BDR is, and has always been, to add necessary features to core PostgreSQL to perform asynchronous loosely-coupled multi-master logical replication.

BDR improvements to core PostgreSQL

Since it’s such a large set of changes it was necessary to structure development as a series of discrete features. A natural dividing line was “things that require changes to the core PostgreSQL code” vs “things that can be done in an extension”. So the code was structured accordingly, making BDR a set of patches to core plus an extension that uses the added features to implement a multimaster node. The required features were developed in a branch of PostgreSQL and then extracted and submitted one by one to PostgreSQL core. BDR is, and has always been, much too big to simply commit to PostgreSQL in one go.

Getting the BDR extension running on unmodified PostgreSQL

BDR 1.0 is still in two parts – a set of patches to add necessary features to core PostgreSQL and an extension that uses the features to implement multimaster. The original goal was to have BDR running on stock 9.4 without patches, but it just took too long to get all the required patches through the community process and into core. This isn’t entirely a bad thing, since it’s led to the eventual features being of higher quality when they were committed to 9.4, 9.5 and 9.6.

Now, as of PostgreSQL 9.6, all of the patches required to make it possible to implement BDR-style replication have been included in PostgreSQL core. As mentioned in an earlier post, BDR on 9.6 should run as an extension on an unmodified PostgreSQL. The implementation of BDR itself is still an outside extension, so this doesn’t mean “BDR is in 9.6”, but it’s big progress.

Other enhancements related to BDR

Meanwhile, the “pglogical” single-master logical replication tool was extracted from BDR, enhanced, simplified, and submitted to core PostgreSQL as a further building block toward getting full multi-master logical replication into core. The submission was not successful in its original form, but the 2ndQuadrant team is now working on a replication feature for PostgreSQL 10 that is based on that work.

Once logical replication (not just logical decoding) is incorporated into PostgreSQL core, the BDR extension will be adapted to use the in-core logical replication features. Over time, more parts of the functionality of the BDR extension will be submitted to PostgreSQL until all BDR functionality is in community PostgreSQL.

This process has taken years and still has a ways to go. PostgreSQL’s development is careful and prioritizes stability. Submissions often need multiple rounds of revisions before they are accepted for community distribution. Sometimes they’re completely rejected and a different design must be pursued.

In the meantime, the BDR extension – and pglogical – will continue to meet user needs.

Features added to core PostgreSQL for BDR

Along the way, we’ve delivered some great features to PostgreSQL that other projects can use too:

  • Background workers let you run tasks inside PostgreSQL, like schedulers, etc. They’ve already been used to help with things like parallel query.
  • Event triggers and DDL deparse hooks let you set up user defined triggers on database change events like table creation. They’re the foundation of DDL replication in BDR and are usable by other products that are interested in tracking, auditing, and replicating changes to database structure.
  • Replication slots simplify WAL retention management for physical standbys and provide a foundation for logical decoding of changes.
  • Replication origins are the other side of replication slots. They let a downstream server efficiently keep track of how much they’ve replayed from a replication slot. Unlike most of these features, replication origins are not useful for much except logical replication.
  • Logical decoding reads a stream of low level binary changes from WAL on a server and turns it into a stream of logical row changes in a format suitable for replication across PostgreSQL versions or even to different products and applications. It’s the foundation on which BDR, pglogical, and in-core logical replication are built.
  • Logical WAL messages let extensions, user defined functions, etc write special messages into WAL that are not associated with any particular table. They may optionally bypass normal transactional rules, like a very limited version of an autonomous transaction. These are used when different nodes signal each other about changes; for example, BDR uses them for global DDL locking. These messages will also be useful to applications that adopt the logical decoding interface for change streaming, auditing, etc – for example, to record the identity of a user who performs an action when streaming audit history to Kafka.
Tags: BDR, history, in-core-logical-replication, logical replication, pglogical, Postgres-BDR
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

Get in touch with us!

Recent Posts

  • Random Data December 3, 2020
  • Webinar: COMMIT Without Fear – The Beauty of CAMO [Follow Up] November 13, 2020
  • Full-text search since PostgreSQL 8.3 November 5, 2020
  • Random numbers November 3, 2020
  • Webinar: Best Practices for Bulk Data Loading in PostgreSQL [Follow Up] November 2, 2020

Featured External Blogs

Tomas Vondra's Blog

Our Bloggers

  • Simon Riggs
  • Alvaro Herrera
  • Andrew Dunstan
  • Craig Ringer
  • Francesco Canovai
  • Gabriele Bartolini
  • Giulio Calacoci
  • Ian Barwick
  • Marco Nenciarini
  • Mark Wong
  • Pavan Deolasee
  • Petr Jelinek
  • Shaun Thomas
  • Tomas Vondra
  • Umair Shahid

PostgreSQL Cloud

2QLovesPG 2UDA 9.6 backup Barman BDR Business Continuity community conference database DBA development devops disaster recovery greenplum Hot Standby JSON JSONB logical replication monitoring OmniDB open source Orange performance PG12 pgbarman pglogical PG Phriday postgres Postgres-BDR postgres-xl PostgreSQL PostgreSQL 9.6 PostgreSQL10 PostgreSQL11 PostgreSQL 11 PostgreSQL 11 New Features postgresql repmgr Recovery replication security sql wal webinar webinars

Support & Services

24/7 Production Support

Developer Support

Remote DBA for PostgreSQL

PostgreSQL Database Monitoring

PostgreSQL Health Check

PostgreSQL Performance Tuning

Database Security Audit

Upgrade PostgreSQL

PostgreSQL Migration Assessment

Migrate from Oracle to PostgreSQL

Products

HA Postgres Clusters

Postgres-BDR®

2ndQPostgres

pglogical

repmgr

Barman

Postgres Cloud Manager

SQL Firewall

Postgres-XL

OmniDB

Postgres Installer

2UDA

Postgres Learning Center

Introducing Postgres

Blog

Webinars

Books

Videos

Training

Case Studies

Events

About Us

About 2ndQuadrant

What does 2ndQuadrant Mean?

News

Careers 

Team Profile

© 2ndQuadrant Ltd. All rights reserved. | Privacy Policy
  • Twitter
  • LinkedIn
  • Facebook
  • Youtube
  • Mail
Back to the Future Part 3: pg_rewind with PostgreSQL 9.6 PostgreSQL and the Platinum Guarantee
Scroll to top
×