2ndQuadrant is now part of EDB

Bringing together some of the world's top PostgreSQL experts.

2ndQuadrant | PostgreSQL
Mission Critical Databases
  • Contact us
  • EN
    • FR
    • IT
    • ES
    • DE
    • PT
  • Support & Services
  • Products
  • Downloads
    • Installers
      • Postgres Installer
      • 2UDA – Unified Data Analytics
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
  • Postgres Learning Center
    • Webinars
      • Upcoming Webinars
      • Webinar Library
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Blog
    • Training
      • Course Catalogue
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
    • Books
      • PostgreSQL 11 Administration Cookbook
      • PostgreSQL 10 Administration Cookbook
      • PostgreSQL High Availability Cookbook – 2nd Edition
      • PostgreSQL 9 Administration Cookbook – 3rd Edition
      • PostgreSQL Server Programming Cookbook – 2nd Edition
      • PostgreSQL 9 Cookbook – Chinese Edition
    • Videos
    • Events
    • PostgreSQL
      • PostgreSQL – History
      • Who uses PostgreSQL?
      • PostgreSQL FAQ
      • PostgreSQL vs MySQL
      • The Business Case for PostgreSQL
      • Security Information
      • Documentation
  • About Us
    • About 2ndQuadrant
    • 2ndQuadrant’s Passion for PostgreSQL
    • News
    • Careers
    • Team Profile
  • Blog
  • Menu Menu
You are here: Home1 / Blog2 / Craig's PlanetPostgreSQL3 / BDR is coming to PostgreSQL 9.6
craig.ringer

BDR is coming to PostgreSQL 9.6

September 21, 2016/39 Comments/in Craig's PlanetPostgreSQL /by craig.ringer

I’m pleased to say that Postgres-BDR is on its way to PostgreSQL 9.6, and even better, it works without a patched PostgreSQL.

BDR has always been an extension, but on 9.4 it required a heavily patched PostgreSQL, one that isn’t fully on-disk-format compatible with stock community PostgreSQL 9.4. The goal all along has been to allow it to run as an extension on an unmodified PostgreSQL … and now we’re there.

The years of effort we at 2ndQuadrant have put into getting the series of patches from BDR into PostgreSQL core have paid off. As of PostgreSQL 9.6, the only major patch that Postgres-BDR on 9.4 has that PostgreSQL core doesn’t, is the sequence access method patch that powers global sequences.

This means that Postgres-BDR on 9.6 will not support global sequences, at least not the same way they exist in 9.4. The 9.6 version will incorporate a different approach to handling sequences on distributed systems, and in the process address some issues that arose when using global sequences in production.

Since Postgres-BDR on 9.6 is now passing regression tests (except for global sequences), I expect to have an alpha release out not long after the release of PostgreSQL 9.6 itself. Keep an eye out!

There’s also a pglogical 1.2.0 update coming to coincide with the release of PostgreSQL 9.6.

At the same time, Petr has submitted another revision of logical replication for core PostgreSQL to the 10.0 development series.

Along with updating Postgres-BDR in concurrence with 9.6, there is work in progress to enhance Postgres-BDR’s HA capabilities. I’ll have more to say on that in an upcoming post.

We’ve been busy here at 2ndQuadrant.

Tags: BDR, logical decoding, logical replication, pglogical, Postgres-BDR, PostgreSQL, replication
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
39 replies
  1. Rolphy Reyes
    Rolphy Reyes says:
    September 21, 2016 at 2:06 pm

    Hi.

    Excuseme my ignorance.

    Are you telling that Postgres 9.6 will include this feature built-in? Not a fork version?

    Sorry for my bad english.

    Reply
    • craig.ringer
      craig.ringer says:
      September 22, 2016 at 2:09 am

      You will still need to add the BDR extension. But you will be able to use unmodified PostgreSQL 9.6, not the BDR PostgreSQL fork.

      So “sort-of”. It’s now more like installing pglogical or PostGIS, something you can add to an existing PostgreSQL install.

      Reply
  2. andreak
    andreak says:
    September 22, 2016 at 7:11 am

    So, to the “I want this feature” question; Will BDR ever support EXCLUTION-constraints?
    If not, do you have any suggestions on how to design around it?

    Thanks.

    Reply
    • craig.ringer
      craig.ringer says:
      September 22, 2016 at 7:33 am

      The only way BDR can ever support exclusion constraints is by doing synchronous changes. Once we can decode 2-phase transactions, we’ll be able to PREPARE TRANSACTION and replicate the xact to all nodes, where we PREPARE TRANSACTION there too. Only once all nodes confirm success will we COMMIT PREPARED on all nodes.

      Needless to say, this will be very slow, and only work when all nodes are up. It’s the same approach we want to use to clean up DDL handling.

      Fundamentally, exclusion constraints just don’t make sense in an asynchronous multi-master replication scheme.

      Reply
      • andreak
        andreak says:
        September 23, 2016 at 9:37 am

        How is this different from having triggers ensuring consistency? One can use triggers to achieve the same as EXCLUSION-constraints…

        Reply
        • craig.ringer
          craig.ringer says:
          September 27, 2016 at 2:52 am

          BDR doesn’t fire triggers on replicated rows, it runs as session_replication_role = replica. So you can try, but it won’t work.

          State between nodes is asynchronously replicated, so it’s eventually consistent. That means that anything relying on a single-point-in-time globally consistent view of the system state will not work.

          If you need global point-in-time consistency you need another product that does global transaction management, global lock management, has global snapshots, etc. The trade-off is it’ll have poor performance over high latency networks and won’t tolerate network partitions.

          Reply
  3. Dejv
    Dejv says:
    September 22, 2016 at 7:45 am

    Amazing! Thanks, i really glad to hear it. We are waiting on release:)

    Reply
  4. Tomas
    Tomas says:
    September 22, 2016 at 2:25 pm

    What is the plan regarding Windows implementation?

    Reply
    • craig.ringer
      craig.ringer says:
      September 23, 2016 at 1:37 am

      Customer priorities are elsewhere, so it’s not likely to happen soon unless a support/consulting customer needs it and funds it. Right now we’re more focused on HA improvements.

      Adding Windows support probably isn’t that hard. Just have to make it work with EXEC_BACKEND and add a Windows-friendly CreateProcessEx(...) variant of the bdr_dump invocation code in bdr_init_copy.c. We go to some effort not to rely on direct access to memory inherited by fork()ing from the postmaster in the BDR management routines so it shouldn’t be a ton of work.

      If you or someone else who needs it submits a good quality patch I’d be happy to review and integrate it.

      Reply
      • John Male
        John Male says:
        October 22, 2017 at 12:20 pm

        Hi

        I am working at a place that really needs the ability to run on Windows. I do not mind taking a look to see if I can get that to work. I do think it will be some mind-boggling fun to see if I can get this great product to fill our niche need.

        If I could ask some simple questions. Would the location of the code change now need to be in bdr_init_replica.c in the bdr_init_exec_dump_restore method. I was looking at the code and it looks like this is probably the correct location.

        Reply
        • craig.ringer
          craig.ringer says:
          February 2, 2018 at 3:05 pm

          There was prior discussion of this topic on the BDR mailing list; you may want to start with the archives there.

          It’s not a big job, just not a high priority at this time.

          Reply
  5. harry
    harry says:
    September 23, 2016 at 8:45 am

    what is that pglogical 1.2.0 update.?

    curious to know.

    Reply
    • craig.ringer
      craig.ringer says:
      September 27, 2016 at 2:53 am

      The release notes will come along with the release in a week or so. It’s a maintenance release not a major feature update.

      Reply
  6. rberger
    rberger says:
    September 25, 2016 at 11:38 pm

    Does this mean that BDR will be albe to work with AWS RDS once AWS RDS supports PostgreSQL 9.6?

    Reply
    • craig.ringer
      craig.ringer says:
      September 27, 2016 at 2:48 am

      That’s up to Amazon. Amazon curates the extensions they allow to run on RDS, you cannot just install any extension you like on their sealed-system platform.

      To run BDR you must both be able to install C extensions and make connections as superuser. RDS does not allow anyone except Amazon to do either, so it’s up to them.

      Reply
  7. Uttam
    Uttam says:
    September 29, 2016 at 8:05 am

    You mentioned different approach to handle global sequences. Can you please elaborate or give link to details of different approach we need to take?

    Reply
    • craig.ringer
      craig.ringer says:
      September 29, 2016 at 8:09 am

      I’m working on a new model for inclusion in 9.6-compatible BDR, but it won’t be in the alpha release, which will not have global sequences at all. When complete it’ll be covered in the documentation. I haven’t written it up at this point and don’t have time to explain in detail right now.

      If you decide to try the alpha of 9.6bdr or want to avoid use of seqam-based global sequences on 9.4bdr you can use simple step/offset sequences.

      Reply
  8. cilurnum
    cilurnum says:
    September 29, 2016 at 9:00 am

    What about 9.5?

    Reply
    • craig.ringer
      craig.ringer says:
      September 29, 2016 at 12:17 pm

      There will not be BDR support for 9.5, as I’ve been saying for some time.

      The patches required to make it work on 9.5 didn’t get in in time, they’re only in 9.6. So BDR for 9.5 would require another custom-patched PostgreSQL, which is more code maintenance burden. The only reason to bother supporting 9.5 would’ve been if it let us deliver BDR on unpatched PostgreSQL earlier. So it’s only targeting 9.6.

      We’re continuing to support customers using BDR on 9.4 and will continue to enhance it for 9.4bdr and 9.6, with a growing focus on 9.6 as support matures.

      Reply
  9. mahorfaly
    mahorfaly says:
    September 29, 2016 at 11:52 am

    Is this mentioned in the release notes of 9.6?

    Reply
    • craig.ringer
      craig.ringer says:
      September 29, 2016 at 12:17 pm

      No. 3rd party extensions usually aren’t.

      Reply
  10. dhanuj
    dhanuj says:
    September 30, 2016 at 10:33 am

    I’ve installed postgres-9.6 (unmodified). Where can I get the bdr rpm to install on top of this to enable BDR extension ?

    Reply
    • craig.ringer
      craig.ringer says:
      September 30, 2016 at 1:15 pm

      You don’t, yet. This post isn’t “hey, BDR for PostgreSQL 9.6 is ready and there are packages out”. It’s a heads-up that it’s well under way and will be entering early preview soon.

      When released for preview it will probably only be available as source code for you to compile yourself. Partly because at that stage we won’t want anyone getting the idea it’s ready for production use on 9.6 yet. The updated DDL replication and sequence handling will probably still be too raw.

      Keep an eye out, when there are RPMs it’ll be announced. But it’ll be quite a while yet, well after we do a preview release and likely an alpha or two. We’re unlikely to do RPMs before beta.

      If you’re really really keen you can try the current codebase from the relevant branch of the BDR project github. Be aware it’s subject to rebasing and regular incompatible updates.

      Reply
      • Sinc
        Sinc says:
        November 4, 2016 at 3:02 pm

        Can you give us an idea when “quite a while” might be? Is that likely to be 2016, or March or late 2017, or when? My company is looking for this feature and we’re wondering when a relatively safe version based on 9.6 might be ready.

        (And of course I understand that you can’t commit to a date, but I’d love a best guess at this point.)

        Thanks

        Reply
        • craig.ringer
          craig.ringer says:
          November 7, 2016 at 9:52 am

          This year, “when it’s ready”. If it’s important to you consider whether it’s something you’d like to work with us more directly on.

          Reply
  11. fve
    fve says:
    October 10, 2016 at 2:34 pm

    Hello

    Thanks for all this. If I’d like to experiment with this, should I source install from https://github.com/2ndQuadrant/bdr/tree/bdr-plugin/dev-bdr96 ?

    Many thanks

    Freddie

    Reply
    • craig.ringer
      craig.ringer says:
      October 11, 2016 at 3:03 am

      Yep. Be aware that the branch is subject to rebasing and forced pushes. The docs aren’t updated yet, and some tests are still known to fail on 9.6. The extension control files are updated in place, so pulling new versions of the branch may render existing installs incompatible. There’s a known bug with GRANTs not replicating. Don’t let it anywhere near production systems yet.

      I’d be interested in your comments on the DDL replication model used for 9.6bdr. We’re currently evaluating whether it’s good enough or whether we need to update DDL deparse and return to an event trigger based approach.

      Once the new global sequence code is completed I’d value input on it too; see bdr_seq2.c . Right now it works but doesn’t allocate node sequence IDs yet, I’m working on the required voting for that.

      Reply
  12. fve
    fve says:
    October 11, 2016 at 4:16 pm

    Hi Craig

    Thanks for this.

    https://github.com/2ndQuadrant/bdr/tree/bdr-plugin/next (upd. 8 days ago) has a README.bdr mentioning PG 9.4.

    I’ll ignore that README and take the dev-bdr96 as you suggest.

    Likewise your page (https://github.com/2ndQuadrant) says

    postgres
    forked from postgres/postgres
    Patches and working trees for commitfests and public hackers posts.
    Updated a day ago

    https://github.com/2ndQuadrant/postgres/tree/master has a last update of Apr 17. The other most active branches are 6+ days old.

    Should I take the ‘official’ PG from https://github.com/postgres/postgres (last updated 19 hours ago) ?

    It seems Apr 17 is too far back; and I probably should not take your 6-day-old branch (dev/backport-95-tap-to-95 Updated 6 days ago by ringerc).

    I’ll try both in order for now.

    Of particular interest is whether the ALTER TABLE limitation described in http://stackoverflow.com/questions/32748630/django-1-8-migration-with-postgres-bdr-9-4-1

    Thanks

    Freddie

    Reply
    • craig.ringer
      craig.ringer says:
      October 12, 2016 at 1:06 am

      Short-version answer for others here: build against stock community postgres 9.6. The 2ndQuadrant ‘postgres’ repo is unrelated and, per the description, used to track and collaborate on patches destined for core.

      The docs are not updated at all in 9.6bdr yet, that includes the README.

      The limitation on full table rewrites in ALTER is not lifted. It’s technically possible, but requires extra work that isn’t currently roadmapped. If it’s important to you please get in touch with the consulting team – our priorities are driven by customer needs.

      Reply
  13. fve
    fve says:
    October 11, 2016 at 6:23 pm

    Hello

    I’m encountering some issues building bdr from source (branch bdr-plugin/dev-bdr96) but don’t want to spam here; could you tell me where to mail/post/etc if relevant and possible ?

    F

    Reply
    • craig.ringer
      craig.ringer says:
      October 12, 2016 at 1:01 am

      Use [email protected] or https://groups.google.com/a/2ndquadrant.com/forum/#!forum/bdr-list (same place, different interfaces).

      Lets take the discussion there; I’ll wait for your post and follow up.

      Reply
  14. Kong
    Kong says:
    February 15, 2017 at 12:00 am

    @craig.ringer, would you kindly update your tentative release date for Postgres-BDR 9.6, so that I can plan my project around it?

    Reply
    • craig.ringer
      craig.ringer says:
      March 13, 2017 at 2:26 am

      Announcements will be made when it’s ready. If you depend on it for your project, we’re prioritising customer needs, so feel free to get in touch with [email protected]

      Reply
  15. Uttam
    Uttam says:
    March 21, 2017 at 9:49 am

    It is March 2017 now will be ever available for Postgres 9.6

    Reply
    • craig.ringer
      craig.ringer says:
      March 22, 2017 at 1:51 am

      Yes, actually it’s now available for early access to 2ndQuadrant customers, and customers are testing it for production deployments.

      Get in touch with [email protected] if you’d like to learn more about the customer early access program.

      Public GA will be in the 2nd half of 2017.

      Reply
  16. Tony Hansen
    Tony Hansen says:
    July 10, 2017 at 1:57 am

    It’s July 2017 now. Any updates?

    Reply
    • craig.ringer
      craig.ringer says:
      July 10, 2017 at 2:00 am

      BDR 2.0 is now available to 2ndQuadrant support customers on early access. A public release is not yet scheduled.

      Get in touch with [email protected] if you’d like to learn more about the customer early access program.

      Reply
  17. Stephen
    Stephen says:
    October 10, 2017 at 12:00 pm

    “A public release is not yet scheduled.” – So it’s closed source, and a public release may never be available?

    Reply
    • craig.ringer
      craig.ringer says:
      October 13, 2017 at 7:24 am

      The last statement on BDR 2.0 release plans is at https://blog.2ndquadrant.com/news-and-roadmap-for-bdr-multi-master-postgresql/ . That’s all I have for now.

      Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

Get in touch with us!

Recent Posts

  • Random Data December 3, 2020
  • Webinar: COMMIT Without Fear – The Beauty of CAMO [Follow Up] November 13, 2020
  • Full-text search since PostgreSQL 8.3 November 5, 2020
  • Random numbers November 3, 2020
  • Webinar: Best Practices for Bulk Data Loading in PostgreSQL [Follow Up] November 2, 2020

Featured External Blogs

Tomas Vondra's Blog

Our Bloggers

  • Simon Riggs
  • Alvaro Herrera
  • Andrew Dunstan
  • Craig Ringer
  • Francesco Canovai
  • Gabriele Bartolini
  • Giulio Calacoci
  • Ian Barwick
  • Marco Nenciarini
  • Mark Wong
  • Pavan Deolasee
  • Petr Jelinek
  • Shaun Thomas
  • Tomas Vondra
  • Umair Shahid

PostgreSQL Cloud

2QLovesPG 2UDA 9.6 backup Barman BDR Business Continuity community conference database DBA development devops disaster recovery greenplum Hot Standby JSON JSONB logical replication monitoring OmniDB open source Orange performance PG12 pgbarman pglogical PG Phriday postgres Postgres-BDR postgres-xl PostgreSQL PostgreSQL 9.6 PostgreSQL10 PostgreSQL11 PostgreSQL 11 PostgreSQL 11 New Features postgresql repmgr Recovery replication security sql wal webinar webinars

Support & Services

24/7 Production Support

Developer Support

Remote DBA for PostgreSQL

PostgreSQL Database Monitoring

PostgreSQL Health Check

PostgreSQL Performance Tuning

Database Security Audit

Upgrade PostgreSQL

PostgreSQL Migration Assessment

Migrate from Oracle to PostgreSQL

Products

HA Postgres Clusters

Postgres-BDR®

2ndQPostgres

pglogical

repmgr

Barman

Postgres Cloud Manager

SQL Firewall

Postgres-XL

OmniDB

Postgres Installer

2UDA

Postgres Learning Center

Introducing Postgres

Blog

Webinars

Books

Videos

Training

Case Studies

Events

About Us

About 2ndQuadrant

What does 2ndQuadrant Mean?

News

Careers 

Team Profile

© 2ndQuadrant Ltd. All rights reserved. | Privacy Policy
  • Twitter
  • LinkedIn
  • Facebook
  • Youtube
  • Mail
Those darn Large Objects The Best Barman Ever
Scroll to top
×