News and Roadmap for BDR (Multi-master PostgreSQL)
Postgres-BDR is an open source project from 2ndQuadrant that provides multi-master features for PostgreSQL. We have pursued a joint strategy of providing both working code available now and also submitting the features into core PostgreSQL.
Postgres-BDR 1.0 runs on a variant distro of PG9.4. This is in Production now and receives regular maintenance and security updates. 2ndQuadrant will support this until 9.4 End of Life in December 2019.
One of the greatest achievements to come out of our work on BDR is the logical replication technology. Our engineers spent a considerable amount of energy to contribute the tech to PostgreSQL core and I feel especially proud that this is a headline feature of the upcoming PG10 release.
And Now BDR 2.0 …
BDR 2.0 runs on community PG9.6 as an extension. 2ndQuadrant is making this available now to Support customers only. BDR 2.0 allows our customers to upgrade to PostgreSQL 9.6 while we work on some new BDR features to be made available in version 3.0, which will be released as open source.
BDR 3.0 will be available by Q1 2018, maybe earlier. We will be working with our customers to prioritize feature content and it already looks exciting. We expect BDR3 to be the main future development path and will be supported from its release for 5 years.
We will be submitting further enhancements of performance and robustness to logical replication into PG11. These are all important infrastructure features that will pave the path for inclusion of multi-master replication in PostgreSQL core.
Multi-master functionality from BDR 3.0 will be submitted as core patches to PG12 in 2018, allowing us to work hard towards getting it full function multi-master into core for Production use by Sept 2019 onwards. At that point the project will have taken us 7 years to complete integration.
Hi,
great news!
Thanks for BDR 🙂
— Chris.
wow, Great BDR. thank 2ndQuadrant
As we understand the current BDR 1.x and 2.x are the multiple master with asynchronous replication in the peer-to-peer mesh-up connection. For the multi-master BDR 3.0, will it have the SYNC replication as well as loosely-couple connection (using any communication protocol)? Could you elaborate what’s the exact meaning for the multi-master BDR 3.0? Thanks.
Yes, synchronous_commit will work with a variety of options to provide synchronous replication. It’s simply a function of the transport layer, so it still works with logical replication.
Thanks for the quick reply. I understood that the “synchronous_commit” configuration option is available in the postgres 9.4 and can work with the 9.4 BDR asynchronous replication when it’s turned on. But, it can only provide two nodes synchronous_commit when cross the networking. I observed and confirmed this feature works in the 9.4 BDR cluster. But, it can’t provide N-node (I confirmed it with Craig Ringer).
I wonder in the BDR 3.x release, whether this option is true for N-node synchronous_commit or not? I heard that this feature is very hard to implement.
Yes, it all works now
You mean, in the BDR 3.x release, it will have N-node synchronous_commit, right? For a 4-node BDR cluster, the commit on one node will be completed until all of other three nodes commit. Could you describe the “it all works now”? This is one of very important features we are looking for. Please clarify my understanding here. Thanks.
Richard, it’s necessary to be a bit more specific about what commit completion means.
BDR2 on PostgreSQL 9.6 already supports n-nodes synchronous commit. It’s the same as PostgreSQL core’s feature for physical replication. The transaction commits on the originating node, and confirmation of commit to the client is delayed until all synchronous peer nodes have confirmed they have replayed the commit.
It does not support distributed two phase commit. In this case, the commit is prepared on the originating node, all the peers replay the PREPARE TRANSACTION, and only then do any of them commit. This feature is planned as an option on top of the coming BDR 3. If it is important for your use case, please get in touch about options for funding development so you can influence the roadmap and development priorities.
Interesting: two phased commit (2PC). So I’m guessing the goal is the “A” and “D”, not the “C” and “I” in ACID, correct? Only a distributed database like Postgres XL with a Global Lock Manager (GLM) can satisfy all ACID requirements. And then there is the issue of what kind of 2PC to implement, strict, etc. and how do we deal with the increasing likelihood of deadlocks…
Correct. People also refer to it, somewhat misleadingly as an “AP” system in CAP terms. It’s more helpful to understand it in terms of PACELC (https://en.wikipedia.org/wiki/PACELC_theorem).
HI,
I’m rather new to PostgreSQL and replication, so please forgive a newbee question.
Can the publication/subscription functionality that is being developed for PostgreSQL 10 be used to maintain a set of multi-master databases?
Thanks,
Joe
Short answer is No
Not if there’s any overlap between the subscription sets. You can have two single-master DBs mutually replicating a disjoint set of tables, though.
Tried PostgreSQL 10 logical replication with 2 servers subscribing to each other. A to B and B to A. I did see that that Simon said ‘Short answer is No’. I was still curious. The result is an infinite loop for every insert, update, and delete. BDR must have a method to prevent this infinite loop. Maybe a unique ID per event and then run the event only once per subscriber.
BDR uses replication origins, which are present in PostgreSQL since 9.5.
Logical replication in Pg 10 will apply with a replication origin set. I haven’t checked if it supports filtering out non-local writes using replication origins, as I haven’t yet worked with it, but if it doesn’t then adding such support is not hard.
However, multimaster is so much more than that. Consistent schema changes and schema management, consistent node addition and removal under load, etc.
If you have 2 databases on A and B and you used logical replication at the DB level, on db1 (A) to db1 (B) and on db2 (B) to db2 (A) then i think it will work for you.
Hi,
Is ‘synchronous_comit’ option working for BDR 1.0 ? I mean to ask will the commit on one node of the BDR group will wait until all other nodes made that change visible?
Is postgreSQL 10 coming up with an native Master/Master Clustering solution(shared disk)? Eager to know about these…
Thanks,
PavanTeja Achanta
Yes, synchronous commit works on BDR 1.0 – and on in-core logical replication in PostgreSQL 10 too.
PostgreSQL 10 definitely will not have native MM clustering, shared-disk or replication wise. We continue to work on getting more functionality related to MM replication into PostgreSQL. I am not aware of anyone working on shared-disk based clustering for PostgreSQL.
Thanks Craig for your brief update
I have a doubt I did a 2-node BDR set up. But the database size on the both the nodes is slightly varying around 58MB. But both the node status are running and BDR is active on both nodes.
Is this a serious issue to consider regarding performance aspects? And will BDR have any performance overhead caused to the servers in terms of network/CPU etc ?
Hi,
So, there’s no way that PostgreSQL 9.6 or PostgreSQL 10 having multi-master because BDR 1.x is not supporting any of these?
BDR 3.0 will support PostgreSQL 10. As it says above, BDR 3.0 will be available by Q1 2018, maybe earlier.
Great news. I’m currently using PG10 and waiting for BDR 3.0
It’s time for postgresql community to integrate the N-node multi-master option since all major other SQL and NO SQL DB integrated it already
Any availability of V3? Or is it for support customers only.
Keep an eye on the blog for more news on product availability and releases.