It’s been busy few months as we work towards merging Postgres-XL with the latest and greatest release of PostgreSQL. Postgres-XL is an open source fork of PostgreSQL that provides a scalable platform for OLTP and Business Intelligence. The current release of Postgres-XL is based on PostgreSQL 9.2, so it lacks all the improvements made to PostgreSQL over the last three years.
2ndQuadrant and other companies are working on bringing distributed scalability into PostgreSQL core as well as building tools and extensions outside the core. As part of that, Postgres-XL has a number of features that we’d like to bring back into core PostgreSQL, so 2ndQuadrant has picked up the task of updating the Postgres-XL code base to the latest PostgreSQL release as the first step. After more than 3 months work, PostgreSQL 9.5 is still in alpha stage, so we wanted to give a progress report on how the work is proceeding. I also need to say the magic words: This ongoing work on Postgres-XL is part of the AXLE project, funded by the European Union under grant agreement 318633.
Preparation for the Merge
Since PostgreSQL and Postgres-XL both use GIT as source control system, it makes the merge process much simpler since GIT provides many tools to assist the process. But as soon as we tried the merge, we faced the first hurdle.
We realised that the current Postgres-XL repository is based on an older minor 9.2 release of PostgreSQL. That means there were commits and changes in the Postgres-XL master branch which either never made to PostgreSQL’s master branch or had different commit ids. So merge with PostgreSQL master branch threw a lot more conflicts than what we would have other expected. So the first task we must accomplish was to rebase the Postgres-XL 9.2 repository on a later commit point. This obviously required careful treading, making sure that nothing breaks during the process. Once we had the basic rebase done, we also merged all Postgres-XL bug fixes and enhancements, created a Postgres-XL 9.2 stable branch and merged the 9.2 branch with the latest available PostgreSQL 9.2 minor release.
Challenges Faced During the Merge
The actual merge with PostgreSQL master branch was not an easy task either. Note that we were jumping 3 major releases of PostgreSQL, which almost accounted for 3 years of development work. Thankfully, git-mergetool comes very handy for such large scale merges. You can use your favourite editor (vimdiff in our case) to nicely see the merge conflicts and resolve them. While some of the conflicts are straightforward and require minor adjustments, many require careful reading and understanding of the code. While it’s not trivial to support all new features, we tried to preserve as much as possible and we have been quite successful.
The other major challenge was merging documentation changes. Since Postgres-XL project had created a copy of the existing SGML documentation, the GIT merge with the master branch did not yield any updates to the copy. This required manual merge. To ensure that this is not required again in future merges, we now make the documentation changes in-place.
There are many things that needs to be completed before we can release Postgres-XL 9.5 to general public:
- Enhance regression test coverage for Postgres-XL
- Fix bugs and add support for new features
- Do systematic performance tests and tuning
- Create Postgres-XL 9.5 branch and merge with latest PostgreSQL 9.5 stable branch
We aren’t yet ready for review of Postgres-XL, but we expect Postgres-XL 9.5 Beta to be ready round about the same time as PostgreSQL 9.5 Beta.
Look for my next blog post in about a month’s time for the next update.