2ndQuadrant is now part of EDB

Bringing together some of the world's top PostgreSQL experts.

2ndQuadrant | PostgreSQL
Mission Critical Databases
  • Contact us
  • EN
    • FR
    • IT
    • ES
    • DE
    • PT
  • Support & Services
  • Products
  • Downloads
    • Installers
      • Postgres Installer
      • 2UDA – Unified Data Analytics
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
  • Postgres Learning Center
    • Webinars
      • Upcoming Webinars
      • Webinar Library
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Blog
    • Training
      • Course Catalogue
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
    • Books
      • PostgreSQL 11 Administration Cookbook
      • PostgreSQL 10 Administration Cookbook
      • PostgreSQL High Availability Cookbook – 2nd Edition
      • PostgreSQL 9 Administration Cookbook – 3rd Edition
      • PostgreSQL Server Programming Cookbook – 2nd Edition
      • PostgreSQL 9 Cookbook – Chinese Edition
    • Videos
    • Events
    • PostgreSQL
      • PostgreSQL – History
      • Who uses PostgreSQL?
      • PostgreSQL FAQ
      • PostgreSQL vs MySQL
      • The Business Case for PostgreSQL
      • Security Information
      • Documentation
  • About Us
    • About 2ndQuadrant
    • 2ndQuadrant’s Passion for PostgreSQL
    • News
    • Careers
    • Team Profile
  • Blog
  • Menu Menu
You are here: Home1 / Blog2 / PostgreSQL3 / Future of Postgres-XL
Tomas Vondra

Future of Postgres-XL

May 26, 2017/4 Comments/in PostgreSQL, Tomas' PlanetPostgreSQL /by Tomas Vondra

You probably know that Postgres-XL is a distributed database based on PostgreSQL. A few days ago we pushed the XL 9.6 code into the public git repository. Additional details about the new stuff available in Postgres-XL 9.6 are available here.

The topic of this blog post is quite different, though. I’d like to discuss some changes to the project management and development practices, and why (and how) we plan to tweak it.

At first sight, the XL community may not seem particularly active, particularly if you only look at code the number of contributors or traffic on mailing lists. We know this is not entirely accurate, as we get a lot of off-list interest from customer and developers building exciting stuff on Postgres-XL. But it also shows that perhaps we could improve this side of the project, to make it easier to contribute code or provide feedback.

We also know there are quite a few Postgres-XL forks. We don’t expect people to stop working on them and move back to XL; some forks address use cases that are not the primary aim of XL. But perhaps those forks might benefit from upstreaming some of the generic improvements (e.g. bugfixes or some of the boring infrastructure bits), lowering the maintenance burden and reducing merge conflicts.

Obviously, this is a long term goal and there is not one particular thing that would make it happen. So feel free to propose other changes, or point out additional annoyances that keep you from contributing to XL.

Growing the community

One of the goals of these changes is to growing the XL community and making it more active. That includes not only getting more messages on the mailing lists, more downloads, bug reports (or whatever is metric you pick). I also means sharing control of the project with a wider community, including for example granting commit rights to experienced contributors, etc.

It’s not a question of “if” but “when.” We don’t have an exact schedule or deadlines for adding committers, but my estimate is that it’ll happen sooner rather than later.

Keep XL close to PostgreSQL

One of the reasons why we don’t want to adopt a more complete (and complex) development platform is that we want to keep Postgres-XL as close to PostgreSQL as possible, both in terms of code and development practices. And PostgreSQL uses a very simple process, based on sending patches to a mailing list. That is both simple and also serves as a simple “audit trail.”

So we do not plan to move the development to github or gitlab, but there’s nothing preventing you from embracing those technologies while working on XL, as long as the final patches get sent to the mailing list. We’re using github internally, for example.

Move off Sourceforge

Long time ago, sourceforge was a great place to host open source projects. But nowadays the site seems pretty much in maintenance-only mode, faced various controversies related to bundling adware to downloads, etc. It’s time to move on.

Luckily, we don’t need that much – a project website, a git repository and a few mailing lists and. The first two items – website and the git repository are already hosted off sourceforge.

So we only need to do something about the mailing lists, which we can easily host on http://www.postgres-xl.org (and we can even import the current archives, so that we don’t lose the history).

The plan is to do this change sometime next week. If you’re subscribed to any of the mailing lists, you’ll be automatically subscribed to the new mailing lists, and you’ll receive message with all the details. The main change will be a change of the domain, from @lists.sourceforge.net to @lists.postgres-xl.org.

Tags: postgres-xl
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
4 replies
  1. Adam
    Adam says:
    May 26, 2017 at 10:35 pm

    It’s awesome to see the work you all put into XL. We appreciate that you are working on it, and keeping up to date with mainline for the most part. It’s easy to fork, and forget ever updating. It’s not so easy to fork and keep up to date.

    Reply
  2. Santiago
    Santiago says:
    May 30, 2017 at 9:21 am

    I am happy someone keep on working in XL.

    Reply
  3. mha
    mha says:
    May 30, 2017 at 10:21 am

    This all sounds like very good directions! A small tip, though. I’m not sure if I am misreading this, as I’m not following XL closely, so if I am then ignore the comment 🙂 But this:

    A few days ago we pushed the XL 9.6 code into the public git repository

    Is a bad indicator. That sounds like the development did not happen in public, but instead dropped as a batch of commits once already done. If so, I suggest that’s the first thing you should change. However, if it just means that “untli then it was just a patch living on the mailinglists” then it’s obviously fine — the part that’s potentially bad is if there is an internal, non-public, git-repo where development was done. That is a great way for making sure that outside contributors don’t feel part of the effort.

    The efforts are of course much appreciated and very nice to have regardless of that, but a structure like that would make the even more important effort of growing the community so much harder.

    Anyway, I hope I’m reading that wrong, but given that it is what showed up in my RSS reader as the intro to the post, it had me worried.

    That said, now off to try to find some time to test the new functionality 🙂

    Reply
    • Tomas Vondra
      Tomas Vondra says:
      May 30, 2017 at 12:13 pm

      You are reading it both right and wrong. Let me explain.

      You’re right that we pushed most of the 9.6 merge at once, and that it lived in a private repository until that point. I’m not particularly happy about that either, and I agree it’s something we need to change. And we’re making steps in that direction.

      However, let me explain why it was done that way. It’s not because we want to keep the code private and gain some dubious advantage by doing that, but more about the challenges of maintaining large fork. Immediately after the “git merge” from upstream the repository is utterly broken, because of merge conflicts. Initially it does not even compile, then after a lot of whacking it compiles but initdb promptly crashes with ten different segfaults, then initdb passes but everything else segfaults, etc.

      I’d argue that’s not something you want to push to the master branch, because it’ll immediately break everyone else’s patches and fixing those issues is mostly serial effort. Admittedly, we could have pushed the code sooner, once it gets into mostly-working state.

      We’re looking for ways to improve this for the next large merge, but I’m not sure we can eliminate that entirely – it’s simply a part of XL being a fork. If you have a better idea how to do this (say, some smart git workflow), we’d love to hear it.

      Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

Get in touch with us!

Recent Posts

  • Random Data December 3, 2020
  • Webinar: COMMIT Without Fear – The Beauty of CAMO [Follow Up] November 13, 2020
  • Full-text search since PostgreSQL 8.3 November 5, 2020
  • Random numbers November 3, 2020
  • Webinar: Best Practices for Bulk Data Loading in PostgreSQL [Follow Up] November 2, 2020

Featured External Blogs

Tomas Vondra's Blog

Our Bloggers

  • Simon Riggs
  • Alvaro Herrera
  • Andrew Dunstan
  • Craig Ringer
  • Francesco Canovai
  • Gabriele Bartolini
  • Giulio Calacoci
  • Ian Barwick
  • Marco Nenciarini
  • Mark Wong
  • Pavan Deolasee
  • Petr Jelinek
  • Shaun Thomas
  • Tomas Vondra
  • Umair Shahid

PostgreSQL Cloud

2QLovesPG 2UDA 9.6 backup Barman BDR Business Continuity community conference database DBA development devops disaster recovery greenplum Hot Standby JSON JSONB logical replication monitoring OmniDB open source Orange performance PG12 pgbarman pglogical PG Phriday postgres Postgres-BDR postgres-xl PostgreSQL PostgreSQL 9.6 PostgreSQL10 PostgreSQL11 PostgreSQL 11 PostgreSQL 11 New Features postgresql repmgr Recovery replication security sql wal webinar webinars

Support & Services

24/7 Production Support

Developer Support

Remote DBA for PostgreSQL

PostgreSQL Database Monitoring

PostgreSQL Health Check

PostgreSQL Performance Tuning

Database Security Audit

Upgrade PostgreSQL

PostgreSQL Migration Assessment

Migrate from Oracle to PostgreSQL

Products

HA Postgres Clusters

Postgres-BDR®

2ndQPostgres

pglogical

repmgr

Barman

Postgres Cloud Manager

SQL Firewall

Postgres-XL

OmniDB

Postgres Installer

2UDA

Postgres Learning Center

Introducing Postgres

Blog

Webinars

Books

Videos

Training

Case Studies

Events

About Us

About 2ndQuadrant

What does 2ndQuadrant Mean?

News

Careers 

Team Profile

© 2ndQuadrant Ltd. All rights reserved. | Privacy Policy
  • Twitter
  • LinkedIn
  • Facebook
  • Youtube
  • Mail
Kanban & devops culture at 2ndQuadrant – Part 1 Visualizing sar data
Scroll to top
×