Postgres-XL is a massively parallel database built on top of – and very closely compatible with – PostgreSQL. It is different because it supports both Business Intelligence workloads and high-volume transactional write and read workloads all on the same platform.
Postgres-XL is designed to be horizontally scalable and flexible enough to handle various workloads including:
- OLTP write-intensive workloads
- Business Intelligence requiring OLAP with MPP parallelism
- Operational data store
- Key-value store including JSON
- GIS Geospatial
- Mixed-workload environments
What Our Clients Say
“Postgres-XL has been key to the success of the Gaia variability studies group in Geneva. Flexibility enabled by Postgres for the Extra Large scientific challenge, solid architecture, and rich environment stood out when we researched for credible long-term big data platforms. Capability of having millisecond access to information about specific light sources or processing results, with the power of horizontal scalability without sacrificing ACID properties, has been a clear win for our project.”
Krzysztof Nienartowicz, Manager ESA Gaia – Data Processing Center, European Space Agency
Postgres-XL 10R1.1 Released – February 18, 2019
You can download Postgres-XL 10R1.1 here
Please send an email to [email protected] if you want to contribute.
You can contribute by fixing regressions and checking if new features of PostgreSQL 10 work with XL 10.
Postgres-XL 9.5 R1.6 Released – August 24, 2017
You can download Postgres-XL here.
Please report all issues to [email protected]
Our internal testing for BI workloads reveals a dramatic boost in performance compared to native PostgreSQL. Results on a 16 node cluster show linear scalability for most queries.
Some of the most prominent features of Postgres-XL are listed below:
- Scalable – partition or replicate tables across multiple nodes
- Fully ACID compliant including MVCC
- Open Source
- Cluster-wide consistency ensured by Global Transaction Monitor
- Based on PostgreSQL 9.5, fully utilizing new BI features
- Compatible with all PostgreSQL drivers, including JDBC, ODBC, OLE DB, Python, Ruby, perl DBI, Tcl, and Erlang.
- CREATE TABLE syntax compatibility with Greenplum and Redshift
In order to perform as well as XL does, a number of core enhancements were made and are described below.
Massively Parallel Processing (MPP)
- Multi-node distributed Parallel Query – Allows SQL queries to be executed up to N times faster on N datanodes, spreading both CPU and I/O utilization evenly across a cluster.
- Three stage Aggregation – Parallelizes aggregate queries to minimize distributed data logistics.
- Dynamic Redistribution of data for complex SQL queries – Automatically re-arrange data to allow complex SQL queries, not just simple Star Schema queries or Map/Reduce-style tasks.
- Cooperative Scans – Queries work together to avoid repeated re-scanning of same data.
Distributed Transaction Processing (OLTP)
- Distributed Global Consistency – Database appears to be a single consistent database to users.
- Write Scalability, as well as Read Scalability – Scale OLTP writes across the nodes.
Extensibility & Scalability
- Linear scalability – Performance increases linearly for many typical operations for both OLTP and Parallel Query.
- Extensible – Cluster can be extended to increase processing power.
- Very Large Database Support – Cluster easily supports 10s of TBs.
- High Availability – Physically replicated data nodes to ensure High Availability.
- Disaster Recovery – Disaster Recovery via Streaming Replication.
Platform Testing & Support
- XL is tested on Linux and Mac OS X.
Postgres-XL has been evolving over many years, originating with the GridSQL project and PostgresXC, later combined as TransLattice Storm (StormDB). TransLattice open sourced the project, resulting in Postgres-XL 9.2. More recently, the EU-funded Big Data project AXLE funded the main work to bring the code up to date as Postgres-XL 9.5, allowing XL to take advantage of the rapidly increasing BI features plugged into the core of PostgreSQL.
Support for Postgres-XL
Postgres-XL is open source and fully supported for 2ndQuadrant Support customers. We also offer consulting services to help organisations with application development, cluster design, deployment, feature development and performance analysis.
Need more help?
Want to know more? Need some help to set up XL? As active community developers for the project, we’re probably the best people in the world to help you get up and running with Postgres-XL. We have consultants available to provide assistance, plus our unique 24/7 Production Support subscription service also covers Postgres-XL.
Postgres-XL Knowledge Base
An in-depth and practical course covering this horizontally scalable extension to PostgreSQL. This course will help delegates understand and implement massively parallel databases using Postgres-XL. This course will also discuss the problems with parallelism.
Postgres-XL and global MVCC/0 Comments/in 2ndQuadrant, Koichi’s PlanetPostgreSQL, Postgres-XL /by Koichi Suzuki
Back to the PG I’m very excited to become a 2ndQuadrant member. I was involved in PostgreSQL activities in NTT group (Japanese leading ICT company, see here and here), including log shipping replication and PostgreSQL scale out solution as PostgresXC and PostgresXL. At NTT I had several chances to work very closely with 2ndQuadrant. After three years involvement in deep […]
pgxc_ctl: Teaching Postgres-XL in New York City/0 Comments/in 2ndQuadrant, Tom's PlanetPostgreSQL /by Tom Kincaid
At the 2ndQuadrant PostgreSQL conference in New York City this year, I had the pleasure of delivering a training on Postgres-XL together with Andrew Dunstan. We capped the attendees at 8 because we wanted to have a lot of lab time with the students and spend time helping them get set up with Postgres-XL. The class […]
Postgres-XL with OmniDB/1 Comment/in OmniDB, William's PlanetPostgreSQL /by William Ivanski
1. Introduction Postgres-XL (or just XL, for short) is an open source project from 2ndQuadrant. It is a massively parallel database built on top of PostgreSQL, and it is designed to be horizontally scalable and flexible enough to handle various workloads. Here we will show how to build a test environment to play with XL […]