2ndQuadrant is now part of EDB

Bringing together some of the world's top PostgreSQL experts.

2ndQuadrant | PostgreSQL
Mission Critical Databases
  • Contact us
  • EN
    • FR
    • IT
    • ES
    • DE
    • PT
  • Support & Services
  • Products
  • Downloads
    • Installers
      • Postgres Installer
      • 2UDA – Unified Data Analytics
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
  • Postgres Learning Center
    • Webinars
      • Upcoming Webinars
      • Webinar Library
    • Whitepapers
      • Business Case for PostgreSQL Support
      • Security Best Practices for PostgreSQL
    • Blog
    • Training
      • Course Catalogue
    • Case Studies
      • Performance Tuning
        • BenchPrep
        • tastyworks
      • Distributed Clusters
        • ClickUp
        • European Space Agency (ESA)
        • Telefónica del Sur
        • Animal Logic
      • Database Administration
        • Agilis Systems
      • Professional Training
        • Met Office
        • London & Partners
      • Database Upgrades
        • Alfred Wegener Institute (AWI)
      • Database Migration
        • International Game Technology (IGT)
        • Healthcare Software Solutions (HSS)
        • Navionics
    • Books
      • PostgreSQL 11 Administration Cookbook
      • PostgreSQL 10 Administration Cookbook
      • PostgreSQL High Availability Cookbook – 2nd Edition
      • PostgreSQL 9 Administration Cookbook – 3rd Edition
      • PostgreSQL Server Programming Cookbook – 2nd Edition
      • PostgreSQL 9 Cookbook – Chinese Edition
    • Videos
    • Events
    • PostgreSQL
      • PostgreSQL – History
      • Who uses PostgreSQL?
      • PostgreSQL FAQ
      • PostgreSQL vs MySQL
      • The Business Case for PostgreSQL
      • Security Information
      • Documentation
  • About Us
    • About 2ndQuadrant
    • 2ndQuadrant’s Passion for PostgreSQL
    • News
    • Careers
    • Team Profile
  • Blog
  • Menu Menu
You are here: Home1 / Blog2 / Simon's PlanetPostgreSQL3 / Cool Runnings
Simon Riggs

Cool Runnings

October 3, 2016/2 Comments/in Simon's PlanetPostgreSQL /by Simon Riggs

In an earlier post I commented on a post by Evan Klitzke on his reasons for recommending a move from PostgreSQL to MySQL.

The summary was that the technical details were incorrect, apart from two points. This post returns to those points to discuss what we’ve done about them.

1. When one indexed column is updated then currently all indexes need to be maintained. When you have lots of indexes this causes additional write traffic to disk and to the transaction log. The effect was described as “Write Amplification”, though that term is emotionally charged and implies something non-linear; it would be better to say just simply that the use case could be much more fully optimized than the current state.

My colleague Pavan Deolasee has written a patch to optimize this case better, which he calls the Write Amplification Reduction Method or WARM. That’s a great name because in technical terms the optimization is a relaxation of the HOT optimization, so it’s quite literally a cooler name. But most importantly it works very well, measured at 77% better performance for UPDATEs on tables with 4 indexes and well over 100% performance improvement for cases with more indexes. The patch for that has been submitted to PostgreSQL project for review.
WARM TPS Comparison

2. PostgreSQL indexes refer to the heap location (via Tuple Identifier, or TID) directly, whereas MySQL secondary indexes refer to the tuple they index indirectly via the Primary Key. For MySQL, this capability avoids some, but not all of the penalty associated with write amplification, though at the cost of slowing down MySQL index reads.

My colleague Alvaro Herrera has developed a prototype for Indirect Indexes for PostgreSQL, based on enhancements to the btree index type. That seems a straightforward feature that we can add to PostgreSQL, though looks like it will work best with integer Primary Keys, much the same as MySQL. We’re seeing a 46% improvement on updates from the worst case. We have more work to do yet before we submit, but that is a pretty good start.

Yes, Evan highlighted some cases where PostgreSQL could benefit from some tuning. So thanks very much for that, we fully and genuinely appreciate that. Again I would highlight that those are not all cases, nor even the common case for most applications.

What I’d like to point out is that it’s about 8 weeks since Evan’s blog was published and we’ve already got two useful and effective solutions to the areas of poor performance highlighted. And what that shows is that these problems are not architectural limitations in the very heart of Postgres, they are just simple use cases that can be tuned, like many others. We’re hopeful that at least one of the above mentioned solutions is likely to get into the next release, PostgreSQL 10.0.

If Evan had come to us with those concerns earlier then we could have fixed them sooner. PostgreSQL is rapidly moving forwards – we made more than 500 improvements in PostgreSQL 9.6 and will be making even more in the future. It’s evolving quickly because we have lots of happy users experimenting with new and interesting use cases, including many technically savvy people who outline what they want to see (like Evan!).  Check out the list of some of the other major improvements made in 9.6.

Oh, and if you haven’t seen it, you really should see “Cool Runnings”, the film that is.

Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on LinkedIn
2 replies
  1. Mohammad Alhashash
    Mohammad Alhashash says:
    October 3, 2016 at 10:25 pm

    I think indirect indexing would have a much greater benefit for GIN indexes where concurrent updates limit the performance.

    Reply
    • Simon Riggs
      Simon Riggs says:
      October 4, 2016 at 11:53 am

      Agreed. We already have a design to add indirection for GIN indexes, so its on the plan.

      Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

Get in touch with us!

Recent Posts

  • Random Data December 3, 2020
  • Webinar: COMMIT Without Fear – The Beauty of CAMO [Follow Up] November 13, 2020
  • Full-text search since PostgreSQL 8.3 November 5, 2020
  • Random numbers November 3, 2020
  • Webinar: Best Practices for Bulk Data Loading in PostgreSQL [Follow Up] November 2, 2020

Featured External Blogs

Tomas Vondra's Blog

Our Bloggers

  • Simon Riggs
  • Alvaro Herrera
  • Andrew Dunstan
  • Craig Ringer
  • Francesco Canovai
  • Gabriele Bartolini
  • Giulio Calacoci
  • Ian Barwick
  • Marco Nenciarini
  • Mark Wong
  • Pavan Deolasee
  • Petr Jelinek
  • Shaun Thomas
  • Tomas Vondra
  • Umair Shahid

PostgreSQL Cloud

2QLovesPG 2UDA 9.6 backup Barman BDR Business Continuity community conference database DBA development devops disaster recovery greenplum Hot Standby JSON JSONB logical replication monitoring OmniDB open source Orange performance PG12 pgbarman pglogical PG Phriday postgres Postgres-BDR postgres-xl PostgreSQL PostgreSQL 9.6 PostgreSQL10 PostgreSQL11 PostgreSQL 11 PostgreSQL 11 New Features postgresql repmgr Recovery replication security sql wal webinar webinars

Support & Services

24/7 Production Support

Developer Support

Remote DBA for PostgreSQL

PostgreSQL Database Monitoring

PostgreSQL Health Check

PostgreSQL Performance Tuning

Database Security Audit

Upgrade PostgreSQL

PostgreSQL Migration Assessment

Migrate from Oracle to PostgreSQL

Products

HA Postgres Clusters

Postgres-BDR®

2ndQPostgres

pglogical

repmgr

Barman

Postgres Cloud Manager

SQL Firewall

Postgres-XL

OmniDB

Postgres Installer

2UDA

Postgres Learning Center

Introducing Postgres

Blog

Webinars

Books

Videos

Training

Case Studies

Events

About Us

About 2ndQuadrant

What does 2ndQuadrant Mean?

News

Careers 

Team Profile

© 2ndQuadrant Ltd. All rights reserved. | Privacy Policy
  • Twitter
  • LinkedIn
  • Facebook
  • Youtube
  • Mail
2ndQuadrant speaks at PostgreSQL sessions 2UDA – Installing PostgreSQL 9.6 the easy way
Scroll to top
×