Announcing Release 5 of the PostgreSQL Buildfarm Client

Release 5 of the PostgreSQL Buildfarm Client has been released and can be downloaded from http://buildfarm.postgresql.org/downloads/releases/build-farm-5.tgz

In a similar move to PostgreSQL version numbering, with this release we move to a one part numbering system.

In addition to a number of bug fixes and very minor changes, this release has the following features / changes:

• Cross-version pg_upgrade module is no longer experimental – see below
• TAP tests now run the “check” target, but in most cases redundant installs are avoided
• Improved and expanded TAP test running on MSVC animals – these now run the same tests as other animals
• Automatic garbage collection of git repositories, once a week by default. This should improve the speed of git operations, especially on older git releases.
• Improve logging in SEpgsql module
• Revert 4.19 setting of TZ by default. If you want it set then do so in the config file. Doing so can speed up initdb. Setting it by default caused too many anomalies.
• The environment setting BF_LOG_TIME (settable in the config file) causes command output to be prefixed with a timestamp on non-MSVC platforms if /usr/bin/ts is present.

The cross-version upgrade module (TestUpgradeXVersion) has long been an experimental module. It has been refactored and set to report its results properly to the server, so it’s now ready for wider use. Its use requires quite a lot of disk space, so it’s not configured by default. You need to have at least 5Gb of available space in order to use it. Normally in a buildfarm run all the build products are removed. This module saves them in a special area so that they are available for testing against different branches. Each branch is tested against itself and any earlier branches that the module has saved. So currently for the HEAD (i.e. master) branch it can test upgrades from 9.2, 9.3, 9.4, 9.5 and 9.6 as well as itself. The tests cover all the databases in the install, such as each contrib database, not just the standard regression database.

The module has a heuristic test for success. Once the upgrade has run successfully, we compare a pre-upgrade dump with a post-upgrade dump. If it’s a same branch upgrade we expect 0 lines of difference, but for a cross-branch upgrade we allow up to 2000 lines of difference. Clearly this isn’t a perfect test, but experience with the module over an extended period has shown that pretty much all problems either result in an upgrade failure or a diff that is larger than 2000. See http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2017-06-13%2012%3A55%3A45&stg=xversion-upgrade-REL9_2_STABLE-HEAD for an example of output in upgrading REL9_2_STABLE to HEAD.

The tests take about 60 to 90 seconds per test on my test machine – so in a full buildfarm run across all live branches (21 upgrade tests in all) that adds about 30 minutes.