A few weeks ago, we added support for PostgreSQL 15. You can choose the version of PostgreSQL you want to use (PostgreSQL v14 or v15) when you create a Neon project.

This post describes how we add support for multiple PostgreSQL versions in our Rust code, which might be helpful for other Rust projects that want to support multiple versions of the same C library simultaneously.

Neon architecture recap

A Neon installation consists of compute nodes and a Neon storage engine.  

Compute nodes are stateless PostgreSQL nodes backed by the Neon storage engine.

From a code perspective, a compute node is a patched PostgreSQL that enables the Neon API to access the storage engine.

Storage consists of several modules, but in this blog post, we will consider it as one piece of software. If you are interested in learning about the individual modules, please refer to the Architecture decisions in Neon blog post.  

Storage accepts Postgres write-ahead-log (WAL), durably stores it, digests it, and turns it into Postgres data pages. Storage is implemented in Rust, and to handle Postgres data and WAL file formats, it needs to use Postgres C code.
To do that, we use the bindgen crate.

Using bindgen to generate Rust code from C headers

We use the crate bindgen to generate Rust code from C headers. It’s a very convenient tool that does all the heavy lifting. It auto-generates Rust code which is a direct translation of C code.

All of our Postgres-specific code is encapsulated in the postgres_ffi module. The examples below are simplified extracts from past and present Neon code with file paths relative to this module.

First, we prepare input for bindgen.

bindgen_deps.h includes all the PostgreSQL headers required to auto-generate Rust structs.

Then we run bindgen to generate Rust code from C headers.

in build.rs, we generate the bindings and store them in the intermediate file bindings.rs.

Finally, in src/lib.rs, we include the generated bindings.

From now on, we can use the generated structs and functions in our code like this:

That’s great, but how do we support multiple PostgreSQL versions?

Using bindgen with multiple versions of Postgres

The PostgreSQL on-disk file format is subject to change in each major PostgreSQL version.

For example, between v14 and v15, the WAL page format did change. So did the XLogPageHeaderData structure and XLOG_PAGE_MAGIC version indicator.

In Neon, we parse WAL pages and extract WAL records from them, so we need to maintain a separate version of the generated code for each version.

Binding with multiple versions of the same codebase is a bit tricky. Let’s review changes to the one-version code step by step:

As before, we need to prepare the input for bindgen.

bindgen_deps.h hasn’t changed in our case. (If header files have changed between versions, we would have to maintain separate versions of bindgen_deps.h for each version.)

Next, we call bindgen for each version.

in build.rs

This produces bindings_v14.rs and bindings_v15.rs files, which we include in lib.rs.

After this step, we have version-specific postgres_ffi submodules, which can be used like this:

and

We avoid exposing the version-specific modules to the rest of the codebase, so

all such code is wrapped into functions that accept the version as a parameter.

in src/lib.rs

Version-independent constants and structures are explicitly re-exported in lib.rs and can be used just like before:

This approach allows us to support multiple versions of Postgres without code duplication and with minimal changes to the existing codebase.

Changes to compute node code

This section is not as complicated as the previous one.

Neon requires a small set of patches to PostgreSQL code. We need to maintain those patches for each supported PostgreSQL version, but the changes are small and isolated, so they are not challenging to rebase.

This also means that we can easily support new versions of Postgres extensions. As soon as they are compatible with the latest Postgres version, we can add support for them in Neon.

What’s next?

We will add support for more PostgreSQL extensions in future Neon releases.

What else do you want to see in Neon? Join us and share your ideas at https://community.neon.tech.