At AWS re:Invent last week, Amazon announced hardware acceleration for the Redshift data warehousing service. Amazon claims Redshift AQUA (Advanced Query Accelerator) will outperform other cloud data warehouses by 10x, enabled by a new FPGA-powered caching layer. With this news, hardware acceleration for databases will likely be a hot topic in 2020.
Hardware acceleration for databases has been available for decades–mainly to enterprises with big IT budgets, in the form of costly data warehouse appliances like Netezza, Teradata, and others. But in recent years, hardware accelerators (e.g., FPGAs) have increased in affordability, ease of use, and accessibility–nearly all cloud platforms now offer FPGA-equipped instances. As a result, mainstream database users can easily follow Amazon’s example.
Hardware acceleration options for Postgres?
If you are running Postgres and you care about performance, there are a number of hardware acceleration options to help with different use cases.
Before you lose too much more sleep tuning, or before taking more drastic measures like migrating your database to bigger servers or to a distributed architecture, consider hardware acceleration. In many cases, it’s an easier and more economic way of achieving the performance and scalability you require.
Here are three hardware acceleration options that we believe will have the most immediate impact on Postgres users in 2020:
FPGA – the increasingly ubiquitous database accelerator
FPGAs are integrated circuits that can be reprogrammed dynamically to accelerate a specific workload such as SQL execution and data compression. The Swarm64 DA FPGA accelerator extends Postgres with hundreds of SQL reader and writer processes that work in parallel on the FPGA. It’s akin to adding 100 new cores to boost parallel processing on your server.
FPGA acceleration has been shown to speed up Postgres query performance by over 50x and data ingestion by 35x in TPC-H benchmarks. FPGAs are available on nearly all cloud platforms, including Amazon, Alibaba, and Azure, and you don’t need special skills to use them. This video shows how quickly Postgres can be spun up on an FPGA-equipped Amazon EC2 F1 instance.
When you’re planning your next Postgres deployment, consider an FPGA-equipped server on-premises or in the cloud (try FPGA-accelerated Postgres for free).
Intel Optane DC – the in-memory OLTP accelerator
Thinking of running your Postgres database in memory to speed up transaction processing? If so, check out Intel® Optane™ DC persistent memory. It revolutionizes database processing by providing larger, more affordable memory that is also persistent. Unlike traditional in-memory (DRAM-based) computing architectures, where memory is small, expensive, and volatile, Intel Optane DC persistent memory makes it possible to run larger Postgres databases (terabytes) in memory for higher performance. We developed a Postgres demo for Strata NYC using the Intel Optane DC, and it easily handled 200K inserts of financial market data per second.
Samsung SmartSSD – elegant computational storage with embedded Xilinx FPGA
Samsung Electronics recently introduced the Samsung SmartSSD™. The SmartSSD is a computational storage device that combines fast, solid-state (SSD) storage with a Xilinx-based FPGA. A peer-to-peer connection on the device transfers data from the SSD to the FPGA without consuming CPU resources and allowing data to be processed directly on the SmartSSD. We tested the SmartSSD in the Swarm64 labs with Swarm64 DA running on the FPGA, and saw TPC-H queries on Postgres speed up by as much as 40x, and join queries speed up 10x-15x.
If you’re thinking about changing your drives to SSD/NVMe to speed up Postgres performance, we recommend you consider the SmartSSD. You’ll get the acceleration of an SSD/NVMe, plus the parallel processing of FPGA with low-latency access to storage to speed up scanning, filtering, and decompression. It’s an easy way to increase Postgres performance by at least one order of magnitude.
Give them a try…
These are some of our favorite hardware acceleration options for Postgres. We’ll write up in-depth articles describing our experiences with each of these, We’ll also be comparing FPGA vs. GPU acceleration options for Postgres (e.g., pgstrom), so stay tuned!
If you’d like to experience FPGA-accelerated Postgres on an Amazon F1 instance, request a free Swarm64 trial or install it from the AWS Marketplace. And as always, you can ping us if you’d like to discuss specific use cases or performance issues; we’re happy to help!