Fast and Safe Performance Optimization Using Production Grounded Synthetic Data
September 30 - October 2
Validating database changes is a labor-intensive and risky process. We know that production has specific performance behavior due to characteristics such as cross-column-correlations and data skew. AI can accelerate the process of benchmarking changes, but developers need a safe way to test migrations, index changes, and DML queries. And if we stray too far from production, the benchmark might give false confidence that a change is safe and performant.
In this talk we'll look at how to improve the accuracy of using synthetically generated data for benchmarking purposes to validate risky changes before deployment. Using lessons learned building the validation harness used by Datadog’s Bits Database Optimization system, we’ll show how extracting metadata about production and using it to guide the data generation makes better synthetic data. You’ll see how various pieces of data can be used to improve this accuracy, from column statistics, to extended statistics, to additional purpose-built queries.
