Firefly2024-09-302024-10-023https://postgresql.us/events/pgconfnyc2024/schedule/08:3001:00Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1828/Session09:3000:20The Forumhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1829/Session09:4500:25The Forumhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1860/Session10:1000:20The Forum2024 was the biggest year yet for Postgres at Microsoft. Our team of Postgres open source developers & committers made many contributions to Postgres 17 while on the Azure side, we've continued to add more enterprise and AI features at a rapid pace. This keynote will share 3 things with you: (1) an overview of our recent open-source contributions to Postgres 17 and its ecosystem, (2) a rundown of new features that are now available in our managed service; and (3) a peek—with demos—into new capabilities we'll soon deliver into Azure Database for PostgreSQL.https://postgresql.us/events/pgconfnyc2024/schedule/session/1850/SponsorCharles Feddersen10:3000:30Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1830/Session11:0000:50The ForumPostgreSQL 16 is either the latest or the second-to-latest version of PostgreSQL, depending on exactly when version 17 is released as it's scheduled to happen right around the conference time. This talk will take a quick look at many of the new features in this version.https://postgresql.us/events/pgconfnyc2024/schedule/session/1757/SessionMagnus Hagander11:0000:50Club ParlorPartitioning has come a long way in Postgres since the Postgres 10 days, as has sharding via the Citus extension. If you need to scale your Postgres, your friends may recommend you look into partitioning and/or sharding. But what’s the difference between these two approaches? This beginner’s guide is for those who want to start with the basics and advance from there. You’ll learn what partitioning and sharding are in Postgres; how they’re different; as well as why (and when) these capabilities can help improve query performance. You’ll learn why some people call partitioning invaluable and others call sharding a lifesaver. But there are gotchas and they’re not for every use case. As David Rowley says, “Understanding why partitioning helps is key to making the decision about whether to partition, what to partition on, and the type and number of partitions.” Similarly, understanding why sharding helps is key to deciding whether sharding is right for you. The concept of data locality will be touched on as well. And for those who prefer to learn by example and not just theory, yes, there will be examples.https://postgresql.us/events/pgconfnyc2024/schedule/session/1773/SessionClaire Giordano11:0000:50The HubIn this session we'll unravel the intricacies of PostgreSQL's internal data organisation, shedding light on the Free Space Map (FSM) mechanism. Delving into the heart of PostgreSQL's data architecture, we'll explore ways to strategically reorganise the data model capitalising on data alignment within blocks. The goal? To demonstrate techniques that not only optimise disk space usage but also enhance memory efficiency, potentially leading to significant cost savings and performance improvements. Discover the art of aligning data inside blocks, a crucial facet often overlooked in PostgreSQL optimisation. By strategically aligning data, we can wield the power to reduce the footprint of stored information on both disk and memory. As we navigate through practical examples and insightful strategies, you'll gain a new understanding of how these optimisations directly impact your budgetary considerations and system performance.https://postgresql.us/events/pgconfnyc2024/schedule/session/1581/SessionCharly Batista11:0000:50Library (Sponsor)Companies get significant business value out of the sensitive data they’ve collected. However, keeping it secure and staying compliant with data privacy and security regulations is increasingly difficult. Postgres doesn't offer transparent data encryption (TDE) but even if it did, TDE doesn't solve for most attack vectors. The solution is cryptographic protection that follows regulated data while allowing analytical and operational computations on the data. This is sometimes called "privacy enhanced computation" In this talk, learn how the combination of a SQL proxy and a PL/Rust Postgres extension can enable operations on encrypted data without the use of specialty solutions with hardware enclaves, such as Intel SGX or the performance impacts of homomorphic or secure multiparty encryption. The development of PostgreSQLK trusted language extensions brings this capability to managed databases in the cloud quickly and without any code changeshttps://postgresql.us/events/pgconfnyc2024/schedule/session/1761/Sponsor12:0000:50The ForumWe'll explore the intricate world of PostgreSQL parameter tuning, where the database management system reveals a multitude of configurable parameters that govern its runtime behavior. As the system evolves, the increasing number, the non-linear relationship and the complexity of these parameters highlight the importance of their correct configuration in optimizing application performance and cloud resource utilization. This talk delves into various tuning approaches. We'll start with the traditional, meticulous method of manual tuning, relying on expert analysis of metrics like query response times and resource usage. We'll then explore heuristic-based tools like PGTune and PostgreSQL Configurator, offering one-size-fits-all solutions. Finally, we'll set our sights on the future: auto-tuning with machine learning. We'll showcase how these more sophisticated methods can be customized to a specific workload and machine resources. We'll highlight the challenges and the advantages in augmenting PostgreSQL with machine learning. We will share valuable lessons learned while developing DBtune, a machine-learning-powered optimization service designed to unlock the full potential of your PostgreSQL instance.https://postgresql.us/events/pgconfnyc2024/schedule/session/1802/SessionCosta Alexoglou12:0000:50Club Parlor... or, Data Modeling in the Age of Social Networks. We're now building systems that collect data on every aspect of someone's life: Not just their name and address, but their gender(s), family, relationships, movements, preferences... And all of the assumptions we make about people in these models are probably wrong. What's a "name"? What's a "family"? What's a "relationship"? How do we model the vast complex reality of human life in a data system... and do so ethically, legally, and with an eye to the realities of how people live? We'll talk about the range of issues this brings up, from basic low-level schema issues through user experience concerns, as well as touching on ethical, legal, and regulatory compliance issues in collecting this sort of data.https://postgresql.us/events/pgconfnyc2024/schedule/session/1605/SessionChristophe Pettus12:0000:50The HubNowadays, increasingly many PostgreSQL extensions are developed in Rust. This is partly because Rust provides memory safety and performant high-level abstractions out of the box, and partly because Cargo, the Rust package manager, and PGRX, the PostgreSQL extension development framework in the language, facilitate many extension development lifecycle tasks. In this hands-on session, you will walk through the extension development lifecycle journey on a sample project that adds analytics capabilities to PostgreSQL. During the journey, you will learn many important topics ranging from the high-level project structuring, dependency management, compliance enforcement, and telemetry collection capabilities to the low-level foreign function interface (FFI), single instruction, multiple data (SIMD), and asynchronous IO- and CPU-bound operation capabilities. Don’t miss this opportunity to learn how the Rust ecosystem can be leveraged to build high-performance PostgreSQL extensions!https://postgresql.us/events/pgconfnyc2024/schedule/session/1826/SessionArda Aytekin12:0000:50Library (Sponsor)Amazon Aurora is a fully managed relational database designed for high performance and availability at global scale with full MySQL and PostgreSQL compatibility. Aurora provides managed high availability (HA) and disaster recovery (DR) capabilities in and across AWS Regions. In this session, explore Aurora HA and DR capabilities and discover design patterns that enable the development of globally resilient applications. Learn how to establish single Region and cross-Region HA and DR using Aurora features, including multi-AZ deployments and Amazon Aurora Global Database.https://postgresql.us/events/pgconfnyc2024/schedule/session/1732/SponsorShayon Sanyal12:5001:10Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1831/Session14:0000:50The ForumI’m excited to share the work we’ve done to scale our Postgres databases at Notion, keeping up with our incredible growth over the past few years. In this talk, I’ll dive into several key areas: - Journey to Sharding: Learn how we migrated from a single database to a sharded architecture and the challenges we faced along the way. - Major Version Upgrades: Discuss how we perform major version upgrades without any downtime for our users. - Best Practices for Large Scale Deployments: Pick up practical tips and tricks we’ve implemented to keep our sanity while providing a reliable datastore service for our growing engineering team. I hope some of the challenges we faced and solutions we implemented will be useful to other companies dealing with similar issues. I look forward to sharing my experiences and learning from the community!https://postgresql.us/events/pgconfnyc2024/schedule/session/1698/SessionArka Ganguli14:0000:50Club ParlorIn PostgreSQL, Vacuuming is a routine maintenance procedure that improves database speed and frees up space. It leads to clearing out old or deleted rows from tables and indexes, as well as refreshing query planner statistics. By deleting records that are no longer required, PostgreSQL's vacuum operation maintains the database's speed and space efficiency. In PostgreSQL, autovacuum is enabled by default and can be customized using a number of settings in the postgresql.conf file. There are several vacuum-related PostgreSQL options that may be customized to manage how the vacuum process functions. In this session, Suyog will cover: •  What is the Vacuum and Autovacuum process. •  How to tune Autovacuum parameters to optimize database performance. •  When and how to use Vacuum full, reindex, and pg_repack •  Best practices and lessons learned from real-world case studies.https://postgresql.us/events/pgconfnyc2024/schedule/session/1677/SessionSuyog Sampat Pagare14:0000:50The HubWhen your system is active, data changes very quickly. You have made a backup, and the next hour, it won't match with the database anymore. You migrated the data from your old database to a new one, but while you are finalizing all processes, data stops being synchronized. Of course, you can compare the data in tables and find the differences. It's not difficult, isn't it? People don't usually consider comparison as a complicated operation. In real life, we do this all the time. That's why we take the data comparison task lightly. But it turns out that it's not as easy as we expect. Let's review the main difficulties together: - What are the criteria to compare the data? - How to compare data between different databases? - What if you have billions of rows? - How to store the differences? - What is the best way to show the comparison result? We have already gone through all these questions when developing the Data Comparison feature in DBeaver. Now, we are ready to share some tips and tricks with you.https://postgresql.us/events/pgconfnyc2024/schedule/session/1657/SessionTatiana Krupenya14:0000:50Library (Sponsor)In today’s fast-paced, globally connected world, applications must be always on, always available and always highly responsive. pgEdge Distributed PostgreSQL delivers the ultra-high availability, low latency and disaster tolerance that makes this possible. This presentation will explore the core use cases that benefit from deploying distributed PostgreSQL with asynchronous logical replication in multi-master and multi-region configurations, addressing challenges like slow response times, maintenance downtime, and disaster recovery. Additionally, we will share the advantages of distributed PostgreSQL in a managed cloud, including improved security, simplified deployment and management, seamless scaling, and automated backups. Attendees will learn how to more easily leverage multi-master capabilities, latency-based routing and more in a distributed cluster to ensure optimal performance and maximal reliability for users worldwide.https://postgresql.us/events/pgconfnyc2024/schedule/session/1853/Sponsor15:0000:50The ForumHigh-traffic PostgreSQL databases face significant challenges with connection management, including high memory usage, substantial CPU resource consumption, and scalability constraints. Traditional external connection poolers like PgBouncer introduce latency and complexity. In-database connection pooling addresses these pain points by reusing existing connections, reducing resource consumption, and improving load management. This open source extension leverages PostgreSQL's extensible architecture, providing seamless integration and enhanced performance. Attendees will learn about the architecture, benefits, and implementation of this solution, offering a scalable and efficient approach to handle large numbers of concurrent connections directly within Postgres.https://postgresql.us/events/pgconfnyc2024/schedule/session/1776/SessionJonah H. Harris15:0000:50Club ParlorMigrating from Oracle to PostgreSQL is a complex, multi-stage process that involves a variety of technologies and skills. This presentation will explore the key challenges faced during this migration, from the initial assessment to the final cutover. We will discuss critical issues such as converting SYSDATE and NUMBER datatypes, which, if not addressed correctly, can significantly impact database performance. The session will also cover the extensive time required to rewrite functionalities like PIVOT and hierarchical queries, and the importance of correctly handling NULL values to prevent application failures. Additionally, we will dive into the complexities of mapping system catalogs and the differences in partitioning between Oracle and PostgreSQL. Attendees will gain valuable insights and practical strategies to navigate these challenges, ensuring a successful and efficient migration process.https://postgresql.us/events/pgconfnyc2024/schedule/session/1715/SessionBAJI SHAIKSameer Malik15:0000:50The HubLearn how to do Postgres schema migrations without breaking dependent applications by leveraging the expand/contract pattern. pgroll is a new open-source migration tool for Postgres that keeps multiple versions of a schema live during a migration, helping you roll out database schema changes without downtime. Database schema migrations often pose significant challenges to developers, particularly when striving for zero downtime migrations. There are many things that can go wrong: - Schema changes breaking client applications - Unexpected table locking causing downtime - Human mistakes causing data loss This talk presents a new approach to schema migrations using an ‘expand/contract’ pattern where multiple versions of a database schema are maintained during the migration, allowing old and new versions of client applications to run side-by-side during an application rollout. Each version of the application sees the version of the database schema with which it is designed to work. The talks covers the ideas behind this approach, and the challenges to be overcome such as: - Limitations of existing tools and techniques for Postgres schema migrations. - The use of Postgres views to present multiple versions of an underlying table. - Backfilling data between old and new schema versions. - Techniques to avoid unexpected table locking during migrations. - Finally we introduce pgroll, an open-source tool that puts these ideas into practice and we show how it can be used to facilitate safe application rollouts across database schema changes.https://postgresql.us/events/pgconfnyc2024/schedule/session/1678/SessionAndrew Farries15:0000:50Library (Sponsor)Have you ever heard about Turn-key Postgres Platform? If you don't prefer to migrate your db workloads to the public cloud but still want to use the turn-key PostgreSQL on your data center, it is time to gain all the benefits of Kubernetes and Postgres faster and more easily. You don't have to worry about compute, storage and networking infrastructure because they are all included as well as the management console designed for large, multi-zone deployments. In this session, we will show how easy is to create a PostgreSQL cluster that meets the stringent performance and high availability SLAs required in enterprise environments.with a single UI. We will also walk through real use cases we have experienced.https://postgresql.us/events/pgconfnyc2024/schedule/session/1857/SponsorOnur Eroğlu15:5000:40Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1832/Session16:3000:50The ForumWe want it to be fast, reliable and flexible. We want JSON and we want OLTP with ACID. Can all it work together or we need to choose 2 out of three of our wishes again? What technology can help to speed up the JSON queries? What index should I use for my JSON data and will it help me? Those questions and some other tips about JSON data in Postgres.https://postgresql.us/events/pgconfnyc2024/schedule/session/1692/SessionGleb Otochkin16:3000:50Club ParlorIn the beginning, there was one database - a labor of love, manually constructed. The roles and privileges – birthed directly from psql by an optimistic engineer at the dawn of our journey. Then, another database emerged. And another. Quickly, this becomes untenable. Perhaps you've dabbled in scripting the creation of new roles, but what happens when the script takes a dark turn? Or worse, when you must alter the ancient permissions? Have you ever tumbled into the abyss of peculiar legacy users and arcane database configurations? The task of provisioning and managing an organization's infrastructure has been a longstanding challenge. However, over the past decade, a new paradigm has emerged - declarative configuration, with Terraform at its helm. While Terraform has been widely adopted to provision various AWS resources, the question arises - can we harness its power for our databases? Over the past two years, Brex has embarked on a journey of evolution in managing our users on a rapidly expanding database fleet. We've transitioned from manual care, to scripts, to custom software, and now, to Terraform. In this talk, we'll traverse the peaks and valleys of our journey, and share insights on what to be mindful of as we adopt and import our legacy systems into Terraform for Postgres.https://postgresql.us/events/pgconfnyc2024/schedule/session/1785/SessionSelena Flannery16:3000:50The HubIn this talk, I'll discuss the incremental backup feature which I developed for PostgreSQL 17. The talk will discuss how we determine what data has changed, and why the chosen approach was selected. It will then review in some detail how incremental backups can be taken and restored, and why things work as they do. It will briefly touch on use cases for the feature and possible future work in this area.https://postgresql.us/events/pgconfnyc2024/schedule/session/1661/SessionRobert Haas16:3000:50Library (Sponsor)As organizations manage data spread across various systems, moving, and integrating data can become a complex, time-consuming task. This presentation will showcase how a new solution from UK-based Liberatii which allows teams to work with data across multiple platforms—whether on-premise, in the cloud, or hybrid environments—without worrying about the underlying database. For example, moving from Oracle to Postgres no longer requires rewriting application or integration code, because Liberatii abstracts the database. Datavail is using this powerful tool for reducing complexity, speeding up deployment times, and future-proofing your infrastructure. We will also highlight how using this approach can help enhance security and provide faster access to data for analytics and reporting, all while ensuring that you don't need to rewrite even one line of code. Attendees will learn how to streamline their data infrastructure, reduce costs, and make database migrations or changes far more seamless, allowing teams to focus on innovation rather than data logistics.https://postgresql.us/events/pgconfnyc2024/schedule/session/1855/SponsorEric RussoJames ReynoldsNima Torabi17:2001:40Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1833/Session08:4501:00Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1834/Session09:5000:20The ForumAt AWS, we’ve supported customers running PostgreSQL in Amazon RDS for over a decade. We’ve learned a lot about their workloads, and because we’re customer obsessed, we continuously look for ways to create an even better experience for using PostgreSQL. In this talk, we’ll discuss how we’ll explore how we’ve applied the lessons from our customers to help contribute to PostgreSQL, and review a few recent enhancements to PostgreSQL.https://postgresql.us/events/pgconfnyc2024/schedule/session/1861/SponsorDavid Rader10:1000:30Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1837/Session10:4000:50The ForumThis talk will focus on explaining the key features like incremental backup, upgrades of logical replication setup, etc., and performance improvements done for PostgreSQL 17. Then this talk will also cover what is being discussed in the PostgreSQL community for the next and future releases of PostgreSQL. This talk will also summarize in brief how PostgreSQL has evolved over the years.https://postgresql.us/events/pgconfnyc2024/schedule/session/1745/SessionAmit Kapila10:4000:50Club ParlorAre you indexing your table's columns? It's time to explore a different approach. Instead of simply indexing columns, consider indexing the specific values and ranges you will search for. While in a highly normalized schema, this typically translates to one or more columns, modern applications often require searching for criteria within a text fragment, an attribute within a JSON structure, or the maximum values across multiple rows. In this session, we will explore several examples and evaluate the effectiveness of indexing by examining the execution plan.https://postgresql.us/events/pgconfnyc2024/schedule/session/1672/SessionFranck Pachot10:4000:50The HubBusinesses want to leverage private data for GenAI because it allows them to develop applications that can save them time, money, and resources. They can also derive new insights and use their data for a competitive advantage. Postgres plays an important role in enabling GenAI applications, not only as a vector database provider viae pgvector, but also as the data source at the beginning of GenAI pipelines. In this capacity, organizations have the opportunity to secure data stored in Postgres in a manner that ensures GenAI systems using the data downstream comply with data security and privacy regulations that would otherwise slow or block rollout of GenAI applications. In this talk, we will discuss: the implications of security, compliance, and privacy on GenAI applications the role of Postgres as GenAI data source and how to anonymize sensitive data in Postgres so it can be used in GenAI projects how to enforce access controls on sensitive data values presented through a typical GenAI application using RAG to meet data security and privacy compliance requirementshttps://postgresql.us/events/pgconfnyc2024/schedule/session/1759/SessionMin-Hank Ho10:4000:50Library (Sponsor)PostgreSQL operational pain points and solutions As part of our research, we have spoken with a large number of companies using Postgres at various levels of scale and sophistication. We have discussed in particular pain points around schema changes, replication / CDC, monitoring/observability, and scaling. As part of this talk, we’ll present the biggest milestones and the associated issues that people are hitting in their Postgres journey, as well as how we (at Xata) are trying to address these issues. We will be covering topics like: - schema changes gotchas, and our open source project, pgroll - replication to search and logical replication events, as well as our open source project, pgstream - major version upgrades using logical replication - scaling and sharding strategieshttps://postgresql.us/events/pgconfnyc2024/schedule/session/1849/SponsorMonica SarbuTudor Golubenco11:4000:50The ForumIn the past few decades, the role of the Postgres Database Administrator (DBA) has evolved significantly, giving rise to the "Database Platform Engineer." This role emphasizes building developer-owned database tooling, and managing/optimizing databases as a cohesive fleet. Across my last several jobs, I have contributed to & led several database platform teams, and managed Postgres for a wide range of projects. In this talk, I will share Postgres database platform engineering "best practices" refined through my experience at companies of varying size, level of engineering maturity, and data scale. These best practices focus on a wide range of technical and human-focused topics, including: * Metadata management * Logging & migration Postgres configurations * Networking/DNS * The role of databases in a microservice world * The advantages vs risks of a "self-service" infra platform * Maintaining quality of schema design & access patterns when your team doesn't own datahttps://postgresql.us/events/pgconfnyc2024/schedule/session/1789/SessionChelsea Dole11:4000:50Club ParlorWouldn’t it be nice if you could query your database as it was ten minutes, ten hours, or ten days ago? It would make audits, troubleshooting, and period vs. period reporting so much easier. Once upon a time, Postgres supported time travel queries, but it was deprecated almost thirty years ago because of its abysmal performance. In this talk, we present a new implementation of time travel in Postgres using logical replication to create a special versioned database read replica. We’ll discuss algorithms to construct this replica and efficiently query it, and mechanisms to compact the replica’s data and bound its size.https://postgresql.us/events/pgconfnyc2024/schedule/session/1711/SessionQian Li11:4000:50The HubIt is often said that "an untested backup is not a backup" but how can we turn that saying into something actionable? And in an organization with limited engineering bandwidth, what is the most important thing to prioritize? In this presentation, I aim to provide a practical guide on what I believe will give the greatest "bang for buck" with backup testing, and how to fold it into your operations, modeled after how we test our postgres backups at Academia.edu. This talk covers: * What are "physical" backups and what advantages do they offer? (For example: point-in-time recovery out of the box) * How does one test a "physical" postgres backup? * What are reasonable goals to set for Recovery Point Objective (RPO) and Recovery Time Objective (RTO) and how do we measure them? * How to couple the testing of your backups with operations * What additional monitoring to implement, with a worked example for pgBackRest and Datadog In addition, I’ll go over the scariest close calls with postgres backups I’ve witnessed and what we can learn from them.https://postgresql.us/events/pgconfnyc2024/schedule/session/1735/SessionNicholas Meyer11:4000:50Library (Sponsor)PGX is a boutique PostgreSQL consultancy, and we see a lot of broken databases. One of the first things we do is information-gathering to find out what the problem is. But what information do we look for? This talk is intended to help clients, and anyone, do initial analysis of common PostgreSQL issues so no one has to play Twenty Questions.https://postgresql.us/events/pgconfnyc2024/schedule/session/1856/SponsorChristophe Pettus12:3001:30Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1838/Session14:0000:50The ForumVectors are a centuries old, well-studied mathematical concept, yet they pose many challenges around efficient storage and retrieval in database systems. The heightened ease-of-use of AI/ML has lead to a surge of interest of storing vector data alongside application data, leading to some unique challenges. PostgreSQL has seen this story before with JSON, when JSON became the lingua franca of the web. So how can you use PostgreSQL to manage your vector data, and what challenges should you be aware of? In this session, we'll review what vectors are, how they are used in applications, and what users are looking for in vector storage and search systems. We'll then see how you can search for vector data in PostgreSQL, including looking at best practices for using pgvector, an extension that adds additional vector search capabilities to PostgreSQL. Finally, we'll review ongoing development in both PostgreSQL and pgvector that will make it easier and more performant to search vector data in PostgreSQL.https://postgresql.us/events/pgconfnyc2024/schedule/session/1862/SessionJonathan S. Katz14:0000:50Club ParlorLocking is critical for providing high concurrency for any database — you cannot fully utilize your hardware if locking is throttling its use. This talk explores all aspects of locking in Postgres by showing queries and their locks; covered lock types include row, table, shared, exclusive, and advisory lock types. The high concurrency provided by Multiversion Concurrency Control (MVCC) is also covered.https://postgresql.us/events/pgconfnyc2024/schedule/session/1598/SessionBruce Momjian14:0000:50The HubPostgreSQL is a robust relational database management system that's been under active development for 35+ years. What makes it tick? In this talk, I'll share my journey of how I discovered Postgres and began working with it from the first time. Join me as we discuss: What makes PostgreSQL so successful? What can be improved? What are the different ways you can contribute to the community?https://postgresql.us/events/pgconfnyc2024/schedule/session/1813/SessionEmma Saroyan14:0000:50Library (Sponsor)Microsoft has been building major enhancements to its flagship managed Postgres service, Azure Database for PostgreSQL - Flexible Server. This session will dive into the most recent improvements with a whirlwind tour covering availability with Geo Disaster Recovery, improved performance with Premium Disk V2, and enhanced data protection using Azure Defender and Entra ID (AAD). We’ll also look at seamless upgrades with near-zero downtime scaling, planned failovers, new hardware offerings, and AI integration. Expect a technical session with demos you can try at home.https://postgresql.us/events/pgconfnyc2024/schedule/session/1851/SponsorGuy BowermanKanchan Bharati15:0000:50The ForumAs your database grows, the performance and maintenance of large tables can become challenging. Fear not! PostgreSQL has the right tool for the job: declarative table partitioning. In this talk, I will explore the benefits of partitioning in PostgreSQL, including improved performance and simplified maintenance. After introducing the benefits of table partitioning, I’ll discuss the different types of partitioning available in PostgreSQL, such as range, list, and hash partitioning, and highlight common use cases and trade-offs. With practical examples, I’ll demonstrate how to implement declarative partitioning in PostgreSQL while maintaining transparency to the application. We will also discuss some key considerations and best practices when designing the schema for partitioning. I’ll finish with a look at popular PostgreSQL extensions like TimescaleDB and Citus for similar workloads. By the end of this session, you will be able to create partitioned tables in PostgreSQL that improve the performance and maintenance of your application, allowing you to scale with more control than ever before.https://postgresql.us/events/pgconfnyc2024/schedule/session/1713/SessionRyan Booz15:0000:50Club ParlorSecurity in your database needs to be handled at quite a few different levels, from individual users, to in transit calls, and the server and storage itself. This talk is a high level introductory overview of security topics and best practices in Postgres. We’ll start with security and roles inside your database, including row level security. Next, we’ll review security for the machines on different deployment methods including Kuberentes and Cloud. Of course, we will review security for the Postgres code itself and how CVEs are addressed. Next time you have assessment boxes to check or the CEO calls you in to ask about how secure your database is, you’ll be ready to answer the tough questions.https://postgresql.us/events/pgconfnyc2024/schedule/session/1682/SessionElizabeth Christensen15:0000:50The HubStep into the future of data-driven decision-making with real-time predictions using PostgreSQL and PL/Python. In this session, we will explore how to seamlessly integrate machine learning models into PostgreSQL to perform real-time predictions. Through practical examples, you will learn how to set up a real-time prediction pipeline, implement machine learning models with PL/Python, and deploy these models for instant, actionable insights. This talk is ideal for data scientists, database administrators, and application developers looking to enhance their PostgreSQL databases with real-time predictive capabilities.https://postgresql.us/events/pgconfnyc2024/schedule/session/1738/SessionDrishti Jain15:0000:50Library (Sponsor)While PostgreSQL 17 boasts significant advancements, some DBA-centric features remain on the horizon. This presentation delves into seven key areas where DBAs might encounter limitations and offers practical workarounds to navigate these challenges effectively. Discover how to optimize some administrative limitations and more, all while maximizing the potential of your PostgreSQL 17 environment.https://postgresql.us/events/pgconfnyc2024/schedule/session/1858/SponsorVibhor Kumar16:0000:30Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1839/Session16:3001:00Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1840/Session08:4501:00Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1841/Session09:5000:20The ForumAt Havelsan, our R&D has focused on this area with the development of open-source technologies in the database world since 2020. Through these efforts, we have undertaken numerous projects in database transformation operations and infrastructure layers. In this session, we will share what we learned, what we did wrong, and our goals.https://postgresql.us/events/pgconfnyc2024/schedule/session/1843/SponsorN. Semih Demirtoka10:1000:30Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1844/Session10:4000:50The ForumAmazon Relational Database Service (Amazon RDS) operates a variety of large, very-high-throughput PostgreSQL databases in unusual architectures. Over the years, we've experienced substantial performance and availability challenges that we've had to mitigate. We'd like to share our experience in maintaining large PostgreSQL instances that reach vertical scaling limits, our long-term solutions, how we bought time when migrating to more performant architectures.https://postgresql.us/events/pgconfnyc2024/schedule/session/1769/SessionAlisdair OwensAndrei Dukhounik10:4000:50Club ParlorThe talk will start with motivations for why full-text search in Postgres needs improvements. This talk will discuss how we improved full-text search in Postgres by integrating Tantivy, a Lucene-inspired library, to offer a BM25 index in Postgres. We will discuss the history of search in Postgres, pgvector's recent rise to fame, and its integration with ParadeDB pg_search. The talk will conclude with some real-world examples and motivations for other areas of Postgres which can be improved as part of extensions.https://postgresql.us/events/pgconfnyc2024/schedule/session/1782/SessionPhilippe Noël10:4000:50The HubSSL support in PostgreSQL is old, and SSL itself is even older. New versions have appeared and many new setup and configuration options have been developed. On the other side, regulations, policies, and security concerns affecting many users are complicated and evolving. In this talk I will discuss how to set up SSL for PostgreSQL in a modern and robust way, which versions and options to select and which ones to forget, how to select protocol versions and ciphers, how to deal with keys and certificates, and how to address (some) regulatory and policy issues. I'll also look at how connection proxies and poolers such as PgBouncer affect SSL deployments. Finally, we'll look at some new facilities in PostgreSQL 17 that will hopefully make SSL deployments for PostgreSQL more efficient and secure in the future.https://postgresql.us/events/pgconfnyc2024/schedule/session/1728/SessionPeter Eisentraut10:4000:50Library (Sponsor)Everybody is building generative AI apps on PostgreSQL these days. And to provide fast, accurate responses, you have choices to make. Approximate nearest neighbor (ANN) or k-nearest neighbor (kNN) search? APIs or SQL queries to retrieve the data? With or without an orchestration framework? Using which gen AI model and on which platform? We’ll dive into the main use cases we’ve encountered on Google Cloud and share some learnings from Google Search. You’ll gain insights into tradeoffs you may want to think about for your next application, whether you build it on our AlloyDB database or on standard PostgreSQL.https://postgresql.us/events/pgconfnyc2024/schedule/session/1859/SponsorSandy Ghai11:4000:50The Forum"Advancing Search with PostgreSQL" is a session that details how PostgreSQL's features can be effectively used to significantly enhance the search functionality in any application. In this session, we'll share insights on how we used PostgreSQL to facilitate precise searches across multiple fields in our mobile application. The techniques include using LIKE and ILIKE operators and integrating a trigram-based search to handle potential misspellings, thereby increasing the search accuracy. Additionally, we implemented search based on similarity using data in vector types and created a Retrieval-Augmented Generation (RAG) application. By leveraging PostgreSQL's features and functionality, we developed a comprehensive search solution, which led us to victory at the TIDE NATO Hackathon. While our application's case study involves a drug search, the techniques and principles shared in this session can be adapted to improve search functionality in a wide range of applications. Join us to learn how PostgreSQL can be harnessed to enhance your application's search capability.https://postgresql.us/events/pgconfnyc2024/schedule/session/1821/SessionTaras Kloba11:4000:50Club ParlorSome signs that you may want to think about upgrading postgres: * You have to click on the “Unsupported versions” row to double check everything you read in the docs * You hear about a feature that has been around since postgres 11 (or 9.3) and wish that you were able to use it * Whenever you mention the version you are running, people mention how out of date it is While there is a lot of advice emphasizing the benefits of upgrading, sometimes it can feel like there are fewer details on how to deal with trickier situations you may be in. I hope to partially fill this niche with our story of how we at Academia.edu went from 9.2 to 15. In this talk we will cover: * Running pg_upgrade when you have streaming replicas and want to minimize downtime * Using pglogical with DDL replication and the challenges that come with it * Various complications that occurred along this journey * Greatest technical frustrations I have experienced while upgrading * My personal list of favorite new features from 9.3 to 15https://postgresql.us/events/pgconfnyc2024/schedule/session/1819/SessionNicholas Meyer11:4000:50The HubPostgreSQL is free and open-source, so there is no license fee to pay. However, maintaining the project—and growing it successfully—requires the work of many people: engineers, website managers, event organizers, project administrators, advocates, social media managers, and on and on. It takes a global community to support the project, and community participation is the license fee we pay to ensure the project's continuity. I'll give an overview of how the project is organized, and also provide a myriad of ideas for becoming involved in ways that meet your interests and skills, even if—and especially if—you are not an engineer.https://postgresql.us/events/pgconfnyc2024/schedule/session/1689/SessionStacey Haysler11:4000:50Library (Sponsor)Managing database schema changes in a DevOps culture can be a daunting task. Having a process in place that provides for quick, predictable iterations across a team of developers and DBAs can accelerate your ability to deliver value to customers. In this session, we'll talk about the various methods of deploying changes from dev to production, the key factors that drive success in a development team, and how to begin your automation journey. From there we'll take a hands-on journey through the process using Flyway and GitHub Actions to automatically deploy changes. Along the way, we'll discuss how to verify changes, check for errors, and celebrate a faster, predictable process.https://postgresql.us/events/pgconfnyc2024/schedule/session/1852/SponsorRyan Booz12:3001:30Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1845/Session14:0000:50The ForumThis session discusses how at Tines we use logical replication in PostgreSQL to facilitate major version upgrades and tenant migrations with minimal downtime. Participants will discover the concepts of logical replication and its different uses. We'll also discuss the use of pg_easy_replicate, a tool developed at Tines, which has been instrumental in performing seamless upgrades across 50+ production databases via blue/green setup with DNS based switchovers. The talk aims to provide practical insights into leveraging logical replication and pg_easy_replicate for efficient database transitions with minimal impact on operations.https://postgresql.us/events/pgconfnyc2024/schedule/session/1577/SessionShayon Mukherjee14:0000:50Club ParlorWant to become a PostgreSQL performance monitoring star? Come and explore the world of the PostgreSQL Cumulative Statistics System, a.k.a. the pg_stat views, and see how this system can give you insights on critical workload activity such as visibility into I/O activity and much more that can help in real-time troubleshooting as well as proactive monitoring. In this talk, we will explore why this system exists, the wealth of feature it offers, effective usage and highlight recent development, such as “pg_stat_io” which was added in PostgreSQL 16. We will also walk through real world examples of using this system and provide helpful scripts.https://postgresql.us/events/pgconfnyc2024/schedule/session/1763/SessionChirag DaveSami Imseih14:0000:50The HubEngineers are not great at documentation, it’s just a fact of the industry. Data people are in the same group. In this talk, I’ll speak specifically to DBA and operational Data professionals to help them understand why Documentation is critical. I will also show some great methods to make it less painful and provide the solution to the biggest problem for all data professionals, Not having enough time for the projects they really want to work on. This talk will show ways to make documentation a lot faster and better for everyone.https://postgresql.us/events/pgconfnyc2024/schedule/session/1628/SessionPat Wright14:0000:50Library (Sponsor)In this session we'll walk through best practices of optimizing a slow query in Postgres, and how we can have a better mental model for how Postgres plans out a query, be it join order, index usage, or planner costing - and getting a sense for knowing when it goes wrong. We'll then discuss different ways of finding tuning opportunities for a given query, and how to benchmark them. These approaches can be used on any Postgres database without additional tooling. Additionally we'll also discuss real world slow query examples and improve them using new query optimization features in pganalyze. pganalyze enables your whole engineering team to tackle Postgres performance issues more effectively, and identify the root cause of slow queries.https://postgresql.us/events/pgconfnyc2024/schedule/session/1854/SponsorLukas Fittl15:0000:50The ForumToggl Track engineering team embarked on a journey to develop a Postgres-based near real-time OLAP database system to better serve our reporting needs. Toggl Track has been using a monolithic Postgres database since its beginnings and the goal was to progressively shift away our report queries from the transactional database creating a CDC on top of it, leveraging logical replication by using data-changing events, and applying whatever transformations are required to reach the desired - report-friendly - data model, OLAP. After overcoming some difficulties in developing our own logical replication client, mostly due to incorrect assumptions on the usage – and track - of logical replication events coming out of Postgres, and after the successful production deployment of the first few use cases, the ETL pipeline supporting our OLAP infrastructure has been used as a cornerstone in our push towards an event-based architecture, becoming the base of some of our largest engineering projects, not all related with OLAP usages, from pre-computed authorization sessions to business intelligence functionalities, passing by user-facing event streams such as WebSockets or mobile notifications. Our ETL pipeline is – mostly - composed of data-changing events sourced from our transactional Postgres databases through logical replication using our client. This talk is a continuation of “A journey into postgresql logical replication” presented in PGConf.EU 2023 where we presented the intricacies of our solution and went deep(er) into the way that logical replication events are expected to be used, and the challenges that we faced while developing Toggl Track logical replication client, but also the needs and problems that created the right environment for us to embrace the challenge. While briefly presenting the road that led us here - months of experimentation, debugging, and workarounds, leading to the conclusion that we were losing data due to the incorrect interpretation of LSN positions - this time we want to share the many quirks and kinks learned while using Postgres not only as a source of data – transactional data – but also to handle the OLAP side of things, from the transformation heavy lifting, feed by ETL, to strategies to handle the load while avoiding locking while keeping up with data changes in near real-time.https://postgresql.us/events/pgconfnyc2024/schedule/session/1662/SessionJosé Neves15:0000:50Club ParlorOur mission relies on open-source software for its flexibility and cost-effectiveness. However, we've encountered a significant hurdle: Greenplum, the distributed database we use, has shifted from open source to proprietary. This change directly impacts our mission, which requires a scale-out database that supports specific PostgreSQL extensions like PgSphere and q3c. While open-source software offers numerous advantages, it also carries risks, including unexpected licensing changes that can disrupt vital operations. In this presentation, we delve into the critical role of open-source software in our project, explore the challenges introduced by Greenplum's new status, highlight the inherent risks of relying on open-source solutions, and outline our proactive strategy to transition to a new open-source database—ensuring our mission continues seamlessly.https://postgresql.us/events/pgconfnyc2024/schedule/session/1666/SessionJoaquim OliveiraPilar de Teodoro15:0000:50The HubI use pg_hint_plan to learn about the optimizer, to understand the query planner choices, and, sometimes workaround a problem in production. But hinting is not easy. Fixing the exact plan needs more than one hint. In this live demo session, we will go though understanding and examples on how to use pg_hint_plan in a predictable way. It is also a good occasion to understand the access paths, join order and methods, and how to read an execution planhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1675/SessionFranck Pachot15:0000:50Library (Sponsor)The most popular way that the databases implement for data at rest encryption is Transparent Data Encryption (TDE). It’s implemented in all other top 5 popular databases (according to db-engines.com). To add this security feature to PostgreSQL databases, Percona is working on an open source extension to fill this need, the pg_tde project. This talk will detail the journey of implementing pg_tde from a developer’s perspective. I’ll discuss the challenges we faced making design decisions, the implementation process, and lesson learned. My goal is to show the PostgreSQL why we created the extension the way we did and show its value to the ecosystem. At the end of the talk we hope that you try pg_tde, test it, give us feedback, join the conversation around why implementing pg_tde is a win for the PostgreSQL community.https://postgresql.us/events/pgconfnyc2024/schedule/session/1848/SponsorMuhammad Usama16:0000:15Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1846/Session16:1500:15Otherhttps://postgresql.us/events/pgconfnyc2024/schedule/session/1847/Session