Field Signal · Curated Reading
The Engineering
Signal.
High-signal field notes, architecture teardowns, and engineering writing from 50 publications. A focused reading queue for becoming a stronger end-to-end engineer. Updated 19 Aug.
Poisoned Postgres connection pools
An engineer's worst nightmare is waking up to a seemingly read-only database with no clear issue in sight. Let's dig into why this happens, and how AI can help solve it.
What is a data topology?
A data topology describes the sharding scheme a Neki router uses to map logical PostgreSQL tables to physical shards and route queries.
How We’re Building Scam Alert on WhatsApp With End-to-End Encryption and Verifiability Guarantees
WhatsApp is committed to helping people stay safe while protecting the privacy of their messages. As scam tactics evolve — from impersonation to social engineering to AI-generated lures — we’re always evolving as well, so that our protections stay ahead of scammers while protecting people’s pe...
How Medium Powers Real-Time Recommendations at 1M OPS
Inside Medium’s move from relational features to list features in its ScyllaDB-based feature store “Keep readers reading” is the not-so-simple goal of Medium’s recommendations system. To predict what’s most likely to appeal to a particular reader at any given time, Medium continuously processe...
The dangers of Postgres subtransactions
Subtransactions can slow down your entire PostgreSQL server and break your high availability strategy by keeping new read replicas from accepting connections.
ScyllaDB Customer Experience Spotlight: Susie Solis
Meet Susie Solis, a Technical Support Engineer on the Customer Experience team here at ScyllaDB.
Concurrency vs. Throughput: why more parallelism can make databases slower
Increasing throughput sometimes requires reducing parallelism. A look into why this is the case for Vitess + MySQL databases
From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking
Every day, Meta’s recommendation platforms handle billions of user interactions, generating rich temporal signals that capture individual preferences and intent across products, ads, and content. In our 2024 post on sequence learning for ads recommendations, we showed how modeling the order and timi...
DDIA 2nd Edition Excerpt: On Scalability
Martin Kleppmann and Chris Riccomini's scalability considerations for designing data-intensive applications -- from the second edition of the Designing Data-Intensive Applications book
GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model
Meta’s Generative Ads Recommendation Model (GEM), the foundation model behind ads recommendations across Instagram and Facebook, now trains at LLM scale on several thousand of the latest-generation GPUs. This post goes into the details on how we achieved: doubling end-to-end (E2E) training eff...
Massively parallel Postgres backups
PlanetScale backs up petabyte-scale sharded Postgres databases in hours using parallel infrastructure, object storage, and WAL replay.
Lessons Learned from Real-World NoSQL Database Migrations
Discover the strategies, challenges, and trade-offs teams faced in a few real-world migrations to ScyllaDB
Postgres backups under the hood
With backups being such a vital part of keeping your data safe, how do they actually work?
What's new in Postgres 19
`VACUUM` reclaims dead-tuple space but doesn't shrink a table. PostgreSQL 19 Beta 2 adds an in-core online rewrite with REPACK (CONCURRENTLY), and much more.
Offloading I/O to Dedicated Cores: An Asymmetric io_uring Backend for Seastar and ScyllaDB
We moved low-level I/O execution off application cores to dedicated networking cores using Seastar’s new asymmetric_io_uring backend. Explore the architecture design, trade-offs, and benchmark results.
What Matters Most for NoSQL Migrations
How to prioritize the things that matter most for planning, executing and de-risking your NoSQL database migration
Every UPDATE Leaves a Ghost: MVCC, Bloat, and VACUUM in PostgreSQL
What happens to old row versions after nobody needs them? They become dead tuples, and over time they bloat your tables. Go deeper into PostgreSQL MVCC with runnable psql examples.
Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization
Hierarchical Interest Representation is a research area for Meta Ads. We’re exploring an upstream representation layer over the universe of Ads entities – users, advertisers, products, services – learning unified embeddings that connect users’ inferred interests with the breadth of what advert...
Making 768 servers look like 1
How to make 768 distinct Postgres servers look like 1 to your applications.
Build Durable Chat Memory for RAG Using ScyllaDB and LangChain
How to replace LangChain's in-memory chat history with ScyllaDB — so your RAG chatbot retains context across restarts and scales across replicas
Modernizing the Meta Ads Service With an Open-Source Kernel Scheduler
TL; DR At Meta’s scale, a few milliseconds of latency degradation can have a significant negative impact on ads performance. When a Linux kernel upgrade risked regressing latency across Meta’s ad serving fleet, we turned to sched_ext — the upstream, BPF-based extensible scheduling framew...
When the Postgres query planner goes rogue
How PlanetScale Database Traffic Control contained a runaway Postgres query after the planner abandoned an index.
ScyllaDB Is Now Supported in MCP Toolbox for Databases
Connect your AI agents to ScyllaDB using the new ScyllaDB integration in MCP Toolbox for Databases
Deadlocks and downtime
Deadlocks happen when transactions block each other. Learn how they escalate into downtime, how to reduce them through better queries and retry logic, and how Traffic Control can protect your database from your application.
Agent Memory at Monster Scale with Mem0 and ScyllaDB Cloud
Combine Mem0’s memory management with ScyllaDB’s persistence features to deploy large-scale AI agents