Skip to main content

Databases

Database design and optimization

6 posts

Engineering

Debugging a Qdrant Re-indexing Pipeline: From Memory Leak to Efficient Batch Embeddings

Dive deep into solving a complex memory leak in a Qdrant vector database re-indexing pipeline, revealing optimization strategies for batch embeddings processing.

Engineering

Taming Longhorn: How Snapshot Limits Saved Our K3s Cluster from Disk Pressure Disasters

A practical guide to preventing unbounded storage consumption in Kubernetes with Longhorns snapshotMaxCount parameter. Learn how we reduced potential disk usage by 98% and brought cortex-qdrant back to replicated storage.

Engineering

Redis vs PostgreSQL: Why "Replace" Is the Wrong Question

Understanding when to use Redis vs PostgreSQL in modern systems - not a choice between databases, but a complementary relationship leveraging each tool's strengths for optimal performance and durability

Engineering

Deploying Redis-Backed Catalog Service: From 500ms to 1ms Asset Lookups

Rebuilding the Cortex catalog service with Redis backend on K3s cluster - achieving 500x performance improvement with sub-millisecond lookups, high availability, and automated discovery

Engineering

Understanding Database Transactions and ACID: The Foundation of Data Integrity

A deep dive into database transactions and ACID properties. Learn what transactions are, why they matter, and how ACID guarantees keep your data consistent even when things go wrong.

Engineering

Database Sharding Strategies for Scale: From Theory to Production

A comprehensive guide to database sharding - when you need it, how to implement it, and what strategies work best. Learn horizontal partitioning techniques that enable applications to scale beyond single database limits.