Wielding Debug Builds against Heap Corruption

Heap corruptions can be nasty to investigate. I recently had to investigate a bug that manifests under special circumstances in production. Crashes had all flavors of stack traces - the trademark of heap corruption. To debug it, I ran the usually recommended tool for heap corruption: PageHeap. Enabling PageHeap PageHeap is an OS flag that … Continue reading Wielding Debug Builds against Heap Corruption

The Curious Case of the 99.9% Latency Hike

Disclaimer: This is a story from production. I have changed a couple of details, but the gist of the story holds. Exposition On a normal morning, I was checking up on health metrics of a bunch of services, when a new service under test caught my attention. Let's call this service Lucy. Lucy so far … Continue reading The Curious Case of the 99.9% Latency Hike