Shivanjan Chakravorty.
Writing
2026
Scan once, scatter once
cutting route reads 48x in the scheduler of Cursor's MoE megakernel, and the third sighting of the same bug
Machines that imagine ahead
the thesis this notebook keeps circling: acting well requires rehearsing the future, and rehearsal only matters if it costs less than reality
Where the eight seconds were hiding
packing narrow QK into FP8 and hunting memory traffic: 1.297 to 1.167 minutes, 5.15% under the record's same-node baseline
The operating plan
an operating plan written down in public so it can be graded later
96 is enough: shrinking attention's routing vectors
Q and K decide where to look; V carries what's found. They don't need the same width.
Move the index, not the data
a 3.53x MoE routing speedup, and the general principle hiding under it
Three small patches, three training-system lessons
a hook, a metadata dialect, and test coverage: what Megatron-LM taught me about extension seams
Anisotropic voxels, or why medical volumes hate cubes
CT stacks are 6:1 anisotropic. Resampling to cubes costs 6x memory and invents data. Now in Warp 1.13.0.
Making a video model fit: distill, cache, offload
getting a 14B image-to-video pipeline through a 48 GB card, and what each trick trades away
Contact lists that remember
bonded-particle DEM needs pair state that survives the substep. I proposed it upstream; it wasn't taken. The idea still matters.
Guardrails that compile
one policy source emitting both prompt-injection filters and AppArmor profiles, so the model's rules and the pod's rules cannot drift apart
2025
Blockwise parallel generation
diffusion language models trade the autoregressive bottleneck for a denoising one. notes from building an overlay
What FlashAttention actually buys
not fewer FLOPs. the same quadratic arithmetic with linear memory traffic, and why the distinction decides everything downstream
Walk on spheres: solving PDEs with random walks
a Monte Carlo method that never meshes the domain, and why it fits the GPU so well
The KV cache is the product
serving economics from first principles: bytes per token, GQA, paged allocation, and why batch size is set by memory rather than compute
Ten thousand robots per GPU
what vectorized environments and tiled rendering actually buy in policy training
2024
2023
Discovering Eigenvectors, Eigenvalues, and Eigenspaces: A Fun Adventure
A adventure with a robot named Robo to learn Eigenvectors, Eigenvalues, and Eigenspaces
Outliers
wandering mind, wandering thoughts
Hacking-Proof: The Security Features of Golang
Security features that the Golang compiler and semantics provide
IPC: Shared memory
concepts of C in Golang
Golang’s Efficient Garbage Collection Compared to Java
Comparing features of Golang
Signal Attenuation and Recovery in 5GHz WiFi
Understanding the Math Behind It
Simulating Particle Motion in an Explosion
A Mathematical Approach
Exploring the Darknet
A Guide to Exploring the Darknet Safely and Responsibly