Prefill causes computational bottlenecks, and decoding causes bandwidth bottlenecks—this mismatch gives rise to PagedAttention, prefix caching, continuous batch merging, predict-and-check, sparse attention, and linear interleaving. This is the algorithmic layer that will determine the true cost…
Prefill causes computational bottlenecks, and decoding causes bandwidth bottlenecks—this mismatch gives rise to PagedAttention, prefix caching, continuous batch merging, predict-and-check, sparse attention, and linear interleaving. This is the algorithmic layer that will determine the true cost…
O(n log n) doesn't always beat O(n²). With small datasets, it's the CPU cache that makes the difference, not the formula.
Dictionaries in every language are hash tables. Understanding how they handle collisions is key to understanding why they can sometimes be unusually slow.
All vector databases are based on this concept. Understanding it means understanding why semantic search is both fast and not absolutely accurate.
A "safe" false-positive structure: it may falsely report something as present, but it never fails to detect what is actually there.
When dividing data using modulo division, adding a single machine requires transferring almost all of the data. Hash tables solve exactly that problem.
Four common algorithms, each of which allows for a different type of explosion. Choosing the wrong one results in either a false positive or a false negative.
A data structure that allows multiple people to edit simultaneously, even when the network is down, and then merge their changes without the need for a mediator—explained using examples such as shopping lists, counters, and text editors.
