According to @_avichawla, 8 LLM precisions trade accuracy for memory, enabling FP8, INT8, and NF4 gains and KV cache cuts for longer contexts.

According to @_avichawla, moving transforms to GPU cuts CPU GPU transfer 4x; binary quantization shrinks embeddings 32x for fast RAG search.