According to @_avichawla, KMeans inference costs O(kd) per sample as you compare to k centroids in d dimensions, assuming precomputed centroids.