Clustering: K-Means from Scratch

Run CREATE INDEX ON chunks USING ivfflat (embedding vector_l2_ops) WITH (lists = 1000) in Postgres with pgvector, and before the index answers a single query the database clusters your data. It samples 50 rows per list, 50,000 here, picks 1,000 starting centroids with a method called k-means++, iterates until no sampled row changes cluster (at most 500 passes), and files every row under its nearest centroid. FAISS, the library inside several vector databases, does the same when it trains an IVF index. Every query against such an index compares the question with the centroids first and scans only the few lists whose centroids are closest.