ProjectIntent Over Keywords: From Stitched Embeddings to a Production Fashion Agent · Module 4: Speech: Whisper and MiniLM · part 6 of 26
Part 6 · Module 4: Speech: Whisper and MiniLM
Module 4: Speech: Whisper and MiniLM
22 min read·
About 45 minutes · Needs: Module 2 (the stitched pipeline), Module 3 (frames, CLIP and cosine similarity) · Code: part1-video-rag-stitched/src/embedders/local_models.py, part1-video-rag-stitched/src/text/chunking.py
Where this fits
This module: the speech path of Part 1. Whisper and MiniLM are amber because they are models; the text collection is our own code (blue).
Part 1 hears the video through two models. Whisper turns the soundtrack into text with times, and MiniLM turns pieces of that text into vectors that can be searched. This module follows the spoken half of the test video from the audio file to the 12 points in the p1_text collection. Module 3 did the same for the frames and CLIP; Module 5 shows how the two halves are searched and merged.
The rest of this project is yours to keep
This project is bought on its own, once, and stays readable afterwards, including the parts added to it later.