Yes, Yet-Another VSS Post But With Extra RAG Fun!

We’ll give this topic a rest for a while after this post. But, I felt it was only fair to show how to do what we’ve done in SQLite and DuckDB in my fav daily use language (R). Both SQLite and DuckDB have the limitation in that all the vector search operations are performed in memory. Sure, LanceDB, OpenSearch, and a host of other [database] environments can perform VS[S] ops across memory and storage, but a decent chunk of what we all want to search for fits well in memory. So, no need to layer in the additional complexity of a database, when we can do all this in R

So, yes this is an infrequent R-centric Drop, as I try my best to not make this an R blog, but they sneak in every now and again.

HOWEVER!

I’m tossing in some work with Ollama and the relatively new llama3.1 model that is easily transferrable to any language.

So, we’re going to replicate the CISA KEV similarity search from the other day in the first part, but then go on to build a very “lite” RAG (retrieval augmented generation) that will use the embeddings matrix we’ll be making to help perform natural language queries over KEV.

I think it’ll also help to show that there’s a whole lot of good ol’ human programming that goes into making LLMs useful (too much, IMO, but that doesn’t make me popular with the folks who chase after VC money).

Again, this is a code block-heavy post, so we’ll need to go to the companion site once again.

FIN

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev ☮️