Tech Ninja

*Vector databases and why they matter in the LLM and Gen AI world* Large language models like GPT generally represent words and sentences as vectors or embeddings. Vector databases are nothing more than a database of these vectors and work hand-in-hand with Large Language Models (LLMs) specifically for enterprise AI use-cases. These embeddings capture the semantic meaning and allow models to understand "dog" and "puppy" are close to each other. Common AI use-cases for LLMs include enterprise search, chat on internal databases, summarization and key entity extraction. Operationalizing these use-cases involves a bunch of steps and includes Step 1 - Vectorizing the Knowledge - When you have a custom knowledebase (KB), the first order of business is translating every piece of information into a vector - a multi-dimensional mathematical representation of that information Step 2 - Storing and Searching the Vectors - Vector databases help you store and look up these vectors efficiently. Fundamentally, they enable you to find similar items based on a query vector. So you can easily perform searches for "like" or "close to" other words or phrases, at scale. Step 3 - Integration with LLMs - LLMs like GPT4, don't know have information from your custom knowledgebase, when a user sends a prompt/query, we first have to look-up the Vector database for the relevant piece of information For example, if you are looking for information to log-in to your internal HR system, and you ask your custom ChatBot, it needs to go to the vector database to first fetch the web page that has this information The custom ChatBot will then send this information to the LLM API (GPT-4) along with the user-query. The LLM API will then answer the user's question by reading the webpage about your internal HR system Step 4 - Orchestration and Operationalization - Now that we have responded to the first user query, the user may have follow-up questions or start a thread. Developers have to come up with the complex orchestration flow, where they keep state across chats, do multiple vector database look-ups and facilitate a smooth conversation The process of operationalizing all these steps and launching your custom ChatBot or LLM-based Enterprise AI use-case in production is often termed - LLM Operations or LLMOps You can use an end-to-end LLMOps platform to operationalize your use-case or build your own end-to-end system by using all the LEGO blocks such as vector databases, LLM APIs and kubernetes. Key resources: Open source vector databases chroma - trychroma.com/ milvus - github.com/milvus-io/milvus end to end - LLMops platform Abacus - abacus.ai pic credit and a good read - medium.com/thirdai-blog/unde…