This repository is a Retrieval-Augmented Generation (RAG) example using Apache Pinot, LangChain, and OpenAI. The use case is to load documentation and allow an LLM to answer questions provided by a user. This approach enables you to generate AI responses that are fresh and in real time.This RAG example uses LangChain’s RecursiveUrlLoader. It accepts a URL, recursively loads pages, and converts them into documents. These documents are converted into embeddings, submitted to a Kafka topic, and consumed by Apache Pinot.To understand how this recipe processes data, examine the commands executed in the Makefile.
This repo builds the Apache Pinot project. You may get an error No space left on device when building the container. Execute the command below to free resources before building.
docker system prune --all --force
Building the Pinot image will take about 25 minutes to finish.
Run the command below to load Pinot with embeddings from your document site by providing a URL. The loader will recursively read the document site, generate embeddings, and write them into Pinot.
make loader URL=https://dev.startree.ai/docs
If you have a large document site, this loader will take longer. You will see confirmations on the screen as each embedding is sent to Kafka and Pinot.This loader creates an embedding per page so that we can perform an UPSERT in Pinot. If you have larger pages and depending on the AI model you are using, you may get this error:
This model's maximum context length is 8192 tokens
Alternatively, you can chunk the pages into smaller sizes and UPSERT those records in Pinot by URL + ChunkId. The implementation in this repository does not do that.
