Retrieval Augmented Generation (RAG) Tutorial

One of the most common use-cases for LlamaIndex is Retrieval-Augmented Generation or RAG, in which your data is indexed and selectively retrieved to be given to an LLM as source material for responding to a query. You can learn more about the concepts behind RAG.

Set up the project

In a new folder, run:

npm
Yarn
pnpm

npm init
npm install -D typescript @types/node

yarn init
yarn add --dev typescript @types/node

pnpm init
pnpm add -D typescript @types/node

Then, check out the installation steps to install LlamaIndex.TS and prepare an OpenAI key.

You can use other LLMs via their APIs; if you would prefer to use local models check out our local LLM example.

Run queries

Create the file example.ts. This code will

load an example file
convert it into a Document object
index it (which creates embeddings using OpenAI)
create a query engine to answer questions about the data

import fs from "node:fs/promises";

import {
  Document,
  MetadataMode,
  NodeWithScore,
  VectorStoreIndex,
} from "llamaindex";

async function main() {
  // Load essay from abramov.txt in Node
  const path = "node_modules/llamaindex/examples/abramov.txt";

  const essay = await fs.readFile(path, "utf-8");

  // Create Document object with essay
  const document = new Document({ text: essay, id_: path });

  // Split text and create embeddings. Store them in a VectorStoreIndex
  const index = await VectorStoreIndex.fromDocuments([document]);

  // Query the index
  const queryEngine = index.asQueryEngine();
  const { response, sourceNodes } = await queryEngine.query({
    query: "What did the author do in college?",
  });

  // Output response with sources
  console.log(response);

  if (sourceNodes) {
    sourceNodes.forEach((source: NodeWithScore, index: number) => {
      console.log(
        `\n${index}: Score: ${source.score} - ${source.node.getContent(MetadataMode.NONE).substring(0, 50)}...\n`,
      );
    });
  }
}

main().catch(console.error);

Create a tsconfig.json file in the same folder:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "esnext",
    "moduleResolution": "bundler",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true,
    "lib": ["ES2022"],
    "types": ["node"],
    "outDir": "./lib",
    "tsBuildInfoFile": "./lib/.tsbuildinfo",
    "incremental": true,
    "composite": true
  },
  "ts-node": {
    "files": true,
    "compilerOptions": {
      "module": "commonjs"
    }
  },
  "include": ["./**/*.ts"]
}

Now you can run the code with

npx tsx example.ts

You should expect output something like:

In college, the author studied subjects like linear algebra and physics, but did not find them particularly interesting. They started slacking off, skipping lectures, and eventually stopped attending classes altogether. They also had a negative experience with their English classes, where they were required to pay for catch-up training despite getting verbal approval to skip most of the classes. Ultimately, the author lost motivation for college due to their job as a software developer and stopped attending classes, only returning years later to pick up their papers.

0: Score: 0.8305309270895813 - I started this decade as a first-year college stud...

1: Score: 0.8286388215713089 - A short digression. I’m not saying colleges are wo...

Once you've mastered basic RAG, you may want to consider chatting with your data.

Retrieval Augmented Generation (RAG) Tutorial

Set up the project​

Run queries​

Set up the project

Run queries