OpenAI Embeddings & Vector Search

Learn how to build a ChatGPT-style doc search powered by Next.js, OpenAI, and Supabase.

In this tutorial we'll look at how you can build a custom ChatGPT-like search experience for your own knowledge base. See our Supabase Clippy blog post for an example of how this will look.

We assume that you have a Next.js project with a collection of .mdx files nested inside your pages directory. We will start developing locally with the Supabase CLI and then push our local database changes to our hosted Supabase project.

Create a project#

1. Create a new project in the Supabase Dashboard.
2. Enter your project details.
3. Wait for the new database to launch.

Prepare the database#

Let's prepare the database schema. We can use the "OpenAI Vector Search" quickstart in the SQL Editor, or you can copy/paste the SQL below and run it yourself.

Pre-process the knowledge base at build time#

With our database set up, we need to process and store all .mdx files in the pages directory. You can find the full script here, or follow the steps below:

1curl \
2https://raw.githubusercontent.com/supabase-community/nextjs-openai-doc-search/main/lib/generate-embeddings.ts \
3-o "lib/generate-embeddings.ts"

NEXT_PUBLIC_SUPABASE_URL=
NEXT_PUBLIC_SUPABASE_ANON_KEY=
SUPABASE_SERVICE_ROLE_KEY=

# Get your key at https://platform.openai.com/account/api-keys
OPENAI_KEY=

"scripts": {
  "dev": "next dev",
  "build": "pnpm run embeddings && next build",
  "start": "next start",
  "embeddings": "tsx lib/generate-embeddings.ts"
},

Create Text Completion with OpenAI API#

Anytime a user asks a question, we need to create an embedding for their question, perform a similarity search, and then send a text completion request to the OpenAI API with the query and then context content merged together into a prompt.

All of this is glued together in a Vercel Edge Function, the code for which can be found on GitHub.

const embeddingResponse = await fetch('https://api.openai.com/v1/embeddings', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${openAiKey}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'text-embedding-ada-002',
    input: sanitizedQuery.replaceAll('\n', ' '),
  }),
})

if (embeddingResponse.status !== 200) {
  throw new ApplicationError('Failed to create embedding for question', embeddingResponse)
}

const {
  data: [{ embedding }],
} = await embeddingResponse.json()

const { error: matchError, data: pageSections } = await supabaseClient.rpc(
  'match_page_sections',
  {
    embedding,
    match_threshold: 0.78,
    match_count: 10,
    min_content_length: 50,
  }
)

const prompt = codeBlock`
  ${oneLine`
    You are a very enthusiastic Supabase representative who loves
    to help people! Given the following sections from the Supabase
    documentation, answer the question using only that information,
    outputted in markdown format. If you are unsure and the answer
    is not explicitly written in the documentation, say
    "Sorry, I don't know how to help with that."
  `}

  Context sections:
  ${contextText}

  Question: """
  ${sanitizedQuery}
  """

  Answer as markdown (including related code snippets if available):
`

const completionOptions: CreateCompletionRequest = {
  model: 'text-davinci-003',
  prompt,
  max_tokens: 512,
  temperature: 0,
  stream: true,
}

const response = await fetch('https://api.openai.com/v1/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${openAiKey}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(completionOptions),
})

if (!response.ok) {
  const error = await response.json()
  throw new ApplicationError('Failed to generate completion', error)
}

// Proxy the streamed SSE response from OpenAI
return new Response(response.body, {
  headers: {
    'Content-Type': 'text/event-stream',
  },
})

Display the answer on the frontend#

In a last step, we need to process the event stream from the OpenAI API and print the answer to the user. The full code for this can be found on GitHub.

const handleConfirm = React.useCallback(
  async (query: string) => {
    setAnswer(undefined)
    setQuestion(query)
    setSearch('')
    dispatchPromptData({ index: promptIndex, answer: undefined, query })
    setHasError(false)
    setIsLoading(true)

    const eventSource = new SSE(`api/vector-search`, {
      headers: {
        apikey: process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY ?? '',
        Authorization: `Bearer ${process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY}`,
        'Content-Type': 'application/json',
      },
      payload: JSON.stringify({ query }),
    })

    function handleError<T>(err: T) {
      setIsLoading(false)
      setHasError(true)
      console.error(err)
    }

    eventSource.addEventListener('error', handleError)
    eventSource.addEventListener('message', (e: any) => {
      try {
        setIsLoading(false)

        if (e.data === '[DONE]') {
          setPromptIndex((x) => {
            return x + 1
          })
          return
        }

        const completionResponse: CreateCompletionResponse = JSON.parse(e.data)
        const text = completionResponse.choices[0].text

        setAnswer((answer) => {
          const currentAnswer = answer ?? ''

          dispatchPromptData({
            index: promptIndex,
            answer: currentAnswer + text,
          })

          return (answer ?? '') + text
        })
      } catch (err) {
        handleError(err)
      }
    })

    eventSource.stream()

    eventSourceRef.current = eventSource

    setIsLoading(true)
  },
  [promptIndex, promptData]
)

Learn More#

Want to learn more about the awesome tech that is powering this?

• Read about how we built ChatGPT for the Supabase Docs.
• Read the pgvector Docs for Embeddings and vector similarity
• Watch Greg's video for a full breakdown: