Docs
LLM output hook

LLM output hook

Installation

pnpm add @llm-ui/react

Usage

Example:

import { useLLMOutput } from "@llm-ui/react";

const { blockMatches } = useLLMOutput({
  llmOutput: "llm output",
  blocks: [],
  fallbackBlock: {
    // fallback block here
  },
  isStreamFinished: false,
});

useLLMOutput parameters

llmOutput (string)

The output from a single LLM chat response.

You can pass either:

  • A fully completed chat response
  • An in-progress streamed chat response, as you receive more tokens from the LLM

isStreamFinished (boolean)

Whether the LLM has finished streaming the chat response.

isStreamFinished can be used by throttle functions to determine how far to lag behind the LLM output.

blocks (object[])

An array of the blocks to try and match with the LLM output. The highest priority is given to the first block in the array.

For built-in blocks you’ll usually get the block config from @llm-ui/markdown, @llm-ui/code etc.

For your own custom blocks, please see the custom-blocks docs for more details.

const block = {
  // The component to render when the block is matched.
  // Block match contains information about the match.
  component: ({ blockMatch }) => <div>{blockMatch.block.match}</div>,

  // The logic to find a 'complete' match:
  // when a block starts and ends inside of the `llmOutput` string.
  findCompleteMatch: (llmOutput: string) => ({
    startIndex: 1, // the index where the match starts
    endIndex: 10, // the index where the match ends
    outputRaw: "some llm output" // the raw string output of the match
  }),

  // The logic to find a 'partial' match:
  // when a block starts but does not end inside of the `llmOutput` string.
  findPartialMatch: (llmOutput: string) => ({
    startIndex, // the index where the match starts
    endIndex, // the index where the match ends
    outputRaw // the raw string output of the match
  }),

  // A lookback function to look backwards in the LLM output for smooth rendering.
  lookBack: ({
    output: '【{type:"buttons",buttons:[{text:"my bu"',
    isComplete: false, // whether the match is complete or partial
    visibleTextLengthTarget: 10, // the number of visible characters to return
    isStreamFinished: true // whether the LLM has finished streaming
  }) => {
    return {
      // The llm output to return. In some cases this will be the original.
      // In other cases this function may modify the output to make it 'complete'.
      output: '{type:"buttons",buttons:[{text:"my bu"}]}'
      visibleText: "my bu" // the visible text the user will actually see
    }
  }
}

fallbackBlock (object)

The fallback block is the block to use when no other block matches. This is usually the markdown block.

const fallbackBlock = {
  // The component to render when the fallback block is matched.
  // Block match contains information about the match.
  component: ({ blockMatch }) => <div>{blockMatch.block.match}</div>,

  // A lookback function to look backwards in the LLM output for smooth rendering.
  lookBack: ({
    output: "## header 1",
    isComplete: false, // whether the match is complete or partial
    visibleTextLengthTarget: 10, // the number of visible characters to return
    isStreamFinished: true // whether the LLM has finished streaming
  }) => {
    return {
      // The llm output to return. In some cases this will be the original.
      // In other cases this function may modify the output to make it 'complete'.
      output: "## header 1"
      visibleText: "header 1" // the visible text the user will actually see
    }
  }
}

throttle (function) (optional)

Defaults to throttleBasic.

The throttle function determines how useLLMOutput lags behind the actual LLM output.

Read more about throttle functions in the throttle docs.

useLLMOutput return value

isFinished (boolean)

Returns false when the response is still streaming, true once the last character has been streamed and the parameter isStreamFinished is true.

visibleText (string)

The visible text that the user will see. Calculated by joining each block’s lookBack function.

blockMatches (object)

The block matches found in the llmOutput parameter.

const { blockMatches } = useLLMOutput({
  llmOutput: output,
  blocks: [],
  fallbackBlock,
  isStreamFinished,
});

console.log(blockMatches);
// =>
[{
  block: {
    // The component to render
    component: ...,
    // The block's lookBack function
    lookBack: ...,
  };

  // The LLM output for the block
  // (possibly modified by the lookback function)
  output: '{type:"buttons",buttons:[{text:"my bu"}]}',

  // The unmodified LLM output for the block
  outputRaw: '【{type:"buttons",buttons:[{text:"my bu"',

  // The visible text for the block
  visibleText: "my bu",

  // The full LLM output (for all blocks)
  llmOutput: "...",

  // The start index of the block match in the `llmOutput`
  startIndex: 0,

  // The end index of the block match in the `llmOutput`
  endIndex: 21,

  // Is the block complete or partial
  isComplete: true,

  // priority (index in the `blocks` parameter array)
  priority: 2
}]

finishedCount (number)

How many times the output has finished. Only needed for looping examples.

Examples Throttle functions