Extracting structured output

Overview

Large Language Models (LLMs) are emerging as an extremely capable technology for powering information extraction applications.

Classical solutions to information extraction rely on a combination of people, (many) hand-crafted rules (e.g., regular expressions), and custom fine-tuned ML models.

Such systems tend to get complex over time and become progressively more expensive to maintain and more difficult to enhance.

LLMs can be adapted quickly for specific extraction tasks just by providing appropriate instructions to them and appropriate reference examples.

This guide will show you how to use LLMs for extraction applications!

Approaches

There are 3 broad approaches for information extraction using LLMs:

• Tool/Function Calling Mode: Some LLMs support a tool or function calling mode. These LLMs can structure output according to a given schema. Generally, this approach is the easiest to work with and is expected to yield good results.

• JSON Mode: Some LLMs are can be forced to output valid JSON. This is similar to tool/function Calling approach, except that the schema is provided as part of the prompt. Generally, our intuition is that this performs worse than a tool/function calling approach, but don’t trust us and verify for your own use case!

• Prompting Based: LLMs that can follow instructions well can be instructed to generate text in a desired format. The generated text can be parsed downstream using existing Output Parsers or using custom parsers into a structured format like JSON. This approach can be used with LLMs that do not support JSON mode or tool/function calling modes. This approach is more broadly applicable, though may yield worse results than models that have been fine-tuned for extraction or function calling.

Quickstart

Head to the quickstart to see how to extract information using LLMs using a basic end-to-end example.

The quickstart focuses on information extraction using the tool/function calling approach.

How-To Guides

• Use Reference Examples: Learn how to use reference examples to improve performance.
• Handle Long Text: What should you do if the text does not fit into the context window of the LLM?
• Handle Files: Examples of using LangChain document loaders and parsers to extract from files like PDFs.
• Use a Parsing Approach: Use a prompt based approach to extract with models that do not support tool/function calling.

Guidelines

Head to the Guidelines page to see a list of opinionated guidelines on how to get the best performance for extraction use cases.

Use Case Accelerant

langchain-extract is a starter repo that implements a simple web server for information extraction from text and files using LLMs. It is build using FastAPI, LangChain and Postgresql. Feel free to adapt it to your own use cases.

Other Resources

• The output parser documentation includes various parser examples for specific types (e.g., lists, datetime, enum, etc).
• LangChain document loaders to load content from files. Please see list of integrations.
• The experimental Anthropic function calling support provides similar functionality to Anthropic chat models.
• LlamaCPP natively supports constrained decoding using custom grammars, making it easy to output structured content using local LLMs
• JSONFormer offers another way for structured decoding of a subset of the JSON Schema.
• Kor is another library for extraction where schema and examples can be provided to the LLM. Kor is optimized to work for a parsing approach.
• OpenAI’s function and tool calling
• For example, see OpenAI’s JSON mode.