Skip to main content

Running Agent as an Iterator

It can be useful to run the agent as an iterator, to add human-in-the-loop checks as needed.

To demonstrate the AgentExecutorIterator functionality, we will set up a problem where an Agent must:

  • Retrieve three prime numbers from a Tool
  • Multiply these together.

In this simple problem we can demonstrate adding some logic to verify intermediate steps by checking whether their outputs are prime.

from langchain.agents import AgentType, initialize_agent
from langchain.chains import LLMMathChain
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.tools import Tool
from langchain_openai import ChatOpenAI
%pip install --upgrade --quiet  numexpr
# need to use GPT-4 here as GPT-3.5 does not understand, however hard you insist, that
# it should use the calculator to perform the final calculation
llm = ChatOpenAI(temperature=0, model="gpt-4")
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)

Define tools which provide:

  • The nth prime number (using a small subset for this example)

  • The LLMMathChain to act as a calculator

primes = {998: 7901, 999: 7907, 1000: 7919}


class CalculatorInput(BaseModel):
question: str = Field()


class PrimeInput(BaseModel):
n: int = Field()


def is_prime(n: int) -> bool:
if n <= 1 or (n % 2 == 0 and n > 2):
return False
for i in range(3, int(n**0.5) + 1, 2):
if n % i == 0:
return False
return True


def get_prime(n: int, primes: dict = primes) -> str:
return str(primes.get(int(n)))


async def aget_prime(n: int, primes: dict = primes) -> str:
return str(primes.get(int(n)))


tools = [
Tool(
name="GetPrime",
func=get_prime,
description="A tool that returns the `n`th prime number",
args_schema=PrimeInput,
coroutine=aget_prime,
),
Tool.from_function(
func=llm_math_chain.run,
name="Calculator",
description="Useful for when you need to compute mathematical expressions",
args_schema=CalculatorInput,
coroutine=llm_math_chain.arun,
),
]

Construct the agent. We will use OpenAI Functions agent here.

from langchain import hub

# Get the prompt to use - you can modify this!
# You can see the full prompt used at: https://smith.langchain.com/hub/hwchase17/openai-functions-agent
prompt = hub.pull("hwchase17/openai-functions-agent")
from langchain.agents import create_openai_functions_agent

agent = create_openai_functions_agent(llm, tools, prompt)
from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

API Reference:

Run the iteration and perform a custom check on certain steps:

question = "What is the product of the 998th, 999th and 1000th prime numbers?"

for step in agent_executor.iter({"input": question}):
if output := step.get("intermediate_step"):
action, value = output[0]
if action.tool == "GetPrime":
print(f"Checking whether {value} is prime...")
assert is_prime(int(value))
# Ask user if they want to continue
_continue = input("Should the agent continue (Y/n)?:\n") or "Y"
if _continue.lower() != "y":
break


> Entering new AgentExecutor chain...

Invoking: `GetPrime` with `{'n': 998}`


7901Checking whether 7901 is prime...
Should the agent continue (Y/n)?:
y

Invoking: `GetPrime` with `{'n': 999}`


7907Checking whether 7907 is prime...
Should the agent continue (Y/n)?:
y

Invoking: `GetPrime` with `{'n': 1000}`


7919Checking whether 7919 is prime...
Should the agent continue (Y/n)?:
y

Invoking: `Calculator` with `{'question': '7901 * 7907 * 7919'}`




> Entering new LLMMathChain chain...
7901 * 7907 * 7919```text
7901 * 7907 * 7919
```
...numexpr.evaluate("7901 * 7907 * 7919")...

Answer: 494725326233
> Finished chain.
Answer: 494725326233Should the agent continue (Y/n)?:
y
The product of the 998th, 999th and 1000th prime numbers is 494,725,326,233.

> Finished chain.

Help us out by providing feedback on this documentation page: