OpenAI Tool Agent

​
Node Details

  • Name: openAIToolAgentLlamaIndex
  • Type: OpenAIToolAgent
  • Category: [[Agents]]
  • Version: 2.0

​
Description

This agent uses OpenAI Function Calling to pick and execute appropriate tools based on user input. It’s particularly adept at understanding complex queries, breaking them down into actionable steps, and utilizing the provided tools to generate comprehensive responses.

​
Parameters

  1. Tools (Required)
    • Type: Tool_LlamaIndex[]
    • Description: A list of tools that the agent can use to perform tasks or gather information.
  2. Memory (Required)
    • Type: BaseChatMemory
    • Description: The memory component used to store and retrieve conversation history.
  3. OpenAI/Azure Chat Model (Required)
    • Type: BaseChatModel_LlamaIndex
    • Description: The OpenAI or Azure OpenAI model used for generating responses and making decisions.
  4. System Message (Optional)
    • Type: string
    • Description: A custom system message to set the behavior of the agent.
    • Additional Params: true
    • Rows: 4

​
Input

  • A string containing the user’s message or query.

​
Output

  • Either a string containing the agent’s response, or an object with:
    • text: The agent’s response
    • usedTools: An array of tools used by the agent, including tool names, inputs, and outputs

​
How It Works

  1. The agent is initialized with the provided tools, OpenAI/Azure model, memory, and optional system message.
  2. It receives a user input.
  3. The agent then:
    • Retrieves relevant context from its memory
    • Analyzes the input and context to determine the next action
    • Uses OpenAI’s function calling capability to select and execute appropriate tools
    • Generates a response using the OpenAI/Azure model
  4. The response and any used tools are returned.
  5. The conversation history is updated in the memory for future context.

​
Use Cases

  • Complex query resolution requiring multiple steps or tool usage
  • Task automation with natural language input
  • Interactive problem-solving systems
  • Research and analysis assistants
  • Customizable AI agents for specific domains or tasks

​
Special Features

  • LlamaIndex Integration: Designed to work seamlessly within the LlamaIndex framework
  • Function Calling: Utilizes OpenAI’s function calling for precise tool selection and execution
  • Memory Management: Maintains conversation history for contextual understanding
  • Flexible Tool Usage: Can work with a variety of LlamaIndex-compatible tools
  • Debug Mode: Supports verbose output for debugging when DEBUG=true environment variable is set
  • Streaming Support: Can provide streaming responses for real-time interaction

​
Notes

  • The agent supports both streaming and non-streaming response modes.
  • It can handle complex, multi-turn conversations and maintain context effectively.
  • The system message can be customized to tailor the agent’s behavior and capabilities.
  • Compatible with both OpenAI and Azure OpenAI models.
  • Error handling is implemented to provide meaningful feedback if tool execution fails.
The OpenAI Tool Agent node provides a sophisticated solution for building AI systems that require both advanced language understanding and the ability to execute specific tools or actions. Its integration with LlamaIndex and use of OpenAI’s function calling makes it particularly suitable for applications that need to process complex queries, perform multi-step tasks, or provide detailed, context-aware responses in specialized domains.