NVIDIA NIM and LangChain Collaborate to Enhance AI Agent Development
James Ding Aug 07, 2024 14:43
NVIDIA NIM microservices now support tool-calling for models like Llama 3.1, integrating with LangChain to streamline AI agent development.
NVIDIA NIM, part of NVIDIA AI Enterprise, has expanded its capabilities by supporting tool-calling for models such as Llama 3.1 and integrating with LangChain, according to the NVIDIA Technical Blog. This collaboration aims to provide a robust solution for developing agentic workflows.
Building AI Agents with NVIDIA NIM
The Llama 3.1 NIM microservice is designed to facilitate the creation of generative AI applications with advanced functionalities suitable for production environments. Leveraging accelerated open models with state-of-the-art agentic capabilities, developers can now build more sophisticated and reliable AI applications.
NIM offers an OpenAI-compatible tool-calling API, ensuring consistency and ease of use. By integrating LangChain, developers can bind tools to NIM microservices, enabling the creation of structured outputs that enhance agent capabilities in applications. Tools accept structured output from a model, execute an action, and return results in a structured format back to the model, often involving external API calls.
For instance, a weather tool might retrieve the current weather in San Diego, while a web search tool could fetch the latest score of a San Francisco 49ers football game. To support tool usage in an agent workflow, a model must be trained to detect when to call a function and output a structured response like JSON with the function and its arguments. The model is then optimized as a NIM microservice for NVIDIA infrastructure, making it compatible with frameworks like LangChain’s LangGraph.
Using LangChain with NVIDIA NIM
LangChain provides a seamless experience for integrating with models like Llama 3.1 that support tool calling. Developers can install necessary packages and set up the ChatNVIDIA library by following the LangChain NVIDIA documentation. To get a list of models that support tool calling, the following command can be used:
from langchain_nvidia_ai_endpoints import ChatNVIDIA tool_models = [model for model in ChatNVIDIA.get_available_models() if model.supports_tools]
Developers can create custom functions or tools and bind them to models using LangChain’s bind_tools function. For example, a function to get the current weather for a specific location can be written as follows:
from langchain_core.pydantic_v1 import Field
from langchain_core.tools import tool
@tool
def get_current_weather(
location: str = Field(..., description="The location to get the weather for.")
):
"""Get the current weather for a location."""
...
llm = ChatNVIDIA(model=tool_models[0].id).bind_tools(tools=[get_current_weather])
response = llm.invoke("What is the weather in Boston?")
response.tool_calls
Developers can use APIs like the Tavily API for generic searches or the National Weather Service API within the get_current_weather function.
Explore More Resources
LangChain’s LangGraph integrates with NIM microservices, allowing the development of stateful, multi-actor applications for use cases such as customer support, coding assistants, and advanced retrieval-augmented generation (RAG) evaluation. Advanced RAG workflows can be built with LangGraph and NVIDIA NeMo Retriever using strategies like self-RAG and corrective RAG.
For more detailed examples and tutorials, developers can refer to the NVIDIA NIMs with Tool Calling for Agents example on GitHub and other LangGraph resources. This integration empowers developers to build cutting-edge AI applications, leveraging the strengths of both NVIDIA NIM microservices and LangChain.
Image source: Shutterstock.jpg)