Enhancing LLM Safety: Integrating LangChain Templates with NVIDIA NeMo Guardrails
Zach Anderson Jun 01, 2024 10:41
Learn how to build safer LLM apps using LangChain Templates and NVIDIA NeMo Guardrails.
Overview of LangChain Templates and NVIDIA NeMo Guardrails
According to the NVIDIA Technical Blog, LangChain Templates offer a new approach for creating, sharing, maintaining, and customizing LLM-based agents and chains. These templates are designed to help developers get to production faster with custom LLM use cases. By integrating NVIDIA NeMo Guardrails, enterprises can ensure content moderation, enhanced security, and evaluation of LLM responses.
Why Integrate Guardrails with LangChain Templates?
LangChain Templates provide developers with production-ready applications that can be tested for free through LangServe. They integrate seamlessly with FastAPI for building APIs with Python, adding speed and ease of use. As generative AI continues to evolve, implementing guardrails is crucial to ensure that LLMs used in enterprise applications remain accurate, secure, and contextually relevant. The NVIDIA NeMo Guardrails platform offers programmable rules and run-time integration to control both user input and LLM output.
For example, moderation of LLM inputs and outputs can be tailored based on the use case. If the data involves customer personal information, self-checking and fact-checking mechanisms can be put in place to safeguard responses.
Defining the Use Case
LLM guardrails help keep data secure and minimize hallucinations. NeMo Guardrails provide options such as input and output self-check rails for masking sensitive data or rephrasing user input. Dialogue rails influence how LLMs are prompted, while retrieval rails help mask sensitive data in RAG applications.
For instance, in a RAG use case, user input can be rephrased, and sensitive data can be removed from the LLM’s generated output using guardrails. An existing LangChain Template called nvidia-rag-canonical can be used as a starting point. This template comes with a prebuilt chatbot structure, making it easy to customize vector databases, LLM models, and prompt templates.
Downloading the LangChain Template
To download the LangChain Template:
- Install the LangChain CLI with
pip install -U langchain-cli. - Install the LangChain NVIDIA AI Foundation Endpoints package with
pip install -U langchain_nvidia_aiplay. - Create a new application and install the
nvidia-rag-canonicalpackage by runninglangchain app nvidia_rag_guardrails --package nvidia-rag-canonical.
The downloaded template sets up the ingestion pipeline into a Milvus vector database. Adding guardrails to this pipeline can help secure LLM responses.
Adding NeMo Guardrails
Before integrating guardrails into the downloaded template, developers should understand the basics of NeMo Guardrails. Refer to the example to learn how to create a simple guardrails configuration.
To add guardrails, create a directory named guardrails and a configuration file. Define dialogue flows and specify general topics that the LLM can respond to. Add self-check for user inputs and LLM outputs to avoid cybersecurity attacks like Prompt Injection.
Using the Template
The application project consists of an app and packages. The app is where LangServe code resides, and the package contains the chains and agents. To set up the server and interact with the API, add routes to the server.py file and spin up the LangServe instance with langchain serve.
Sample Input/Output:
"Question": "How many Americans receive Social Security Benefits?"
"Answer": "According to the Social Security Administration, about 65 million Americans receive Social Security benefits."Conclusion
Integrating NeMo Guardrails with LangChain Templates ensures secure and accurate LLM responses. This post detailed the steps for creating and implementing rails for user input and LLM output and setting up a LangChain server for API access. To learn more, visit the NeMo Guardrails documentation.
Image source: Shutterstock.jpg)