Introduction

I’m going to cover how to run garak, the AI red-teaming and vulnerability scanner against a LiteLLM proxy using a virtual key, AI model, and a probe selection on AWS. LiteLLM is an open-source library and proxy server that allows users to have a unified interface to call over 100+ Large Language Model (LLM) providers e.g., OpenAI, Anthropic, Azure, Ollama etc) using a single standardized OpenAI format. With these capabilities, it is important to test and secure your models before deployment into production systems. This article is geared towards developers and security engineers wanting to secure their LiteLLM deployment before going to production.

Prerequisites

The following are required to setup and run scans discussed in this blog:

1. LiteLLM Proxy Server: The example in this blog uses a proxy server deployed on AWS Elastic Container Services(ECS). To set this up, I recommend going through the LiteLLM GitHub repo for instructions here for your own particular environment or preference: https://github.com/BerriAI/litellm – you can also go through the LiteLLM docs here for setup on any platform: https://docs.litellm.ai/docs/ (Make sure to enable the AI/LLM models you will be using for testing)
2. EC2 with garak installed: We are going to run garak against a LiteLLM proxy server inside an AWS Virtual Private Cloud (VPC). Installation is straight forward. Garak is developed, backed and supported by the NVIDIA AI Red Team to help identify security flaws on LLMs such as hallucinations, data leakage and prompt injections. For more information on the tool, check out their GitHub repo here: https://github.com/NVIDIA/garak 
3. LiteLLM Virtual Keys and Model: We will need virtual keys from a user or team, and an LLM model that is enabled for that user/team.

Garak Installation on EC2

First create a virtual environment to install garak and its dependencies(the following commands and screenshots use python 3.11, you can use any compatible version of python for garak):

python3.11 -m venv garak_scan
source garak_scan/bin/activate
python3.11 -m pip install -U garak

Once installation completes, check the version:

python3.11 -m garak –version

LiteLLM Virtual Keys and Model Selection

Next, we need to access the LiteLLM proxy server, provision virtual keys and also assign a model to a user for testing.

On the LiteLLM server, go the “Virtual Keys” and click on “Create New Key”.

Add a “Key Name” and select a model to assign to the virtual/API key. We will be using this model to test. You can assign more models to test, but for simplicity on this guide, we’ll be selecting online one to test. In this case we’re using “model_name”

Running Garak Scans on LiteLLM 

After acquiring the virtual keys and assigning a model, we will need to export the LiteLLM API key and proxy server URL. Run the following commands on your EC2 terminal:

export LITELLM_PROXY_API_KEY="sk-replace_with_your_API_key_here"
export LITELLM_BASE_URL="https://replace_with_proxy_URL.com/v1/"

Then do a quick scan with the following(in the target_name tag, enter the model you had selected earlier ):

garak --probes promptinject.HijackHateHumans --generations 1 --target_type litellm --target_name bedrock/mistral.mistral-small-2402-v1:0 

After a few minutes, you should get the results on the terminal along with the location of the scan report. The report should be available in HTML or JSON format for your review.

Conclusion

Running these scans on a regular basis allows you to move beyond simple performance metrics and actively identify critical vulnerabilities like prompt injection, hallucinations, and data leakage before they impact your users. As you move toward production, consider making these vulnerability scans a standard part of your CI/CD pipeline or scheduled security audits. Every vulnerability garak identifies is an opportunity to harden your system against attacks and also improve your guardrails.

– Z333RO