LiteLLM


🚅 LiteLLM
DeveloperBerriAI
Release2023
Written inPython and TypeScript
TypeLLM gateway
LicenseMIT License
Websitewww.litellm.ai
Repositoryhttps://github.com/BerriAI/litellm

LiteLLM is an open-source large language model (LLM) gateway used in AI-assisted software development. It is currently developed by BerriAI, founded by Ishaan Jaffer and Krrish Dholakia. It provides a unified application programming interface (API) for accessing and routing requests to different AI model providers. LiteLLM can be deployed as a self-hosted proxy server or used as a Python library and SDK within applications.

Functionality

LiteLLM provides a common interface for models and services from providers including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Microsoft Azure, and Hugging Face. It translates requests and responses between provider APIs into a consistent OpenAI's completion()-based format, allowing applications to switch between models without separately implementing each provider's interface. LiteLLM also supports the Model Context Protocol (MCP), an open protocol developed by Anthropic for connecting AI applications to external tools and data sources.

The software also provides typical management features for online services, such as model routing, load balancing, fallbacks, authentication, rate limiting, usage monitoring, and cost controls. Its proxy server can act as a centralized API gateway through which multiple applications or users access AI models.

Security

In March 2026, malicious versions of LiteLLM were distributed through the Python Package Index (PyPI) as part of a software supply-chain attack. TechCrunch reported that AI recruiting company Mercor was among the organizations affected by the compromise.

In October 2026, another supply chain attack affected the release infrastructure of LiteLLM, allowing actors to publish malicious versions to PyPI and distribute compromised container images through Docker registries. After their discovery, the compromised versions were subsequently withdrawn.

See also

References

  1. ↑ "LiteLLM: Call every LLM API like it's OpenAI [100+ LLMs]". Y Combinator. Retrieved October 30, 2026.
  2. ↑ MSV, Janakiram (May 15, 2025). "LiteLLM: An open-source gateway for unified LLM access". InfoWorld. Retrieved October 30, 2026.
  3. 1 2 "LiteLLM - Getting Started". LiteLLM. BerriAI. Retrieved October 30, 2026.
  4. ↑ "Introducing the Model Context Protocol". Anthropic. November 25, 2025. Retrieved October 30, 2026.
  5. ↑ Singh, Jagmeet (March 31, 2026). "Mercor says it was hit by cyberattack tied to compromise of open source LiteLLM project". TechCrunch. Retrieved October 30, 2026.
  6. ↑ Read, Benjamin (October 21, 2026). "LiteLLM Compromised, Steals Credentials". Wiz. Retrieved October 29, 2026.
  7. ↑ "Contained: Oct 21, 2026 LiteLLM release v1.107.0 Compromised". LiteLLM. October 21, 2026. Retrieved October 30, 2026.