AIMPACT News, May 17 (UTC+8), BerriAI announced the open source of LiteLLM Agent Platform, a Kubernetes-based self-hosted infrastructure platform designed for running multiple AI Agents in production environments, providing team-level sandbox isolation and persistent session management. The platform implements session-level independent sandbox environments through Kubernetes' agent-sandbox CRD, solving core issues such as state management, session persistence after container restarts, and environment isolation between teams. For local development, use kind (Kubernetes in Docker); for production, AWS EKS is recommended. Regarding the technology stack, the platform uses Next.js to build the management panel, with TypeScript accounting for 92.8%, and PostgreSQL for persistent storage. LiteLLM Agent Platform is built on top of LiteLLM Gateway, which handles model routing, cost tracking, and rate limiting. Local startup requires only two commands: bin/kind-up.sh to configure the kind cluster, and docker compose up to start the services. The project is licensed under the MIT License and is in alpha public preview.

View Original

This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.

10 Likes

Reward
10
7
Repost
Share

Comment

Add a comment

AirdropArchivist

· 7h ago

agent-sandbox CRD this design has some substance; cross-team isolation finally doesn't require reinventing the wheel.

View OriginalReply0

GasFeesForNightRuns

· 7h ago

Running two commands at the same time is definitely great, but the documentation doesn’t really spell out the performance bottleneck when it comes to session persistence.

View OriginalReply0

TheProphetOfToast

· 7h ago

With this combo of state management + restart recovery, long-running Agent services can finally take a break.

View OriginalReply0

Lemon-FlavoredLiquidation

· 7h ago

A multi-agent orchestration finally has a proper open-source implementation; previously, they were all toy projects.

View OriginalReply0

Front-RunningArbitrage

· 7h ago

For self-hosting, the Kubernetes barrier is right here. Is there a pure Docker Compose fallback solution?

View OriginalReply0

GateUser-991fc58a

· 7h ago

MIT License is conscientious, but is it brave enough to go live during the alpha stage?

View OriginalReply0

WatercolorGlassBottle

· 7h ago

Finally, a team has cracked the persistent storage of Agent state, and Kind's one-click startup is very friendly to developers.

View OriginalReply0

Trending Topics
View More
#
ShareYourUSStocksWinNvidia
3.29M Popularity
#
GatePartnersWithAlpacaToBridgeCryptoAndStocks
106.72K Popularity
#
BTCBottomAt66000
110.53K Popularity
#
ZECRalliesAgainstMarket
5.03M Popularity
#
WinGoldBarsWithGrowthPoints
1.29M Popularity

Pinned

Sitemap

BerriAI Open Source LiteLLM Agent Platform, supports deployment of multiple AI Agents in production environments

Trending Topics

ShareYourUSStocksWinNvidia

GatePartnersWithAlpacaToBridgeCryptoAndStocks

BTCBottomAt66000

ZECRalliesAgainstMarket

WinGoldBarsWithGrowthPoints

Pinned