AIBackends

Play the video in the post below

AIBackends is a self-hosted API server that puts one HTTP layer in front of several AI providers: local models through Ollama, LM Studio or LlamaCpp, or cloud calls to OpenRouter, OpenAI, Anthropic, Google AI Studio, Baseten and ZAI. It ships ready-made endpoints for text, vision and data tasks, plus autonomous tool-using agents that plan across multiple turns instead of a single prompt and response, with chat sessions that expire after 30 idle minutes.

Jev gets its own endpoint, /api/evaluate. A request sends a shared state plus a map of typed questions (choice, score or noul) and gets back one answer per question from the jev-latest model. Evaluation providers are kept apart from the generative ones: typesafe isn’t accepted by text endpoints such as /api/summarize, and LLM providers aren’t accepted by /api/evaluate. Upstream 429 and 529 responses are retried with backoff, and Jev can also be reached through Vercel AI Gateway. A Jev Playground page ships with presets for agent routing, support ticket triage, prompt injection guarding, PR risk review and invoice compliance.

The project targets non-coders adding AI features through app builders like Bolt.new, v0 and Lovable, as well as coding tools such as Cursor, Windsurf and Claude Code. Since the endpoints are ready to use, Vivas says you can prompt against the API documentation instead of writing prompts yourself. It’s self-hosted, tested by the author on Railway.

How it uses Jev

 * are retried with exponential backoff, mirroring the official SDK.
 */

export const TYPESAFE_DEFAULT_MODEL = 'jev-latest';
export const TYPESAFE_SYSTEMONE_PATH = '/v1/systemone';

const RETRYABLE_STATUSES = new Set([429, 529]);

const upstreamResponseSchema = z

View in src/services/typesafe.ts

Open on GitHub