SpawnDev.AI

Run and serve local LLMs and image generation, on the desktop and in the browser.

License: MITnet10.0

SpawnDev.AI runs and serves local LLMs and image generation on the SpawnDev.ILGPU.ML inference engine (GGUF models, KV-cache decode, SD-Turbo). SpawnDev.AI.Server is an Ollama-compatible model server as a library that can be hosted over HTTP on the desktop or over a MessagePort in a browser shared worker. SpawnDev.AI holds the core contracts: chat types, tool-calling types, memory store and speaker registry.

Runs on

WebGPU (browser)CUDAOpenCLCPU (via SpawnDev.ILGPU.ML)

Highlights

  • Ollama native endpoints (/api/chat, /api/generate, /api/tags, /api/show), OpenAI /v1/chat/completions SSE and Anthropic Messages /v1/messages SSE.
  • OpenAI image generation endpoint (/v1/images/generations, SD-Turbo) and an MCP surface (/mcp).
  • One transport-free protocol router (AiApiRouter): host it over HTTP on desktop (drop-in on :11434) or over a MessagePort in a browser shared worker.
  • Server-side agentic tool loop with built-in tools; tool calls are parsed server-side into structured calls on every protocol surface.
  • Runs GGUF models such as Qwen3 in the browser on WebGPU and models such as Llama 3.1 on the desktop.
  • Live demo: a room of characters with persona, model, voice and body; chat runs in the browser on WebGPU and can drive a Reachy Mini robot (Hugging Face Space build only).
Program.cs
using SpawnDev.AI.Server;

var store = new OllamaModelStore();                    // ~/.ollama/models (or OLLAMA_MODELS)
var registry = new ModelRegistry(new OllamaCacheModelProvider(store), accelerator);  // any SpawnDev.ILGPU accelerator
var engine = new AiChatEngine(registry);
var router = new AiApiRouter(engine);
// host it: map every request to router.TryHandleAsync(method, path, bodyJson, yourTransport)

Install

dotnet add package SpawnDev.AI
dotnet add package SpawnDev.AI.Server

Building on SpawnDev.AI?

Private tickets, response targets and a direct line to the engineer who wrote it.

See support plans
An unhandled error has occurred. Reload x