SpawnDev.ILGPU.ML

Neural network inference and training in C#, on any GPU, in browser or desktop.

License: MITnet10.0

SpawnDev.ILGPU.ML is an inference and training engine for .NET in which every layer is a C# kernel compiled by ILGPU for the device in front of it. The same library, kernels and models run in the browser (Blazor WebAssembly on WebGPU, WebGL or WebAssembly) and on the desktop (CUDA, OpenCL or a SIMD CPU backend). It has no ONNX Runtime dependency, no native inference binaries and no server.

Runs on

WebGPUWebGLWasmCUDAOpenCLVelocity (SIMD CPU)CPU

Highlights

  • Seven model formats auto-detected from the file: ONNX, TFLite, GGUF, SafeTensors, TensorFlow GraphDef, PyTorch checkpoints and Core ML, lowered to one graph IR.
  • 200+ ONNX operators, a graph optimizer (constant folding, fusion, strength reduction, dead-node elimination) and fused kernels for linear layers, attention and quantized matmul.
  • Generative models ported and checked against the reference implementations: FLUX 3 Image Klein, FLUX.2 klein, Stable Diffusion 1.x / 2.x / SDXL, SD-Turbo, LTX-Video, Wan 2.2, TRELLIS.2 (text and image to textured 3D), SkinTokens and UniRig (rigging), Kimodo (animation).
  • Language and speech: quantized GGUF LLMs with KV-cache decode and TurboQuant cache compression, Whisper speech-to-text, Kokoro and ZipVoice text-to-speech with voice cloning.
  • Training, not only inference: LoRA fine-tuning of GGUF language models on the device, plus backpropagation, SGD and Adam kernels for small networks.
  • Zero-copy by design: pixels, activations and weights stay on the GPU; in the browser, bulk data streams from fetch and OPFS straight into GPU buffers.
  • Every demo and model is tracked as verified, partial or work in progress in Docs/DEMO_AND_MODEL_STATUS.md.
Program.cs
using SpawnDev.ILGPU.ML;
using SpawnDev.ILGPU.ML.Tensors;

using var session = InferenceSession.CreateFromFile(accelerator, File.ReadAllBytes("squeezenet.onnx"));

// Inputs are named tensors; outputs come back as a disposable map of GPU-resident tensors you own.
using var input = OwnedTensor<float>.FromHost(accelerator, pixels, new[] { 1, 3, 224, 224 });
using var outputs = await session.RunOwnedAsync(new Dictionary<string, Tensor<float>>
{
    [session.InputNames[0]] = input,
});

float[] logits = await outputs[session.OutputNames[0]].ToHostAsync();

Install

dotnet add package SpawnDev.ILGPU.ML
dotnet add package SpawnDev.ILGPU.ML.WebTorrent

Building on SpawnDev.ILGPU.ML?

Private tickets, response targets and a direct line to the engineer who wrote it.

See support plans
An unhandled error has occurred. Reload x