Open source · a Canonical project

Make rented GPUs programmable.

OpenLease provisions GPUs, deploys open-source LLMs, reconciles them to a desired state, and serves inference over an OpenAI-compatible API. Drive it from a CLI, a REST API, or an MCP server, so an agent can bring up its own GPU and talk to the model it just deployed. RunPod is the first provider, not the platform.

RunPod gives you GPUs. OpenLease gives you a lifecycle: deploy, observe, recover, proxy, meter, tear down.

Get started View on GitHub

# deploy an open LLM on a GPU, wait for it, then chat
gpu deploy qwen3-0.6b --wait --chat

# or run it as a service
gpu up                       # daemon + OpenAI proxy, in the background
gpu deploy qwen3-32b         # non-blocking; the daemon drives it to READY
gpu costs                    # accrued + projected, per deployment

Conversational GPU deployment

Deploy GPUs like you talk to a teammate.

OpenLease ships an MCP server, so an agent can operate real GPUs on your behalf. Describe the workload, the model, the GPU, the budget, when to shut down, and OpenLease turns it into leases, deployments, health checks, an OpenAI-compatible endpoint, and safe teardown.

Spin up the cheapest available GPU that can run Qwen, and give me an OpenAI-compatible endpoint.
Process this batch, then shut the GPU down when the queue is empty.
Start an embeddings endpoint, and tear it down after an hour.
What is this costing me right now?

Because OpenLease owns the lifecycle, an agent cannot leave a pod billing forever: a terminal deployment keeps no instance, and an orphan sweep reaps anything that outlives its record. It is the safe execution layer between an agent and expensive infrastructure.

Built for ephemeral open-model inference on expensive batch jobs, the kind of workload Shaderdex (video indexing) throws at a GPU: burst up, run the queue, disappear. MCP turns OpenLease from a CLI into an agent-operable GPU control plane.

Why it is built this way

A deployment is a state machine, not a script

Every deployment carries a desired and an observed state. A reconcile loop closes the gap one step per tick, re-reading reality each time. Interruption and crash recovery are free: kill the process mid-provision, restart, and it resumes from wherever the pod actually is.

Cost-safety by construction

The scariest failure in GPU tooling is a pod that bills forever. A terminal deployment never keeps a running instance, a crash mid-create is recovered by adoption, and an orphan sweep reaps any pod with no live record. No pod outlives its purpose.

One core, many interfaces

The engine is the product. A CLI, a REST API, and an MCP server are each thin layers over one Orchestrator facade. Adding a provider is an ABC plus a dict entry, verified by a single contract suite. Adding an interface is parse, call, render.

Three ways in, one core

CLI

gpu

deploy, status, stop, costs, health, availability, chat, proxy, daemon. The whole lifecycle from the terminal.

REST API

gpu serve

Routes mirror the core 1:1, the OpenAI proxy mounted at /v1/*, bearer-token auth, auto-docs at /docs.

MCP server

gpu-mcp

Agent-facing tools over the same core, with confirm-gated destructive actions. Let an agent run the GPUs.

Quickstart

A RunPod key and a few commands.

pip install open-lease
export RUNPOD_API_KEY=...     # and HF_TOKEN for gated models

gpu models                   # the catalog
gpu availability qwen3-0.6b  # which data centers can run it right now
gpu deploy qwen3-0.6b --wait
gpu status                   # id, state, endpoint, uptime, accrued $
gpu stop dep-a1b2c3          # tear down; verified

First deploy of a model is download-bound: the image and weights are pulled onto an ephemeral disk. An opt-in model cache makes warm redeploys fast, and gpu status shows a percent or an elapsed-over-budget ETA so a cold start never looks stuck.

Where it stands

Beta, on PyPI, and validated on real RunPod hardware across the cases that actually bite: deploy to a served endpoint, kill-and-recover, crash-resume, orphan sweep, a runtime-crash-loop cap, and two models serving concurrently with correct proxy routing. It runs any vLLM-servable model on a provider-neutral architecture, with RunPod as the first provider.

Read the docs, or read the code.

Documentation GitHub