ranbot-ai/huggingface-zerogpu

Hugging Face ZeroGPU

When to Use

Use this skill when you need aI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses @spaces.GPU, configuring python_version or requirements.txt for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process...

Rules and patterns for ML demos on Hugging Face Spaces with ZeroGPU hardware. Covers @spaces.GPU, duration and quota tuning, process isolation, the CUDA availability model, concurrency safety, and CUDA build constraints.

Scope

This skill is for Gradio SDK Spaces using ZeroGPU hardware. Docker and Static Spaces cannot schedule onto ZeroGPU, and Streamlit apps now run as Docker Spaces — so this skill applies only to Gradio. For general Gradio coding (components, layouts, event listeners), see the huggingface-gradio skill in this repo. The authoritative ZeroGPU docs live at https://huggingface.co/docs/hub/spaces-zerogpu — refer to them for the current backing GPU, runtime version lists, and tier thresholds, all of which change over time.

Reference Files

| Reference | When to read | |-----------|--------------| | references/concurrency.md | Always read alongside SKILL.md when writing ZeroGPU code — handlers run in parallel by default | | references/how-zerogpu-works.md | When reasoning about cold-starts, worker reuse, why module-scope warmup does not carry to requests, or why returning CUDA tensors hangs | | references/how-quota-works.md | When choosing duration values, debugging illegal duration vs quota exceeded errors, or explaining why default 60s blocks short tasks | | references/cuda-and-deps.md | When installing CUDA-dependent packages (e.g. flash-attn), pinning torch side-cars, or reading wheel filename tags |

Hardware

ZeroGPU exposes two GPU sizes that map to a fraction of the backing card:

| size | Slice of backing GPU | Quota cost | |--------|----------------------|------------| | large (default) | Half | 1x | | xlarge | Full | 2x |

Default large gives half a physical GPU, so memory bandwidth and compute are significantly lower than the full card's specs. Use xlarge only when the workload genuinely needs the extra memory or compute.

Backing GPU changes without notice. ZeroGPU has already migrated across GPU generations several times; older write-ups may name A100 or H200, but those are outdated. For the current backing GPU and exact per-size VRAM, always check the ZeroGPU docs before sizing workloads.

Basic Pattern

import spaces
import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="...", device="cuda")

@spaces.GPU
def generate(prompt: str) -> str:
    return pipe(prompt, max_new_tokens=100)[0]["generated_text"]

Key rules:

1. Instantiate models at module scope and call .to("cuda") eagerly. ZeroGPU handles the actual device mapping transparently (see CUDA availability model below).
2. Decorate GPU functions with @spaces.GPU. The decorator is a no-op outside ZeroGPU, so it is safe to keep in all environments.
3. Set duration to match the realistic worst-case workload (default 60s). The platform pre-checks requested duration against the user's remaining quota — not against the actual run time — so a 10-second task left at the 60s default fails with quota exceeded as soon as the user's remaining quota drops below 60s. Smaller declared duration also ranks higher in the node-level queue. See "Duration and Quota" below.
4. torch.compile is NOT supported. Use PyTorch ahead-of-time compilation (AoTI) (torch 2.8+) instead.
5. Use size="xlarge" sparingly. It allocates the full backing GPU, but costs 2x quota and tends to queue longer.

@spaces.GPU(duration=120)
def generate_image(prompt: str):
    return pipe(prompt).images[0]

CUDA Availability Model

Real GPU access is only available inside @spaces.GPU-decorated functions. Outside those functions, the GPU is not attached to the process.

However, import spaces monkey-patches torch so that:

• torch.cuda.is_available() returns True globally.
• .to("cuda") / device="cuda" calls at module scope succeed without error.

This is intentional. Module-scope model.to("cuda") calls register tensors with the ZeroGPU backend, which writes them to a disk offload directory at a startup "pack" step and frees the corresponding RAM. When a @spaces.GPU call lands, a forked GPU worker process streams those weights from disk into VRAM via a pinned-memory pipeline. Warm workers (reused across requests on the same GPU slot) keep weights resident on the GPU and skip the disk → VRAM step. The user-facing rule: write device="cuda" at module scope and it works — see references/how-zerogpu-works.md for the full lifecycle.

| Action | Where | Why |