richfrem/hf-download

Dependencies

This skill requires Python 3.8+ and standard library only. No external packages needed.

To install this skill's dependencies:

pip-compile ./requirements.in
pip install -r ./requirements.txt

See ./requirements.txt for the dependency lockfile (currently empty — standard library only).

HuggingFace Download Primitives

Status: Active Author: Richard Fremmerlid Domain: HuggingFace Integration Depends on: hf-init (credentials must be configured first)

Purpose

Provides consolidated download operations for all HF-consuming plugins (Primary Agent, local-llm-bench, etc.) to fetch files, models, and snapshots. All downloads include exponential backoff for rate-limit handling.

Available Operations

| Function | Description | Source Repo | |---|---|---| | download_file() | Download a single file | Custom or default repo | | download_folder() | Download an entire folder snapshot | Custom or default repo |

Usage

From Python (as a library)

from hf_download import download_file, download_folder
from pathlib import Path

# Download a single file from dataset repository to local directory
local_file_path = await download_file(
    filename="data/soul_traces.jsonl",
    local_dir=Path("./local_data")
)

# Download a model snapshot (e.g. GGUF weights)
model_dir = await download_folder(
    local_dir=Path("./models"),
    repo_id="unsloth/gemma-4-12b-it-GGUF",
    repo_type="model",
    allow_patterns=["*UD-Q4_K_XL.gguf"]
)

From CLI

# Download a single file
python ./hf_download.py --filename data/soul_traces.jsonl --local-dir ./local_data

# Download a specific model snapshot using glob patterns
python ./hf_download.py \
  --repo-id unsloth/gemma-4-12b-it-GGUF \
  --repo-type model \
  --allow-patterns "*UD-Q4_K_XL.gguf" \
  --local-dir ./models

Prerequisites

1. Run hf-init first to validate credentials and dataset structure.
2. Requires huggingface_hub installed (pip install huggingface_hub).
3. Environment variables: HUGGING_FACE_USERNAME, HUGGING_FACE_TOKEN.

Error Handling

All operations return paths on success or raise appropriate exceptions with exponential backoff retries (up to 5 attempts) on rate limits or API connectivity issues.