huggingface/huggingface-local-models
skills/huggingface-local-models/SKILL.md
Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.
$ install --global
skillsauthnpx skillsauth add huggingface/skills huggingface-local-modelsInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 23, 2026, 6:46 AM37.5s4 files scanned
SKILL.md
- name:
- huggingface-local-models
- description:
- Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.
Hugging Face Local Models
Search the Hugging Face Hub for llama.cpp-compatible GGUF repos, choose the right quant, and launch the model with llama-cli or llama-server.
Default Workflow
- Search the Hub with
apps=llama.cpp. - Open
https://huggingface.co/<repo>?local-app=llama.cpp. - Prefer the exact HF local-app snippet and quant recommendation when it is visible.
- Confirm exact
.gguffilenames withhttps://huggingface.co/api/models/<repo>/tree/main?recursive=true. - Launch with
llama-cli -hf <repo>:<QUANT>orllama-server -hf <repo>:<QUANT>. - Fall back to
--hf-repoplus--hf-filewhen the repo uses custom file naming. - Convert from Transformers weights only if the repo does not already expose GGUF files.
Quick Start
Install llama.cpp
brew install llama.cpp
winget install llama.cpp
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
make
Authenticate for gated repos
hf auth login
Search the Hub
https://huggingface.co/models?apps=llama.cpp&sort=trending
https://huggingface.co/models?search=Qwen3.6&apps=llama.cpp&sort=trending
https://huggingface.co/models?search=<term>&apps=llama.cpp&num_parameters=min:0,max:24B&sort=trending
Run directly from the Hub
llama-cli -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
Run an exact GGUF file
llama-server \
--hf-repo unsloth/Qwen3.6-35B-A3B-GGUF \
--hf-file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
-c 4096
Convert only when no GGUF is available
hf download <repo-without-gguf> --local-dir ./model-src
python convert_hf_to_gguf.py ./model-src \
--outfile model-f16.gguf \
--outtype f16
llama-quantize model-f16.gguf model-q4_k_m.gguf Q4_K_M
Smoke test a local server
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key" \
-d '{
"messages": [
{"role": "user", "content": "Write a limerick about exception handling"}
]
}'
Quant Choice
- Prefer the exact quant that HF marks as compatible on the
?local-app=llama.cpppage. - Keep repo-native labels such as
UD-Q4_K_Minstead of normalizing them. - Default to
Q4_K_Munless the repo page or hardware profile suggests otherwise. - Prefer
Q5_K_MorQ6_Kfor code or technical workloads when memory allows. - Consider
Q3_K_M,Q4_K_S, or repo-specificIQ/UD-*variants for tighter RAM or VRAM budgets. - Treat
mmproj-*.gguffiles as projector weights, not the main checkpoint.
Load References
- Read hub-discovery.md for URL-first workflows, model search, tree API extraction, and command reconstruction.
- Read quantization.md for format tables, model scaling, quality tradeoffs, and
imatrix. - Read hardware.md for Metal, CUDA, ROCm, or CPU build and acceleration details.
Resources
- llama.cpp:
https://github.com/ggml-org/llama.cpp - Hugging Face GGUF + llama.cpp docs:
https://huggingface.co/docs/hub/gguf-llamacpp - Hugging Face Local Apps docs:
https://huggingface.co/docs/hub/main/local-apps - Hugging Face Local Agents docs:
https://huggingface.co/docs/hub/agents-local - GGUF converter Space:
https://huggingface.co/spaces/ggml-org/gguf-my-repo
Related Skills
huggingface/hf-cli
tools
VerifiedTrustedOfficial
Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`.
10,588SKILL.mdUpdated Mar 15, 2026
huggingface/huggingface-zerogpu
development
VerifiedTrustedOfficial
AI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses `@spaces.GPU`, configuring `python_version` or `requirements.txt` for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process isolation, `gr.State` semantics across the worker boundary, no `torch.compile` (use AoTI instead), CUDA wheel-only builds (no `nvcc` at build or runtime), large vs xlarge sizing, and dynamic duration callables. Make sure to use this skill whenever the user mentions ZeroGPU, `@spaces.GPU`, or the `spaces` Python package, or hits ZeroGPU-specific code errors like `PicklingError` across the worker boundary, `illegal duration`, or `flash-attn` wheel-build failures — even when the user does not explicitly ask for ZeroGPU coding guidance. Trigger on `import spaces` or `@spaces.GPU` in code.
10,536SKILL.mdUpdated May 21, 2026
huggingface/train-sentence-transformers
development
VerifiedTrustedOfficial
Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder; dense or static embedding model; for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker; pair scoring for two-stage retrieval / pair classification), and `SparseEncoder` (SPLADE, sparse embedding model; for learned-sparse retrieval). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.
10,411SKILL.mdUpdated May 8, 2026
huggingface/huggingface-datasets
development
VerifiedTrustedOfficial
Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.
10,367SKILL.mdUpdated Mar 27, 2026