intent-solutions-io/nixtla-cross-validator
003-skills/.claude/skills/nixtla-cross-validator/SKILL.md
Performs rigorous time series cross-validation using expanding and sliding windows. Use when needing to evaluate the performance of time series models on unseen data. Trigger with cross validate time series, evaluate forecasting model, time series backtesting.
$ install --global
skillsauthnpx skillsauth add intent-solutions-io/plugins-nixtla nixtla-cross-validatorInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 5, 2026, 11:13 PM8.5s4 files scanned
SKILL.md
- name:
- nixtla-cross-validator
- description:
- Performs rigorous time series cross-validation using expanding and sliding windows. Use when needing to evaluate the performance of time series models on unseen data. Trigger with cross validate time series, evaluate forecasting model, time series backtesting.
- version:
- 1.0.0
- author:
- Jeremy Longshore <[email protected]>
- license:
- MIT
- allowed-tools:
- Read,Write,Bash(python:*),Glob,Grep
Cross-Validator Skill
Evaluates time series model performance using cross-validation.
Purpose
Rigorously assesses how well a time series model generalizes to unseen data by simulating future predictions.
Overview
This skill automates time series cross-validation by splitting historical data into multiple training and validation sets based on expanding or sliding window techniques. It integrates with TimeGPT and StatsForecast to evaluate model performance across various time periods. It reports key accuracy metrics, helping users select the best model.
Prerequisites
Tools: Read, Write, Bash, Glob, Grep
Environment: NIXTLA_TIMEGPT_API_KEY (if using TimeGPT)
Packages:
pip install nixtla pandas statsforecast matplotlib
Instructions
Step 1: Prepare data
Read time series data from CSV file into a pandas DataFrame using the data loader script.
Script: {baseDir}/scripts/load_data.py
The script expects a CSV file with columns: unique_id, ds (timestamp), and y (target value).
Example usage:
python {baseDir}/scripts/load_data.py data.csv
Step 2: Configure cross-validation
Define parameters like window size, step size, and number of folds using the configuration script.
Script: {baseDir}/scripts/configure_cv.py
The script creates expanding window splits for cross-validation. It validates that the data is sufficient for the specified window size and number of folds.
Step 3: Execute cross-validation
Run the cross-validation script with your chosen model and parameters.
Script: {baseDir}/scripts/cross_validate.py
Usage:
python {baseDir}/scripts/cross_validate.py \
--input data.csv \
--model arima \
--window 20 \
--folds 3 \
--freq D
Supported models:
timegpt: TimeGPT API (requires NIXTLA_TIMEGPT_API_KEY)arima: AutoARIMA from StatsForecastets: AutoETS from StatsForecasttheta: AutoTheta from StatsForecastnaive: SeasonalNaive baseline
Step 4: Analyze results
The script automatically calculates and outputs cross-validation metrics (MAE, RMSE) for all folds.
Output
- cv_results.csv: CSV file containing the cross-validation results for each fold.
- metrics.json: JSON file containing overall performance metrics across all folds.
- plots/: Directory containing plots comparing actual vs. predicted values for each fold.
Error Handling
-
Error:
Input file not foundSolution: Ensure the specified input CSV file exists at the given path. -
Error:
Invalid model nameSolution: Use a supported model name: 'timegpt', 'arima', 'ets', 'theta', 'naive'. -
Error:
Insufficient data for cross-validationSolution: Increase the length of the input time series or reduce the window size. -
Error:
Missing required parameterSolution: Specify all required parameters: input, model, window, folds. -
Error:
NIXTLA_TIMEGPT_API_KEY environment variable not set.Solution: Set theNIXTLA_TIMEGPT_API_KEYenvironment variable before running the script when using TimeGPT.
Examples
Example 1: Cross-validating TimeGPT on daily sales
Input:
unique_id,ds,y
store_1,2023-01-01,10
store_1,2023-01-02,12
store_1,2023-01-03,15
...
store_1,2023-12-31,20
Command:
python {baseDir}/scripts/cross_validate.py \
--input sales.csv \
--model timegpt \
--window 30 \
--folds 4 \
--freq D
Output:
fold,unique_id,ds,y,y_hat
1,store_1,2023-11-01,18,17.5
1,store_1,2023-11-02,20,19.2
...
Example 2: Cross-validating ARIMA on monthly demand
Input:
unique_id,ds,y
product_1,2020-01-01,100
product_1,2020-02-01,110
...
product_1,2023-12-01,125
Command:
python {baseDir}/scripts/cross_validate.py \
--input demand.csv \
--model arima \
--window 6 \
--folds 3 \
--freq M
Output:
{
"MAE": 5.2,
"RMSE": 7.1
}
Resources
- StatsForecast documentation: https://nixtlaverse.nixtla.io/statsforecast/
- TimeGPT API documentation: https://docs.nixtla.io/
- Cross-validation best practices: https://otexts.com/fpp3/tscv.html
- Scripts:
{baseDir}/scripts/directory contains all executable code
Related Skills
intent-solutions-io/managing-test-environments
testing
VerifiedTrustedCommunity
This skill enables Claude to manage isolated test environments using Docker Compose, Testcontainers, and environment variables. It is used to create consistent, reproducible testing environments for software projects. Claude should use this skill when the user needs to set up a test environment with specific configurations, manage Docker Compose files for test infrastructure, set up programmatic container management with Testcontainers, manage environment variables for tests, or ensure cleanup after tests. Trigger terms include "test environment", "docker compose", "testcontainers", "environment variables", "isolated environment", "env-setup", and "test setup".
6SKILL.mdUpdated Jun 5, 2026
intent-solutions-io/generating-test-doubles
tools
VerifiedTrustedCommunity
This skill uses the test-doubles-generator plugin to automatically create mocks, stubs, spies, and fakes for unit testing. It analyzes dependencies in the code and generates appropriate test doubles based on the chosen testing framework, such as Jest, Sinon, or others. Use this skill when you need to generate test doubles, mocks, stubs, spies, or fakes to isolate units of code during testing. Trigger this skill by requesting test double generation or using the `/gen-doubles` or `/gd` command.
6SKILL.mdUpdated Jun 5, 2026
intent-solutions-io/generating-test-data
tools
VerifiedTrustedCommunity
This skill enables Claude to generate realistic test data for software development. It uses the test-data-generator plugin to create users, products, orders, and custom schemas for comprehensive testing. Use this skill when you need to populate databases, simulate user behavior, or create fixtures for automated tests. Trigger phrases include "generate test data", "create fake users", "populate database", "generate product data", "create test orders", or "generate data based on schema". This skill is especially useful for populating testing environments or creating sample data for demonstrations.
6SKILL.mdUpdated Jun 5, 2026
intent-solutions-io/analyzing-test-coverage
development
VerifiedTrustedCommunity
This skill analyzes code coverage metrics to identify untested code and generate comprehensive coverage reports. It is triggered when the user requests analysis of code coverage, identification of coverage gaps, or generation of coverage reports. The skill is best used to improve code quality by ensuring adequate test coverage and identifying areas for improvement. Use trigger terms like "analyze coverage", "code coverage report", "untested code", or the shortcut "cov".
6SKILL.mdUpdated Jun 5, 2026