pablodiegoo/data-manipulation
src/datapro/data/skills/data-manipulation/SKILL.md
High-performance data manipulation and transformation using Pandas, Numpy, and DuckDB. Use when Claude needs to: (1) Clean or transform structured data (CSV, Parquet, JSON), (2) Perform large-scale aggregations or analytics, (3) Optimize analysis for performance and memory, (4) Implement the 'Tidy Data' (Wide-to-Long) strategy for reporting.
$ install --global
skillsauthnpx skillsauth add pablodiegoo/data-pro-skill data-manipulationInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: May 26, 2026, 6:04 AM5.6s13 files scanned
SKILL.md
- name:
- data-manipulation
- description:
- High-performance data manipulation and transformation using Pandas, Numpy, and DuckDB. Use when Claude needs to: (1) Clean or transform structured data (CSV, Parquet, JSON), (2) Perform large-scale aggregations or analytics, (3) Optimize analysis for performance and memory, (4) Implement the 'Tidy Data' (Wide-to-Long) strategy for reporting.
Data Manipulation
High-performance data manipulation and transformation suite using Pandas, Numpy, and DuckDB. This skill handles the "T-Layer" (Transformation) of the data pipeline, preparing raw data for statistical analysis or reporting.
1. Core Capabilities
A. Dictionary Mapping & Cleaning
Standardizes raw, cryptic variables into semantic labels using external mapping dictionaries.
- Script:
scripts/dict_mapper.py - Reference: pipeline.md
B. High-Performance Ingestion & Aggregation (DuckDB Track)
Provides extremely fast ingestion and fuzzy cleaning for messy local files or files > 1GB.
- Scripts:
scripts/duckdb_fuzzy_cleaner.py,scripts/quant_analyzer_duckdb.py - Reference: duckdb_analytics.md
C. Sample Weighting
Calculates and applies expansion weights for representative survey analysis.
- Script:
scripts/weighting.py
D. Data Organization
Utilities for managing project directory structures and data discovery.
- Script:
scripts/data_directory_finder.py
2. Technical Guidelines
- Efficiency: Use Pandas Patterns (vectorization, categorical dtypes) for datasets under 1GB.
- Robustness: Use NumPy Stats for low-level numerical transformations.
- Hierarchy: Always prefer Parquet for intermediate data storage to preserve data types.
3. Reference Decision Matrix
| Task | Recommended Tool | Pattern Reference |
| :--- | :--- | :--- |
| Join > 5M rows | DuckDB | Analysis Pattern |
| Wide-to-Long | Pandas melt | Tidy Pattern |
| Clean Outliers | NumPy/SciPy | Stats Pattern |
[!IMPORTANT] This skill focusing on preparation. For statistical inference, multivariate modeling, or causal analysis, defer to the
@data-analysis-suite.
Related Skills
pablodiegoo/time-series-analysis
testing
VerifiedTrustedCommunity
Comprehensive time-series validation and analysis suite. Handles backtesting of trading and non-trading strategies with support for walk-forward validation (training vs testing windows), performance metric calculation (Sharpe, Drawdown, Win Rate), and event-driven resource allocation simulation. Use for: (1) Validating sequential logic on time-series data, (2) Calculating risk-adjusted performance, (3) Simulating constraints in resource distribution, (4) Detecting look-ahead bias through walk-forward testing.
6SKILL.mdUpdated May 26, 2026
pablodiegoo/survey-analytics
tools
VerifiedTrustedCommunity
Core statistical analysis and pipeline automation for survey datasets. Use for: (1) Running standard Crosstabs, NPS, Top-Box calculations, (2) Generating complete EDA or Analytics notebooks, (3) Quantitative and qualitative processing of questionnaire data.
6SKILL.mdUpdated May 26, 2026
pablodiegoo/strategic-frameworks
development
VerifiedTrustedCommunity
Business-level frameworks and actionable reporting for executives. Use for: (1) Plotting Priority Matrices, (2) Generating Pain Curves, (3) Conversion Funnels, (4) Removing Halo Effects to uncover true sentiment.
6SKILL.mdUpdated May 26, 2026
pablodiegoo/machine-learning-lite
testing
VerifiedTrustedCommunity
Tactical and highly interpretable Machine Learning. Use for: (1) Extracting Feature Importance via Random Forest, (2) Running Permutation Tests, (3) Handling Imbalanced Data (SMOTE).
6SKILL.mdUpdated May 26, 2026