mtsbatalha/data-quality-frameworks

Data Quality Frameworks

Production patterns for implementing data quality with Great Expectations, dbt tests, and data contracts to ensure reliable data pipelines.

Use this skill when

• Implementing data quality checks in pipelines
• Setting up Great Expectations validation
• Building comprehensive dbt test suites
• Establishing data contracts between teams
• Monitoring data quality metrics
• Automating data validation in CI/CD

Do not use this skill when

• The data sources are undefined or unavailable
• You cannot modify validation rules or schemas
• The task is unrelated to data quality or contracts

Instructions

• Identify critical datasets and quality dimensions.
• Define expectations/tests and contract rules.
• Automate validation in CI/CD and schedule checks.
• Set alerting, ownership, and remediation steps.
• If detailed patterns are required, open resources/implementation-playbook.md.

Safety

• Avoid blocking critical pipelines without a fallback plan.
• Handle sensitive data securely in validation outputs.

Resources

• resources/implementation-playbook.md for detailed frameworks, templates, and examples.