harsh040506/enterprise-governance

Enterprise AI Governance — Fairness, Explainability, and Privacy

Provides comprehensive frameworks for responsible AI deployment in regulated enterprise environments. Covers algorithmic fairness analysis, SHAP/LIME explainability, adversarial robustness, differential privacy, and compliance with major AI regulatory frameworks.

Regulatory Context

| Regulation | Requirement | Relevant Audit | |---|---|---| | EU AI Act (High-Risk) | Mandatory conformity assessment, human oversight | Full governance audit | | GDPR Article 22 | Right to explanation for automated decisions | SHAP/LIME local explanations | | US ECOA/EEOC | No disparate impact in credit/employment decisions | Disparate Impact Ratio ≥ 0.8 | | HIPAA (Healthcare AI) | Privacy preservation in model training and inference | Differential privacy audit | | Basel III/SR 11-7 (Finance) | Model validation and risk management | Bias, drift, and robustness audits |

Algorithmic Fairness

Fairness Metrics (compute all for any regulated deployment):

• Demographic Parity: P(ŷ=1 | A=0) = P(ŷ=1 | A=1)
  • Violation: model assigns positive outcomes at different rates across groups
• Equal Opportunity: TPR_A=0 = TPR_A=1
  • Violation: model misses positive cases more often for one group
• Equalized Odds: both TPR and FPR equal across groups (stricter than Equal Opportunity)
• Disparate Impact Ratio: min_a P(ŷ=1|A=a) / max_a P(ŷ=1|A=a)
  • Threshold: ≥ 0.8 (EEOC 4/5ths rule) — below this triggers legal review

Bias remediation strategies:

• Pre-processing: reweight samples, resample minority group, relabel (Kamiran & Calders)
• In-processing: Lagrangian fairness constraint during training (Agarwal et al.)
• Post-processing: threshold calibration per demographic group (Hardt et al.)

See references/bias-detection.md for implementation, confidence intervals, and slice analysis methodology.

Explainability

SHAP (SHapley Additive exPlanations):

• Game-theoretic framework: φᵢ = contribution of feature i to prediction f(x)
• Unified property: Σᵢ φᵢ = f(x) − E[f(X)] (completeness)
• Efficient implementations:
  • TreeSHAP: exact, polynomial-time for tree models (XGBoost, LightGBM, Random Forest)
  • KernelSHAP: model-agnostic, samples coalitions for approximation (~200 samples)
  • DeepSHAP: backpropagation-based for neural networks

LIME (Local Interpretable Model-Agnostic Explanations):

• Perturbs input around point of interest; fits a local linear surrogate model g
• Local fidelity: g(x) ≈ f(x) in the neighborhood of x
• Use as sanity check against SHAP — major disagreements indicate model instability

See references/explainability.md for SHAP waterfall plots, global importance bar charts, and compliance report templates.

Adversarial Robustness

Fast Gradient Sign Method (FGSM): x_adv = x + ε · sign(∇_x L(θ, x, y))

Test at ε ∈ [0.01, 0.05, 0.1, 0.3] to construct an accuracy-vs-robustness curve.

Robustness thresholds:

• ε = 0.01 (imperceptible): accuracy should degrade < 5%
• ε = 0.05 (slight perturbation): accuracy should degrade < 20%
• ε = 0.1 (visible distortion): flag if accuracy < 50%

Adversarial training (remediation): augment training data with adversarial examples; retrains model on min_θ E[max_ε L(θ, x+δ, y)] (PGD training, Madry et al.)

Privacy-Preserving ML

Differential Privacy (DP-SGD):

• Add calibrated Gaussian noise to per-sample gradients before aggregation
• Privacy guarantee: (ε, δ)-DP — probability of inferring individual training record bounded
• Typical targets: ε ≤ 10 for "reasonable" privacy, ε ≤ 1 for strong privacy
• Cost: 5–20% accuracy reduction at strict DP settings; lower LR, smaller batch size

Federated Learning:

• Model trains locally on each client's data; only gradient updates (or model deltas) leave the device
• Aggregation: FedAvg — weighted average of client model updates
• Privacy amplification: combine with DP-SGD for strong privacy guarantees
• Communication efficiency: quantize gradients to 8-bit before upload

See references/privacy-preserving.md for full DP-SGD implementation, privacy budget accounting, and FL aggregation patterns.