eric861129/nonlinear-solvers

Nonlinear Solvers

Goal

Provide a universal workflow to select a nonlinear solver, configure globalization strategies, and diagnose convergence for root-finding, optimization, and least-squares problems.

Requirements

• Python 3.8+
• NumPy (for Jacobian diagnostics)
• SciPy (optional, for advanced analysis)

Inputs to Gather

| Input | Description | Example | |-------|-------------|---------| | Problem type | Root-finding, optimization, least-squares | root-finding | | Problem size | Number of unknowns | n = 10000 | | Jacobian availability | Analytic, finite-diff, unavailable | analytic | | Jacobian cost | Cheap or expensive to compute | expensive | | Constraints | None, bounds, equality, inequality | none | | Smoothness | Is objective/residual smooth? | yes | | Residual history | Sequence of residual norms | 1,0.1,0.01,... |

Decision Guidance

Solver Selection Flowchart

Is Jacobian available and cheap?
├── YES → Problem size?
│   ├── Small (n < 1000) → Newton (full)
│   └── Large (n ≥ 1000) → Newton-Krylov
└── NO → Is objective smooth?
    ├── YES → Memory limited?
    │   ├── YES → L-BFGS or Broyden
    │   └── NO → BFGS
    └── NO → Anderson acceleration or Picard

Quick Reference

| Problem Type | First Choice | Alternative | Globalization | |--------------|--------------|-------------|---------------| | Small root-finding | Newton | Broyden | Line search | | Large root-finding | Newton-Krylov | Anderson | Trust region | | Optimization | L-BFGS | BFGS | Wolfe line search | | Least-squares | Levenberg-Marquardt | Gauss-Newton | Trust region | | Bound constrained | L-BFGS-B | Trust-region reflective | Projected |

Script Outputs (JSON Fields)

| Script | Key Outputs | |--------|-------------| | scripts/solver_selector.py | recommended, alternatives, notes | | scripts/convergence_analyzer.py | converged, convergence_type, estimated_rate, diagnosis | | scripts/jacobian_diagnostics.py | condition_number, jacobian_quality, rank_deficient | | scripts/globalization_advisor.py | strategy, line_search_type, trust_region_type, parameters | | scripts/residual_monitor.py | patterns_detected, alerts, recommendations | | scripts/step_quality.py | ratio, step_quality, accept_step, trust_radius_action |

Workflow

1. Characterize problem - Identify type, size, Jacobian availability
2. Select solver - Run scripts/solver_selector.py
3. Choose globalization - Run scripts/globalization_advisor.py
4. Analyze Jacobian - If available, run scripts/jacobian_diagnostics.py
5. Monitor residuals - During solve, use scripts/residual_monitor.py
6. Analyze convergence - Run scripts/convergence_analyzer.py
7. Evaluate steps - For trust region, use scripts/step_quality.py

Conversational Workflow Example

User: My Newton solver for a phase-field simulation is converging very slowly. After 50 iterations, the residual only dropped from 1 to 0.1.

Agent workflow:

1. Analyze convergence:

   python3 scripts/convergence_analyzer.py --residuals 1,0.8,0.6,0.5,0.4,0.3,0.2,0.15,0.12,0.1 --json
   

2. Check globalization strategy:

   python3 scripts/globalization_advisor.py --problem-type root-finding --jacobian-quality ill-conditioned --previous-failures 0 --json
   

3. Recommend: Switch to trust region with Levenberg-Marquardt regularization, or use Newton-Krylov with better preconditioning.

Pre-Solve Checklist

• [ ] Confirm problem type (root-finding, optimization, least-squares)
• [ ] Assess Jacobian availability and cost
• [ ] Check initial guess quality
• [ ] Set appropriate tolerances
• [ ] Choose globalization strategy
• [ ] Prepare to monitor convergence

CLI Examples

# Select solver for large unconstrained optimization
python3 scripts/solver_selector.py --size 50000 --smooth --memory-limited --json

# Analyze convergence from residual history
python3 scripts/convergence_analyzer.py --residuals 1,0.1,0.01,0.001,0.0001 --tolerance 1e-6 --json

# Diagnose Jacobian quality
python3 scripts/jacobian_diagnostics.py --matrix jacobian.txt --json

# Get globalization recommendation
python3 scripts/globalization_advisor.py --problem-type optimization --jacobian-quality good --json

# Monitor residual patterns
python3 scripts/residual_monitor.py --residuals 1,0.8,0.9,0.7,0.75,0.6 --target-tolerance 1e-8 --json

# Evaluate step quality for trust region
python3 scripts/step_quality.py --predicted-reduction 0.5 --actual-reduction 0.4 --step-norm 0.8 --gradient-norm 1.0 --trust-radius 1.0 --json

Error Handling

| Error | Cause | Resolution | |-------|-------|------------| | problem_size must be positive | Invalid size | Check problem dimension | | constraint_type must be one of... | Unknown constraint | Use: none, bound, equality, inequality | | residuals must be non-negative | Invalid residual data | Check residual computation | | Matrix file not found | Invalid path | Verify Jacobian file exists |

Interpretation Guidance

Convergence Type

| Type | Meaning | Action | |------|---------|--------| | quadratic | Optimal Newton | Continue, near solution | | superlinear | Quasi-Newton working | Monitor for stagnation | | linear | Acceptable | May improve with preconditioner | | sublinear | Too slow | Change method or formulation | | stagnated | No progress | Check Jacobian, preconditioner | | diverged | Increasing residual | Add globalization, check Jacobian |

Jacobian Quality

| Quality | Condition Number | Action | |---------|------------------|--------| | good | < 10⁶ | Standard Newton works | | moderately-conditioned | 10⁶ - 10¹⁰ | Consider scaling | | ill-conditioned | > 10¹⁰ | Use regularization | | near-singular | ∞ | Reformulate or use LM |

Step Quality (Trust Region)

| Ratio ρ | Quality | Trust Radius | |---------|---------|--------------| | ρ < 0 | very_poor | Shrink aggressively | | ρ < 0.25 | marginal | Shrink | | 0.25 ≤ ρ < 0.75 | good | Maintain | | ρ ≥ 0.75 | excellent | Expand if at boundary |

Limitations

• No global convergence guarantee: All methods may fail for pathological problems
• Jacobian accuracy: Finite-difference Jacobian may be inaccurate near discontinuities
• Large dense problems: May require specialized solvers not covered here
• Constrained optimization: Complex constraints need SQP or interior point methods

References

• references/solver_decision_tree.md - Problem-based solver selection
• references/method_catalog.md - Method details and parameters
• references/convergence_diagnostics.md - Diagnosing convergence issues
• references/globalization_strategies.md - Line search and trust region

Version History

• v1.0.0 : Initial release with 6 analysis scripts