kurko/coreml-diag

CoreML Diagnostics

Quick Reference

| Symptom | First Check | Pattern | |---------|-------------|---------| | Model won't load | Deployment target | 1a-1c | | Slow first load | Cache miss | 2a | | Slow inference | Compute units | 2b-2c | | High memory | Concurrent predictions | 3a-3b | | Bad accuracy after compression | Granularity | 4a-4c | | Conversion fails | Operation support | 5a-5b |

Decision Tree

CoreML issue
├─ Load failure?
│   ├─ "Unsupported model version" → 1a
│   ├─ "Failed to create compute plan" → 1b
│   └─ Other load error → 1c
├─ Performance issue?
│   ├─ First load slow, subsequent fast? → 2a
│   ├─ All predictions slow? → 2b
│   └─ Slow only on specific device? → 2c
├─ Memory issue?
│   ├─ Memory grows during predictions? → 3a
│   └─ Out of memory on load? → 3b
├─ Accuracy degraded?
│   ├─ After palettization? → 4a
│   ├─ After quantization? → 4b
│   └─ After pruning? → 4c
└─ Conversion issue?
    ├─ Operation not supported? → 5a
    └─ Wrong output? → 5b

---

Pattern 1a - "Unsupported model version"

Symptom: Model fails to load with version error.

Cause: Model compiled for newer OS than device supports.

Diagnosis:

# Check model's minimum deployment target
import coremltools as ct
model = ct.models.MLModel("Model.mlpackage")
print(model.get_spec().specificationVersion)

| Spec Version | Minimum iOS | |--------------|-------------| | 4 | iOS 13 | | 5 | iOS 14 | | 6 | iOS 15 | | 7 | iOS 16 | | 8 | iOS 17 | | 9 | iOS 18 |

Fix: Re-convert with lower deployment target:

mlmodel = ct.convert(
    traced,
    minimum_deployment_target=ct.target.iOS16  # Lower target
)

Tradeoff: Loses newer optimizations (SDPA fusion, per-block quantization, MLTensor).

---

Pattern 1b - "Failed to create compute plan"

Symptom: Model loads on some devices but not others.

Cause: Unsupported operations for target compute unit.

Diagnosis:

1. Open model in Xcode
2. Create Performance Report
3. Check "Unsupported" operations
4. Hover for hints

Fix:

// Force CPU-only to bypass unsupported GPU/NE operations
let config = MLModelConfiguration()
config.computeUnits = .cpuOnly
let model = try MLModel(contentsOf: url, configuration: config)

Better fix: Update model precision or operations during conversion:

# Float16 often better supported
mlmodel = ct.convert(traced, compute_precision=ct.precision.FLOAT16)

---

Pattern 1c - General Load Failures

Symptom: Model fails to load with unclear error.

Checklist:

1. Check file exists and is readable
2. Check compiled vs source model (runtime needs .mlmodelc)
3. Check available disk space (cache needs room)
4. Check model isn't corrupted (re-convert)

// Debug logging
let config = MLModelConfiguration()
config.parameters = [.reporter: { print($0) }]  // iOS 17+

---

Pattern 2a - Slow First Load (Cache Miss)

Symptom: First prediction after install/update is slow, subsequent are fast.

Cause: Device specialization not cached.

Diagnosis:

1. Profile with Core ML Instrument
2. Look at Load event label:
   • "prepare and cache" = cache miss (slow)
   • "cached" = cache hit (fast)

Why cache misses:

• First launch after install
• System update invalidated cache
• Low disk space cleared cache
• Model file was modified

Mitigation:

// Warm cache in background at app launch
Task.detached(priority: .background) {
    _ = try? await MLModel.load(contentsOf: modelURL)
}

Note: Cache is tied to (model path + configuration + device). Different configs = different cache entries.

---

Pattern 2b - All Predictions Slow

Symptom: Predictions consistently slow, not just first one.

Diagnosis:

1. Create Xcode Performance Report
2. Check compute unit distribution
3. Look for high-cost operations

Common causes:

| Cause | Fix | |-------|-----| | Running on CPU when GPU/NE available | Check computeUnits config | | Model too large for Neural Engine | Compress model | | Frequent CPU↔GPU↔NE transfers | Adjust segmentation | | Dynamic shapes recompiling | Use fixed/enumerated shapes |

Profile compute unit usage:

let plan = try await MLComputePlan.load(contentsOf: modelURL)
for op in plan.modelStructure.operations {
    let info = plan.computeDeviceInfo(for: op)
    print("\(op.name): \(info.preferredDevice)")
}

---

Pattern 2c - Slow on Specific Device

Symptom: Fast on Mac, slow on iPhone (or vice versa).

Cause: Different hardware characteristics.

Diagnosis:

// Check available compute
let devices = MLModel.availableComputeDevices
print(devices)  // Different per device

Common issues:

| Scenario | Cause | Fix | |----------|-------|-----| | Fast on M-series Mac, slow on iPhone | Model optimized for GPU | Use palettization (Neural Engine) | | Fast on iPhone, slow on Intel Mac | No Neural Engine | Use quantization (GPU) | | Slow on older devices | Less compute power | Use more aggressive compression |

Recommendation: Profile on target devices, not just development Mac.

---

Pattern 3a - Memory Grows During Predictions

Symptom: Memory increases with each prediction, doesn't release.

Cause: Input/output buffers accumulating from concurrent predictions.

Diagnosis:

Instruments → Allocations + Core ML template
Look for: Many concurrent prediction intervals
Check: MLMultiArray allocations growing

Fix: Limit concurrent predictions:

actor PredictionLimiter {
    private let maxConcurrent = 2
    private var inFlight = 0

    func predict(_ model: MLModel, input: MLFeatureProvider) async throws -> MLFeatureProvider {
        while inFlight >= maxConcurrent {
            await Task.yield()
        }
        inFlight += 1
        defer { inFlight -= 1 }
        return try await model.prediction(from: input)
    }
}

---

Pattern 3b - Out of Memory on Load

Symptom: App crashes or model fails to load on memory-constrained devices.

Cause: Model too large for device memory.

Diagnosis:

# Check model size
ls -lh Model.mlpackage/Data/com.apple.CoreML/weights/

Fix options:

| Approach | Compression | Memory Impact | |----------|-------------|---------------| | 8-bit palettization | 2x smaller | 2x less memory | | 4-bit palettization | 4x smaller | 4x less memory | | Pruning (50%) | ~2x smaller | ~2x less memory |

Note: Compressed weights are decompressed just-in-time (iOS 17+), so smaller on-disk = smaller in memory.

---

Pattern 4a - Bad Accuracy After Palettization

Symptom: Model output degraded after palettization.

Diagnosis:

1. What bit depth? (2-bit most likely to fail)
2. What granularity? (per-tensor loses more than per-grouped-channel)

Fix progression:

# Step 1: Try grouped channels (iOS 18+)
config = OpPalettizerConfig(
    nbits=4,
    granularity="per_grouped_channel",
    group_size=16
)

# Step 2: If still bad, try more bits
config = OpPalettizerConfig(nbits=6, ...)

# Step 3: If still need 4-bit, use calibration
from coremltools.optimize.torch.palettization import DKMPalettizer
# ... training-time compression

Key insight: 4-bit per-tensor has only 16 clusters for entire weight matrix. Grouped channels = 16 clusters per 16 channels = much better granularity.

---

Pattern 4b - Bad Accuracy After Quantization

Symptom: Model output degraded after INT8/INT4 quantization.

Diagnosis:

1. What bit depth?
2. What granularity?

Fix progression:

# Step 1: Use per-block (iOS 18+)
config = OpLinearQuantizerConfig(
    dtype="int4",
    granularity="per_block",
    block_size=32
)

# Step 2: Use calibration data
from coremltools.optimize.torch.quantization import LayerwiseCompressor
compressor = LayerwiseCompressor(model, config)
quantized = compressor.compress(calibration_loader)

Note: INT4 quantization works best on Mac GPU. For Neural Engine, prefer palettization.

---

Pattern 4c - Bad Accuracy After Pruning

Symptom: Model output degraded after weight pruning.

Diagnosis:

1. What sparsity level?
2. Post-training or training-time?

Thresholds (model-dependent):

• 0-30% sparsity: Usually safe
• 30-50% sparsity: May need calibration
• 50%+ sparsity: Usually needs training-time

Fix:

# Use calibration-based pruning
from coremltools.optimize.torch.pruning import LayerwiseCompressor

config = MagnitudePrunerConfig(
    target_sparsity=0.4,
    n_samples=128
)
compressor = LayerwiseCompressor(model, config)
sparse = compressor.compress(calibration_loader)

---

Pattern 5a - Operation Not Supported

Symptom: Conversion fails with unsupported operation error.

Diagnosis:

Error: "Op 'custom_op' is not supported for conversion"

Options:

1. Check if op is in coremltools: May need newer version

pip install --upgrade coremltools

2. Use composite ops: Split into supported primitives

# Instead of custom_op(x)
# Use: supported_op1(supported_op2(x))

3. Register custom op: Advanced, requires MIL programming

from coremltools.converters.mil import Builder as mb

@mb.register_torch_op
def custom_op(context, node):
    # Map to MIL operations
    ...

---

Pattern 5b - Conversion Succeeds but Wrong Output

Symptom: Model converts but predictions differ from PyTorch.

Diagnosis checklist:

1. Input normalization: Ensure preprocessing matches

# PyTorch often uses ImageNet normalization
# CoreML may need explicit preprocessing

2. Shape ordering: PyTorch (NCHW) vs CoreML (NHWC for some ops)

# Check shapes in conversion
ct.convert(..., inputs=[ct.ImageType(shape=(1, 3, 224, 224))])

3. Precision differences: Float16 may differ from Float32

# Force Float32 to match PyTorch
ct.convert(..., compute_precision=ct.precision.FLOAT32)

4. Random ops: Dropout, random initialization differ

# Ensure eval mode
model.eval()

Debug:

# Compare outputs layer by layer
import numpy as np

torch_output = model(input).detach().numpy()
coreml_output = mlmodel.predict({"input": input.numpy()})["output"]

print(f"Max diff: {np.max(np.abs(torch_output - coreml_output))}")

---

Pressure Scenario - "Model works on simulator but not device"

Wrong approach: Assume simulator bug, ignore.

Right approach:

1. Check model spec version vs device iOS version (Pattern 1a)
2. Check compute unit availability (Pattern 2c)
3. Profile on actual device, not simulator
4. Simulator uses host Mac's GPU/CPU, not device Neural Engine

---

Pressure Scenario - "Ship now, optimize later"

Wrong approach: Compress to smallest possible size without testing.

Right approach:

1. Ship Float16 baseline first
2. Profile on target devices
3. Apply compression incrementally with accuracy testing
4. Document compression settings for future optimization

---

Diagnostic Checklist

When CoreML isn't working:

• [ ] Check deployment target matches device iOS
• [ ] Check model file is compiled (.mlmodelc)
• [ ] Profile load: cached vs uncached
• [ ] Profile prediction: which compute units
• [ ] Check memory: concurrent predictions limited
• [ ] For compression issues: try higher granularity
• [ ] For conversion issues: check op support, precision

Resources

WWDC: 2023-10047, 2023-10049, 2024-10159, 2024-10161

Docs: /coreml, /coreml/mlmodel

Skills: coreml, coreml-ref