quic/aipc-toolkit

AIPC Toolkit

Overview

This skill provides a suite of tools and documentation for developing and deploying AI models on Qualcomm AI PCs and edge devices. It covers the end-to-end workflow from AI model export to ONNX, model inspection and conversion, to inference implementation.

!!! cautions and issues!!!

• The script should be run directly without copying to work folder if not sepcified. it is inside apic skill path.
• there are work around to use qairt toolchain. follow the aipc instruction. don't change toolchain. it will make process fail.
• python architecture detection is not reliable in windows because of emulation. use windows builtin command to detect arch.

Prerequisites:

• QAIRT_SDK_ROOT environment variable must be set

Core Tasks

0. QARIRT environment setup

for windows, Refer to references\win_qairt_setup.md. for linux, check offical web page - https://docs.qualcomm.com/doc/80-63442-10/topic/linux_setup.html

1. ONNX Model Export and Validation

Refer to references/model_export_validation.md for the complete guide on exporting models to ONNX and performing multi-stage validation. The ONNX model will be used for QNN/SNPE conversion.

2. Float Model Conversion

Convert ONNX models to either QNN or SNPE format with specified floating-point precision (FP16 or FP32).

SNPE Model Conversion

There are two ways to convert ONNX models to SNPE DLC format:

Method 1: Using qairt-converter (Recommended)

Use qairt-converter to convert ONNX models to SNPE DLC format. This is a unified converter that supports both QNN and SNPE formats.

Usage: Choose host toolchain from system. For Windows, always use x86_64-windows-msvc. For x86 Linux, use x86_64-linux-clang. ARM Linux cross-compilation is not supported.

python ${QAIRT_SDK_ROOT}/bin/HOST_TOOLCHAIN/qairt-converter \
    --input_network model.onnx \
    --output_path model.dlc \
    --float_bitwidth [16|32]

QNN Model Conversion

Use scripts/aipc_convert_fp.py to convert ONNX models to QNN format. This script automates calls to qnn-onnx-converter and qnn-model-lib-generator.

For ARM Windows, need extra " HTP Context Binary Generation" step for inference.

Usage:

python scripts/aipc_convert_fp.py --precision [16|32]

Precision Options:

• --precision 16: FP16 conversion (default, recommended for NPU)
• --precision 32: FP32 conversion

Generated Files and Folders:

• {model_name}.bin - Binary QNN model file
• {model_name}.cpp - C++ representation of the model
• {model_name}.yaml - Input/output configuration file (generated by inspection step)
• {model_name}.dll - Compiled model library (in test_libs_{model_name}_fp{precision}_{target_arch}/{architecture}/ directory)
• test_libs_{model_name}_fp{precision}_{target_arch}/ - Directory containing the compiled model library and supporting files

Cleanup Option: By default, the script now includes a --clean_up flag for the qnn-model-lib-generator which cleans up temporary build files during compilation.

Example:

python scripts/aipc_convert_fp.py --precision 16

INT Model Conversion

Use scripts/aipc_convert_int.py to convert ONNX models to QNN format with integer quantization (INT8/INT16). This script handles quantized model conversion with calibration data support.

Usage:

python scripts/aipc_convert_int.py --precision [8|16]

Precision Options:

• --precision 8: INT8 conversion (quantized, recommended for power efficiency)
• --precision 16: INT16 conversion (quantized)

Generated Files and Folders:

• {model_name}.bin - Binary QNN quantized model file
• {model_name}.cpp - C++ representation of the quantized model
• {model_name}.yaml - Input/output configuration file (generated by inspection step)
• {model_name}.dll - Compiled quantized model library (in test_libs_{model_name}_int{precision}_{target_arch}/{architecture}/ directory)
• test_libs_{model_name}_int{precision}_{target_arch}/ - Directory containing the compiled quantized model library and supporting files

Cleanup Option: By default, the script now includes a --clean_up flag for the qnn-model-lib-generator which cleans up temporary build files during compilation.

Example:

python scripts/aipc_convert_int.py --precision 8

3. Model Patching and Troubleshooting

If model conversion fails, follow this troubleshooting workflow:

1. Verify Conversion Status: Check the conversion output to identify specific errors or unsupported operators.

2. Apply Model Patches: Refer to references/model_export_validation.md for guidance on patching the model. This typically involves:

   • Replacing unsupported operators in the original model
   • Adjusting model architecture for QNN/SNPE compatibility
   • Re-exporting the model to ONNX format

3. Retry Conversion: After patching, repeat the Model Conversion step with the updated ONNX model.

4. Escalation: If conversion continues to fail after patching attempts, do not substitute with a different model. Instead, escalate the issue for technical support to resolve the underlying conversion compatibility problem.

Note: Model patching should address the root cause of conversion failures rather than working around them with alternative models.

4. Model Inspection

Use scripts/aipc_inspect_onnxio.py to inspect ONNX models and generate a YAML configuration file containing input and output tensor names, which will be used for inference scripts.

Usage:

python scripts/aipc_inspect_onnxio.py [model.onnx]

5. QNN/SNPE Model Inference

Run converted models through the ONNX→QNN/SNPE wrapper.

• Copy scripts/aipc and scripts/qai_onnxruntime.py into the working folder.
• Ensure QAIRT_SDK_ROOT is set and SDK runtime folders are on PATH (Windows) or LD_LIBRARY_PATH (Linux).
• Execute:

python aipc path/to/onnx_inference.py

• On Windows, generate the HTP context binary first (section 6) and provide its path.
• If I/O names fail, regenerate YAML:

python scripts/aipc_inspect_onnxio.py model.onnx

Common fixes: add SDK bins to PATH, use absolute context-binary paths, or regenerate the YAML I/O file.

6. HTP Context Binary Generation (On Target Device)

SNPE/DLC Context Binary Generation

Usage: use absolute path for dlc model. in windows

qnn-context-binary-generator --backend QnnHtp.dll --model QnnModelDlc.dll --dlc_path [model.dlc] --binary_file [model.dlc]

in linux

 ${QAIRT_SDK_ROOT}/bin/aarch64-oe-linux-gcc11.2/qnn-context-binary-generator --backend libQnnHtp.so --model libQnnModelDlc.so --dlc_path [model.dlc] --binary_file [model.dlc]

it will create output folder, and model.dlc.bin will in output folder.

QNN Context Binary Generation

Use scripts/aipc_dev_gen_contextbin.py to generate a QNN context binary from a compiled model library (.so). This must be executed on the target device.

Usage:

python scripts/aipc_dev_gen_contextbin.py --model path/to/model.so

7. End-to-End Deploy Workflow

For a complete workflow starting from a PyTorch model:

1. PyTorch to ONNX Export: Export your PyTorch model to ONNX format. Ensure any custom operators are handled or replaced if necessary.

2. Model Inspection: Use aipc_inspect_onnxio.py to inspect the ONNX model and generate I/O configuration.

3. Model Conversion: Convert the ONNX model to SNPE format with your desired precision (FP16 recommended).

4. Conversion Troubleshooting: If conversion fails, apply model patches as described in model_export_validation.md and retry.

5. Context Binary Generation: On the target device, generate the QNN context binary using aipc_dev_gen_contextbin.py.

6. Inference Implementation: Integrate the converted model into your app using qai_onnxruntime and the aipc ONNX Runtime loader (QNN/SNPE).

8.SNPE/QNNE Quantized Model Conversion (INT8/INT16)

For quantized models requiring calibration data:

Refer to references/model_quantization.md for calibration data preparation and detailed conversion steps.

9. QAIRT Environment Setup

Prerequisites: Before using any AIPC toolkit functionality, the QAIRT SDK environment must be properly configured.

Windows Setup: Refer to references/aipcenv_setup_windows.md for detailed Windows-specific installation and configuration instructions.

Linux Setup: Follow the official Qualcomm AI Engine Direct SDK documentation for Linux environment setup. Ensure the QAIRT_SDK_ROOT environment variable is properly set to point to your SDK installation directory.

Verification: After setup, verify your environment by checking:

echo $QAIRT_SDK_ROOT
ls $QAIRT_SDK_ROOT/bin/

The SDK should be accessible and all required binaries should be present in the bin directory.

Examples

Example: Deploy AI Application to Qualcomm Platform

User: "Deploy this AI application to Qualcomm platform"

Agent:

Deployment plan:
1. Export to ONNX and validate with test inference
2. Convert to QNN using aipc_convert_fp.py (FP16)
3. If conversion fails, patch model and retry
4. Inspect I/O with aipc_inspect_onnxio.py
5. Generate context binary on target device
6. Execute inference and validate performance
7. Retrieve and verify results

update and confirm it. let's run fast, don't walk!

Available Resources

Scripts

• scripts/aipc_inspect_onnxio.py: Inspects ONNX files and generates I/O YAML configurations
• scripts/aipc_convert_fp.py: Converts ONNX models to QNN format with FP16/FP32 precision
• scripts/aipc_dev_gen_contextbin.py: Generates QNN context binaries on target devices

Reference Documentation

• references/model_export_validation.md: Complete guide for exporting and validating models, including patching strategies
• references/model_quantization.md: Complete guide for model quantization and optimization techniques