0gfoundation/skills/compute/speech-to-text

Speech-to-Text Transcription

Metadata

• Category: compute
• SDK: @0glabs/0g-serving-broker ^0.6.5, ethers ^6.13.0
• Activation Triggers: "transcribe", "speech-to-text", "Whisper", "audio transcription"

Purpose

Transcribe audio files using 0G Compute Network providers running Whisper Large V3. Supports multiple audio formats and output types (JSON, text, SRT subtitles).

Prerequisites

• Node.js >= 22
• @0glabs/0g-serving-broker and ethers installed
• Funded and acknowledged provider with speech-to-text service
• Audio file in supported format (mp3, wav, ogg, flac, webm)
• .env with PRIVATE_KEY, RPC_URL, PROVIDER_ADDRESS

Quick Workflow

1. Initialize broker
2. Get service metadata (endpoint, model)
3. Create FormData with audio file and parameters
4. Generate auth headers
5. Make transcription request
6. Extract ChatID from ZG-Res-Key header ONLY
7. Call processResponse(providerAddress, chatID, usageData)

Core Rules

ALWAYS

• Use FormData for audio upload (not JSON)
• Get ChatID from ZG-Res-Key header (no body fallback for speech)
• Call processResponse() after every transcription
• Use correct processResponse() param order: (providerAddress, chatID, usageData)
• Include usage data if available in response
• Acknowledge provider before first use

NEVER

• Send audio as base64 in JSON body (use FormData)
• Skip processResponse() after transcription
• Try to get ChatID from response body for speech-to-text
• Hardcode private keys
• Use ethers v5 syntax

Code Examples

Basic Transcription

import { ethers } from 'ethers';
import { createZGComputeNetworkBroker } from '@0glabs/0g-serving-broker';
import * as fs from 'fs';
import 'dotenv/config';

async function transcribe(audioPath: string): Promise<string> {
  const provider = new ethers.JsonRpcProvider(process.env.RPC_URL);
  const wallet = new ethers.Wallet(process.env.PRIVATE_KEY!, provider);
  const broker = await createZGComputeNetworkBroker(wallet);

  const providerAddress = process.env.PROVIDER_ADDRESS!;
  const { endpoint, model } = await broker.inference.getServiceMetadata(providerAddress);
  const headers = await broker.inference.getRequestHeaders(providerAddress);

  const formData = new FormData();
  const audioBuffer = fs.readFileSync(audioPath);
  const audioBlob = new Blob([audioBuffer]);
  formData.append('file', audioBlob, audioPath.split('/').pop());
  formData.append('model', model);
  formData.append('response_format', 'json');

  const response = await fetch(`${endpoint}/audio/transcriptions`, {
    method: 'POST',
    headers: { ...headers },
    body: formData,
  });

  const data = await response.json();

  // ChatID from header ONLY for speech-to-text
  const chatID = response.headers.get('ZG-Res-Key') || response.headers.get('zg-res-key');

  await broker.inference.processResponse(
    providerAddress,
    chatID,
    data.usage ? JSON.stringify(data.usage) : undefined,
  );

  return data.text;
}

// Usage
const text = await transcribe('./audio/podcast.mp3');
console.log('Transcription:', text);

Transcription with Format Options

type OutputFormat = 'json' | 'text' | 'srt' | 'verbose_json';

async function transcribeWithFormat(
  audioPath: string,
  format: OutputFormat = 'json',
  language?: string,
): Promise<any> {
  const provider = new ethers.JsonRpcProvider(process.env.RPC_URL);
  const wallet = new ethers.Wallet(process.env.PRIVATE_KEY!, provider);
  const broker = await createZGComputeNetworkBroker(wallet);

  const providerAddress = process.env.PROVIDER_ADDRESS!;
  const { endpoint, model } = await broker.inference.getServiceMetadata(providerAddress);
  const headers = await broker.inference.getRequestHeaders(providerAddress);

  const formData = new FormData();
  const audioBuffer = fs.readFileSync(audioPath);
  formData.append('file', new Blob([audioBuffer]), audioPath.split('/').pop());
  formData.append('model', model);
  formData.append('response_format', format);
  if (language) formData.append('language', language);

  const response = await fetch(`${endpoint}/audio/transcriptions`, {
    method: 'POST',
    headers: { ...headers },
    body: formData,
  });

  const chatID = response.headers.get('ZG-Res-Key') || response.headers.get('zg-res-key');

  if (format === 'text' || format === 'srt') {
    const text = await response.text();
    if (chatID) {
      await broker.inference.processResponse(providerAddress, chatID);
    }
    return text;
  }

  const data = await response.json();
  await broker.inference.processResponse(
    providerAddress,
    chatID,
    data.usage ? JSON.stringify(data.usage) : undefined,
  );

  return data;
}

// Usage
const srt = await transcribeWithFormat('./audio/meeting.mp3', 'srt', 'en');
fs.writeFileSync('./output/meeting.srt', srt);

Error Handling

async function safeTranscribe(audioPath: string): Promise<string | null> {
  const provider = new ethers.JsonRpcProvider(process.env.RPC_URL);
  const wallet = new ethers.Wallet(process.env.PRIVATE_KEY!, provider);
  const broker = await createZGComputeNetworkBroker(wallet);

  const providerAddress = process.env.PROVIDER_ADDRESS!;

  try {
    // Validate file exists
    if (!fs.existsSync(audioPath)) {
      throw new Error(`Audio file not found: ${audioPath}`);
    }

    // Validate file size (most providers have limits)
    const stats = fs.statSync(audioPath);
    const maxSize = 25 * 1024 * 1024; // 25MB typical limit
    if (stats.size > maxSize) {
      throw new Error(`File too large (${stats.size} bytes). Max: ${maxSize} bytes`);
    }

    const { endpoint, model } = await broker.inference.getServiceMetadata(providerAddress);
    const headers = await broker.inference.getRequestHeaders(providerAddress);

    const formData = new FormData();
    const audioBuffer = fs.readFileSync(audioPath);
    formData.append('file', new Blob([audioBuffer]), audioPath.split('/').pop());
    formData.append('model', model);
    formData.append('response_format', 'json');

    const response = await fetch(`${endpoint}/audio/transcriptions`, {
      method: 'POST',
      headers: { ...headers },
      body: formData,
    });

    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${await response.text()}`);
    }

    const data = await response.json();
    const chatID = response.headers.get('ZG-Res-Key') || response.headers.get('zg-res-key');

    await broker.inference.processResponse(
      providerAddress,
      chatID,
      data.usage ? JSON.stringify(data.usage) : undefined,
    );

    return data.text;
  } catch (error) {
    console.error('Transcription failed:', error);
    return null;
  }
}

Supported Audio Formats

| Format | Extension | Notes | | ------ | --------- | ------------ | | MP3 | .mp3 | Most common | | WAV | .wav | Uncompressed | | OGG | .ogg | Compressed | | FLAC | .flac | Lossless | | WebM | .webm | Web native |

Output Formats

| Format | Description | | -------------- | -------------------------------- | | json | { "text": "..." } | | text | Plain text string | | srt | SubRip subtitle format | | verbose_json | Includes timestamps and segments |

Cost Estimate

~0.0001 0G per minute of audio (varies by provider).

Anti-Patterns

// BAD: Sending audio as JSON
const response = await fetch(endpoint, {
  body: JSON.stringify({ audio: base64Data }), // WRONG — use FormData
});

// BAD: Getting chatID from body
const chatID = data.id; // WRONG for speech — header only

// BAD: Missing processResponse
const data = await response.json();
return data.text; // processResponse() never called!

// BAD: Hardcoding private keys
const wallet = new ethers.Wallet('0xabc123...', provider); // NEVER do this

// BAD: ethers v5 syntax
const provider = new ethers.providers.JsonRpcProvider(url); // v5!

Common Errors & Fixes

| Error | Cause | Fix | | --------------------------- | ------------------- | -------------------------------- | | Insufficient balance | Sub-account empty | Transfer more funds | | unsupported format | Wrong audio format | Use mp3, wav, ogg, flac, or webm | | file too large | Audio file too big | Split into smaller segments | | Fee verification failed | Missing chatID | Check ZG-Res-Key header | | Provider not acknowledged | First-time provider | acknowledgeProviderSigner() |

Related Skills

• Provider Discovery — find speech providers
• Account Management — fund accounts
• Compute + Storage — transcribe and store

References

• Compute Patterns
• Network Config