starlake-ai/kafkaload
.agents/skills/kafkaload/SKILL.md
Load or offload data to/from Kafka topics
$ install --global
skillsauthnpx skillsauth add starlake-ai/starlake-skills kafkaloadInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
Security Scan Results
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
3
Scanners Passed
9
Scanners in report
Clean
TrivyContainer and dependency vulnerability scanner
95%
Clean
SemgrepStatic code analysis for vulnerabilities
95%
Clean
mcp-scan (Snyk)Model Context Protocol security validation
95%
Skipped
Snyk (dep)Open source security scanning
50%
Skipped
Socket.devSupply chain security analysis
50%
Skipped
VirusTotalMulti-engine malware detection
50%
Skipped
CrowdStrikeAdvanced threat intelligence
50%
Skipped
OSV-ScannerOpen Source Vulnerability database check
50%
Skipped
OWASP Dep-Check
50%
Last scanned: Apr 16, 2026, 3:36 AM9.4s1 file scanned
SKILL.md
- name:
- kafkaload
- description:
- Load or offload data to/from Kafka topics
Kafka Load Skill
Loads data from files to Kafka topics or offloads data from Kafka topics to files. Supports both batch and streaming modes with optional transformations.
Usage
starlake kafkaload [options]
Options
Read Options
--config <value>: Kafka topic name (from connection configuration)--connectionRef <value>: Connection reference for the Kafka cluster--format <value>: Read format:parquet,json,csv, etc. (default:parquet)--path <value>: Source file path (for loading to Kafka) or target path (for offloading)--options k1=v1,k2=v2: Spark Reader options
Write Options
--write-config <value>: Write topic name--write-path <value>: Target file path for offloading--write-mode <value>: Write mode for file output--write-options k1=v1,k2=v2: Spark Writer options--write-format <value>: Streaming output format:kafka,console, etc.--write-coalesce <value>: Number of output partitions
Processing Options
--transform <value>: SQL transformation to apply to messages before loading/offloading--stream: Enable streaming mode (continuous processing)--streaming-trigger <value>: Trigger type:Once,Continuous,ProcessingTime--streaming-trigger-option <value>: Trigger interval (e.g.10 seconds)--streaming-to-table <value>: Sink to a table instead of files--streaming-partition-by <value>: Partition output by these columns--reportFormat <value>: Report output format:console,json, orhtml
Examples
Offload Kafka Topic to Parquet
starlake kafkaload --config orders_topic --path /data/output/orders --format parquet
Load Files to Kafka Topic
starlake kafkaload --config orders_topic --path /data/input/orders.json --format json --write-config orders_output
Stream from Kafka to Console
starlake kafkaload --config orders_topic --stream --write-format console
Stream with Processing Time Trigger
starlake kafkaload --config orders_topic --stream --streaming-trigger ProcessingTime --streaming-trigger-option "10 seconds"
Offload with Transformation
Apply a SQL transformation before saving:
starlake kafkaload --config orders_topic --path /data/output --transform "SELECT order_id, total FROM SL_THIS WHERE total > 100"
Related Skills
- load - Load data from files
- esload - Load data into Elasticsearch
- cnxload - Load data into JDBC tables
Related Skills
starlake-ai/starflow-transform-design
development
VerifiedTrustedCommunity
Design SQL transformations for data pipelines with quality checks and dependency management. Use when the user says "design transforms" or "create SQL transformations".
1SKILL.mdUpdated Apr 16, 2026
starlake-ai/starflow-sprint-planning
devops
VerifiedTrustedCommunity
Plan and track sprint progress for data pipeline implementation. Use when the user says "sprint planning" or "plan data sprint".
1SKILL.mdUpdated Apr 16, 2026
starlake-ai/starflow-source-analysis
testing
VerifiedTrustedCommunity
Analyze data sources in depth: schema, quality, volume, and extraction strategy. Use when the user says "analyze data source" or "profile this data source".
1SKILL.mdUpdated Apr 16, 2026
starlake-ai/starflow-schema-design
data-ai
VerifiedTrustedCommunity
Design Starlake-compatible table schemas with types, constraints, privacy, and expectations. Use when the user says "design schema" or "create table definition".
1SKILL.mdUpdated Apr 16, 2026