samuraigpt/muapi-storyboard-to-cooking-video

Storyboard to Cooking Video

Turn a single photo of a person into a polished 15-second cinematic cooking tutorial. The skill first generates a high-end production reference sheet — character look, kitchen environment, and a 9-panel action board — then drives a continuous reference-to-video render that keeps the subject's face, outfit, and kitchen consistent across every frame.

Inputs

| Name | Type | Required | Default | Description | |:---|:---|:---|:---|:---| | person_image | image_url | yes | — | URL of the person photo. Used as identity reference in BOTH the reference sheet and the final video. | | dish | text | no | fresh pasta | The cooking subject (e.g. "fresh pasta", "sushi rolls", "wood-fired pizza", "matcha latte"). Drives the 9-step action board. | | kitchen_style | text | no | warm rustic-modern Italian | The kitchen aesthetic (e.g. "warm rustic-modern Italian", "minimalist Tokyo", "bright Scandinavian", "moody industrial"). | | outfit | text | no | white t-shirt, olive green apron, dark trousers | What the person wears throughout the video. | | duration_seconds | int | no | 15 | Final video duration. Use 15 for the full 9-step arc; 10 collapses to ~6 beats. | | aspect_ratio | text | no | 16:9 | Output aspect ratio. Use 9:16 for vertical/Reels. | | resolution | text | no | 720p | Video resolution. Options: 480p, 720p. |

Steps

Submit the plan with TWO sequential steps. Step 2 depends on the output of Step 1.

Step 1 — Reference Sheet (Composite Storyboard)

Generate the composite "production reference board" image. This is a single image, NOT a video frame — it bundles character sheet + location reference + 9-panel action board.

Endpoint: gpt-image-v2-edit CLI:

muapi image edit \
  --model gpt-image-v2-edit \
  --image "{{person_image}}" \
  --image-size "3840x2160" \
  --quality auto \
  --background auto \
  --moderation low \
  --output-format png \
  --prompt "Create one single composite reference sheet for a {{duration_seconds}}-second realistic {{dish}}-making tutorial video. The image should be a clean, high-end production reference board, not a poster with heavy text. Format: {{aspect_ratio}} wide reference sheet, elegant white margins, clean grid layout, realistic cinematic photography style. Concept: {{dish}} tutorial in a {{kitchen_style}} kitchen.

Top row: motion / choreography guide with 9 numbered cinematic action panels showing the {{dish}} process step-by-step from raw ingredients to final plated dish.

Middle-left: realistic character reference sheet of the uploaded person — preserve their exact face, hair color, hair texture, eye color, skin tone, and all facial features with 100% accuracy. Show the same person in: face close-up, full-body front view, side/action working pose, and back view. Dress them in {{outfit}}. Keep them grounded, approachable, skilled, and cinematic.

Middle-right / background: location reference sheet of an elegant {{kitchen_style}} kitchen with tactile surfaces, natural daylight from a large window, hanging cookware, herbs, and premium cooking atmosphere appropriate to the cuisine.

Style: realistic, cinematic, warm natural light, shallow depth of field, tactile food photography, premium cooking show aesthetic, rich surface textures.

Bottom strip: simple visual icons only for {{duration_seconds}} seconds, {{aspect_ratio}}, realistic, cinematic, tasty, natural camera. Minimal text, no dense paragraphs. Let the visuals do the heavy lifting."

Wait for completion and capture the output URL as {{reference_sheet_url}}. Show it to the user and confirm the character likeness + kitchen mood before moving to Step 2 — Step 2 is the expensive call.

Step 2 — Cooking Video (Reference-to-Video)

Animate the full sequence using both the original person photo (identity anchor) and the reference sheet (narrative + environment guide) as dual references.

Endpoint: bytedance-seedance-2-0-reference-to-video-fast CLI:

muapi video generate \
  --model bytedance-seedance-2-0-reference-to-video-fast \
  --image "{{person_image}}" \
  --image "{{reference_sheet_url}}" \
  --aspect-ratio "{{aspect_ratio}}" \
  --duration "{{duration_seconds}}" \
  --resolution "{{resolution}}" \
  --generate-audio true \
  --prompt "The person in @Image1 is the subject — preserve their exact face, hair, eye color, skin tone, and all facial features with 100% accuracy throughout the entire video.
Use @Image2 as the visual and narrative guide — follow the cooking steps, kitchen setting, outfit, and atmosphere shown in the reference sheet exactly.
A single continuous cinematic video of the person from @Image1 making {{dish}} in the {{kitchen_style}} kitchen shown in @Image2. They wear {{outfit}} throughout.

VIDEO STRUCTURE
Follow the exact 9-step sequence as shown in @Image2, beat by beat, from raw ingredients through preparation to a final plated close-up.

MOTION STYLE
- Slow, deliberate, satisfying transitions between each step
- Natural hand and body movement with clear culinary intent
- Continuous flow with no jump cuts
- Warm and immersive pacing

CAMERA & CINEMATOGRAPHY
- Close-up shots for hands during mixing, kneading, cutting, plating
- Medium shots showing the person working at the counter
- Pull back slightly for the final plating to reveal the full kitchen
- Shallow depth of field — focus on hands and food, soft background blur
- No abrupt cuts — smooth match cuts and fluid transitions

VISUAL STYLE
- Warm natural daylight from a large kitchen window
- Rich tactile textures matching @Image2's environment
- Full color, warm cinematic color grading

CONSISTENCY RULES
- Same character throughout — face of @Image1 in every frame
- Same outfit across entire video
- Same kitchen environment as shown in @Image2

AUDIO
- Soft kitchen ambience, gentle culinary SFX (chopping, sizzling, pouring), light cinematic underscore
- No dialogue, no narration

OUTPUT STYLE
- Duration: exactly {{duration_seconds}} seconds
- Polished, cinematic, premium cooking show quality
- Ends with a beautiful close-up of the finished plated {{dish}}"

After generation:

• Present the final video URL to the user.
• Offer follow-ups: vertical 9:16 re-render for Reels, a longer 30s extended cut, or swap {{dish}} for a different cuisine using the same person image.

Notes

• Two-image reference is the whole trick. @Image1 locks identity, @Image2 locks choreography + environment. Never drop one — single-reference runs lose either the face or the kitchen.
• The reference sheet at Step 1 must be wide (3840x2160). Smaller resolutions blur the 9 action panels and the video model can't read them.
• bytedance-seedance-2-0-reference-to-video-fast natively generates audio when generate_audio=true. Always include an audio direction in the prompt; otherwise the soundtrack is random.
• Real human faces ARE supported here because the person photo is the user's own subject and we route through the reference-to-video endpoint (not the restricted i2v variants).
• If the user wants a non-cooking sequence (e.g., latte art, plating tutorial, mixology), keep the same two-step structure — only {{dish}} and the 9-step description change.
• For shorter pieces (<= 8s), reduce the action board to 5–6 panels in Step 1; cramming 9 beats into 8s degrades motion quality (single-beat rule).

Trigger Keywords

cooking video, cooking tutorial, pasta video, recipe video, food video, chef video, cooking storyboard, kitchen tutorial, cooking reel, tutorial video from photo, storyboard to video

---

Notes for the Executing Agent

• This recipe is LLM-orchestrated: read each phase, gather any missing inputs from the user, then call muapi CLI commands. Use muapi auth configure first if MUAPI_API_KEY is unset.
• For model IDs without a CLI alias yet, fall back to the raw endpoint via curl -X POST https://api.muapi.ai/api/v1/<endpoint> -H "x-api-key: $MUAPI_API_KEY" -H 'content-type: application/json' -d '{...}' and poll with muapi predict wait <request_id>.
• Substitute {{input_name}} placeholders with the user's actual inputs before issuing each call.
• Step 1 must complete and return an output image URL before Step 2 fires — pass that URL as the second --image to the video step.