Asmayaseen/building-chatgpt-apps

ChatGPT Apps SDK Development Guide

Overview

Create ChatGPT Apps with interactive widgets that render rich UI inside ChatGPT conversations. Apps combine MCP servers (providing tools) with embedded HTML widgets that communicate via the window.openai API.

---

window.openai API Reference

Widgets communicate with ChatGPT through these APIs:

sendFollowUpMessage (Recommended for Actions)

Send a follow-up prompt to ChatGPT on behalf of the user:

// Trigger a follow-up conversation
if (window.openai?.sendFollowUpMessage) {
  await window.openai.sendFollowUpMessage({
    prompt: 'Summarize this chapter for me'
  });
}

Use for: Action buttons that suggest next steps (summarize, explain, etc.)

toolOutput

Send structured data back from widget interactions:

// Send data back to ChatGPT
if (window.openai?.toolOutput) {
  window.openai.toolOutput({
    action: 'chapter_selected',
    chapter: 1,
    title: 'Introduction'
  });
}

Use for: Selections, form submissions, user choices that feed into tool responses.

callTool

Call another MCP tool from within a widget:

// Call a tool directly
if (window.openai?.callTool) {
  await window.openai.callTool({
    name: 'read-chapter',
    arguments: { chapter: 2 }
  });
}

Use for: Navigation between content, chaining tool calls.

---

Critical: Button Interactivity Limitations

Important Discovery: Widget buttons may render as static UI elements rather than interactive JavaScript buttons. ChatGPT renders widgets in a sandboxed iframe where some click handlers don't fire reliably.

What Works

• sendFollowUpMessage - Reliably triggers follow-up prompts
• Simple onclick handlers for toolOutput calls
• CSS hover effects and visual feedback

What May Not Work

• Complex interactive JavaScript (selection APIs, etc.)
• Multiple chained tool calls from buttons
• window.getSelection() for text selection features

Recommended Pattern: Suggestion Buttons

Instead of complex interactions, use simple buttons that suggest prompts:

<div class="action-buttons">
  <button class="btn btn-primary" id="summarizeBtn">
    📝 Summarize Chapter
  </button>
  <button class="btn btn-primary" id="explainBtn">
    💡 Explain Key Concepts
  </button>
</div>

<script>
document.getElementById('summarizeBtn')?.addEventListener('click', async () => {
  if (window.openai?.sendFollowUpMessage) {
    await window.openai.sendFollowUpMessage({
      prompt: 'Summarize this chapter for me'
    });
  }
});

document.getElementById('explainBtn')?.addEventListener('click', async () => {
  if (window.openai?.sendFollowUpMessage) {
    await window.openai.sendFollowUpMessage({
      prompt: 'Explain the key concepts from this chapter'
    });
  }
});
</script>

---

Architecture Summary

┌─────────────────────────────────────────────────────────────────┐
│                        ChatGPT UI                                │
│  ┌─────────────────────────────────────────────────────────────┐│
│  │                    Widget (iframe)                          ││
│  │   HTML + CSS + JS                                          ││
│  │   Calls: window.openai.toolOutput({action: "...", ...})    ││
│  └─────────────────────────────────────────────────────────────┘│
│                              │                                   │
│                              ▼                                   │
│                     ChatGPT Backend                              │
│                              │                                   │
│                              ▼                                   │
│              MCP Server (FastMCP + HTTP)                         │
│              - Tools: open-book, read-chapter, etc.              │
│              - Resources: widget HTML (text/html+skybridge)      │
│              - Response includes: _meta["openai.com/widget"]     │
└─────────────────────────────────────────────────────────────────┘

---

Quick Start

1. Create MCP server with FastMCP and widget resources
2. Define widget HTML that uses window.openai.toolOutput
3. Add response metadata with _meta["openai.com/widget"]
4. Expose via ngrok for ChatGPT access
5. Register in ChatGPT Developer Mode settings

---

Widget HTML Requirements

Basic Widget Template

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>My Widget</title>
  <style>
    * { margin: 0; padding: 0; box-sizing: border-box; }
    body {
      font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
      background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
      min-height: 100vh;
      padding: 24px;
      color: white;
    }
    .container { max-width: 600px; margin: 0 auto; }
    .card {
      background: rgba(255,255,255,0.95);
      color: #333;
      padding: 24px;
      border-radius: 16px;
      box-shadow: 0 10px 40px rgba(0,0,0,0.2);
    }
    .btn {
      background: #667eea;
      color: white;
      border: none;
      padding: 12px 24px;
      border-radius: 8px;
      cursor: pointer;
      font-size: 16px;
    }
    .btn:hover { background: #5a6fd6; }
  </style>
</head>
<body>
  <div class="container">
    <div class="card">
      <h1>Widget Title</h1>
      <p>Widget content here</p>
      <button class="btn" onclick="handleAction()">Click Me</button>
    </div>
  </div>
  <script>
    function handleAction() {
      // Communicate back to ChatGPT
      if (window.openai && window.openai.toolOutput) {
        window.openai.toolOutput({
          action: "button_clicked",
          data: { timestamp: Date.now() }
        });
      }
    }
  </script>
</body>
</html>

Key Widget Rules

1. Always check window.openai.toolOutput before calling
2. Use inline styles - external CSS may not load reliably
3. Keep widgets self-contained - all HTML/CSS/JS in one file
4. Test with actual ChatGPT - browser preview won't have window.openai

---

MCP Server Setup (FastMCP Python)

Project Structure

my_chatgpt_app/
├── main.py              # FastMCP server with widgets
├── requirements.txt     # Dependencies
└── .env                 # Environment variables

requirements.txt

mcp[cli]>=1.9.2
uvicorn>=0.32.0
httpx>=0.28.0
python-dotenv>=1.0.0

main.py Template

import mcp.types as types
from mcp.server.fastmcp import FastMCP

# Widget MIME type for ChatGPT
MIME_TYPE = "text/html+skybridge"

# Define your widget HTML
MY_WIDGET = '''<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <style>
    body { font-family: sans-serif; padding: 20px; }
    .container { max-width: 500px; margin: 0 auto; }
  </style>
</head>
<body>
  <div class="container">
    <h1>Hello from Widget!</h1>
    <p>This content renders inside ChatGPT.</p>
  </div>
</body>
</html>'''

# Widget registry
WIDGETS = {
    "main-widget": {
        "uri": "ui://widget/main.html",
        "html": MY_WIDGET,
        "title": "My Widget",
    },
}

# Create FastMCP server
mcp = FastMCP("My ChatGPT App")


@mcp.resource(
    uri="ui://widget/{widget_name}.html",
    name="Widget Resource",
    mime_type=MIME_TYPE
)
def widget_resource(widget_name: str) -> str:
    """Serve widget HTML."""
    widget_key = f"{widget_name}"
    if widget_key in WIDGETS:
        return WIDGETS[widget_key]["html"]
    return WIDGETS["main-widget"]["html"]


def _embedded_widget_resource(widget_id: str) -> types.EmbeddedResource:
    """Create embedded widget resource for tool response."""
    widget = WIDGETS[widget_id]
    return types.EmbeddedResource(
        type="resource",
        resource=types.TextResourceContents(
            uri=widget["uri"],
            mimeType=MIME_TYPE,
            text=widget["html"],
            title=widget["title"],
        ),
    )


def listing_meta() -> dict:
    """Tool metadata for ChatGPT tool listing."""
    return {
        "openai.com/widget": {
            "uri": WIDGETS["main-widget"]["uri"],
            "title": WIDGETS["main-widget"]["title"]
        }
    }


def response_meta() -> dict:
    """Response metadata with embedded widget."""
    return {
        "openai.com/widget": _embedded_widget_resource("main-widget")
    }


@mcp.tool(
    annotations={
        "title": "My Tool",
        "readOnlyHint": True,
        "openWorldHint": False,
    },
    _meta=listing_meta(),
)
def my_tool() -> types.CallToolResult:
    """Description of what this tool does."""
    return types.CallToolResult(
        content=[
            types.TextContent(
                type="text",
                text="Tool executed successfully!"
            )
        ],
        structuredContent={
            "status": "success",
            "message": "Data for the widget"
        },
        _meta=response_meta(),
    )


if __name__ == "__main__":
    import uvicorn
    print("Starting MCP Server on http://localhost:8001")
    print("Connect via: https://your-tunnel.ngrok-free.app/mcp")
    uvicorn.run(
        "main:mcp.app",
        host="0.0.0.0",
        port=8001,
        reload=True
    )

---

Response Metadata Format

Critical: _meta["openai.com/widget"]

Tool responses MUST include widget metadata:

types.CallToolResult(
    content=[types.TextContent(type="text", text="...")],
    structuredContent={"key": "value"},  # Data for widget
    _meta={
        "openai.com/widget": types.EmbeddedResource(
            type="resource",
            resource=types.TextResourceContents(
                uri="ui://widget/my-widget.html",
                mimeType="text/html+skybridge",
                text=WIDGET_HTML,
                title="My Widget",
            ),
        )
    },
)

structuredContent

Data passed to the widget. The widget can access this via window.openai APIs.

---

Development Setup

1. Start Local Server

cd my_chatgpt_app
python main.py
# Server runs on http://localhost:8001

2. Start ngrok Tunnel

ngrok http 8001
# Get URL like: https://abc123.ngrok-free.app

3. Register in ChatGPT

1. Go to https://chatgpt.com/apps
2. Click Settings (gear icon)
3. Enable Developer mode
4. Click Create app
5. Fill in:
   • Name: Your App Name
   • MCP Server URL: https://abc123.ngrok-free.app/mcp
   • Authentication: No Auth (for development)
6. Check "I understand and want to continue"
7. Click Create

4. Test the App

1. Start a new chat in ChatGPT
2. Type @ to see available apps
3. Select your app
4. Ask it to use your tool

---

Common Issues and Solutions

Widget Shows "Loading..." Forever

Cause: Widget HTML not being delivered correctly.

Solution:

1. Check server logs for CallToolRequest processing
2. Verify _meta["openai.com/widget"] in response
3. Ensure MIME type is text/html+skybridge

Cached Widget Not Updating

Cause: ChatGPT caches widgets aggressively.

Solution:

1. Delete the app in Settings > Apps
2. Kill server and ngrok
3. Start fresh ngrok tunnel (new URL)
4. Create new app with new URL
5. Test in new conversation

Widget JavaScript Errors

Cause: window.openai not available.

Solution: Always check before calling:

if (window.openai && window.openai.toolOutput) {
  window.openai.toolOutput({...});
}

Tool Not Showing in @mentions

Cause: MCP server not connected or tools not registered.

Solution:

1. Check server is running and accessible via ngrok URL
2. Verify ngrok tunnel is active: curl https://your-url.ngrok-free.app/mcp
3. Check server logs for ListToolsRequest

---

Verification

Run: python3 scripts/verify.py

Expected: ✓ building-chatgpt-apps skill ready

If Verification Fails

1. Run diagnostic: Check references/ folder exists
2. Check: All reference files present
3. Stop and report if still failing

---

References

• Complete Template - Ready-to-use server + widget template
• Widget Patterns - HTML/CSS/JS widget examples
• Response Structure - Metadata format details
• Debugging Guide - Troubleshooting common issues