ibm/formulate-discovery-problem

Formulating Problems for ado Execution

Create validated discoveryspace and operation YAML for systematic work over an entity space: sampling and measuring entities, searching for entities that meet objectives, or benchmarking configurations (including research and benchmarking studies). Execution and analysis of results happen after resources are created — see conduct-empirical-study for the full workflow.

Tips

• Unless directed otherwise place all YAML and .md files created in a subdirectory of examples/ dedicated to the given problem.
• If you want to change the default value of an optional property use experiment parameterization, rather than setting a single valued property in the entity space
• Before creating a space or actuator configuration, check if one already exists — see resource-yaml-creation
• Learn ado CLI command-line construction and testing
• For metadata conventions, dynamic references (--use-latest, --with, --set), and resource-specific guidance, see resource-yaml-creation

Workflow Overview

The process has two main phases:

1. Create DiscoverySpace YAML - Define experiments and entity space
2. Create Operation YAML - Configure how to explore/analyze the space

Each phase follows a pattern: choose tool for task (experiment/operator) → create YAML for task → validate YAML → iterate.

Phase 1: Create DiscoverySpace YAML

Step 1a: Choose Experiments

List available experiments:

uv run ado get experiments --details

Describe a specific experiment:

uv run ado describe experiment $EXPERIMENT_ID

Key information to gather:

• Required constitutive properties (must be in entity space)
• Optional properties (can use defaults or add to entity space)
• Target properties (what the experiment measures)

What to do if no experiment matching task available

1. Learn how to extend ado: plugin-development.mdc
2. Propose a custom experiment or actuator to user that would provide missing functionality
3. Wait for user input

Step 1b: Create DiscoverySpace YAML

Generate initial template from experiment:

uv run ado template space --from-experiment $EXPERIMENT_ID -o space.yaml

Manual structure:

See skill-manual-structure.yaml.

Step 1c: Validate DiscoverySpace YAML

uv run ado create space -f space.yaml --dry-run

Step 1d: Iterate Until Valid

Fix validation errors and repeat validation until successful.

Phase 2: Create Operation YAML

Step 2a: Choose Operator

List available operators:

uv run ado get operators

Get operator template:

uv run ado template operation --operator-name $OPERATOR_NAME -o operation.yaml

Step 2b: Decide Parameters

Review the template and configure parameters based on:

• User's query/goals
• Operator documentation
• Example operations in examples/

Step 2c: Create Operation YAML

Structure:

See skill-operation-structure.yaml for an example structure.

Step 2d: Validate Operation YAML

uv run ado create operation -f operation.yaml --dry-run

Step 2e: Iterate Until Valid

Fix validation errors and repeat validation until successful.

Critical Rules

Experiment Selection Rules

1. Choose experiments first - Before defining entity space
2. All required inputs must be in entity space - Every requiredProperties (constitutive) from experiments must have a corresponding property in entitySpace
3. Optional properties - Only add to entity space if necessary to answer user's query. Explain why.
4. Default values - Only change default values of optional properties if necessary. Explain why.

Entity Space Refinement Rules

1. Refine domains to reduce size - Narrow property domains based on user's query. Explain the refinement.
2. No redundant dimensions - All entity space properties should be required by at least one experiment (validation will catch this)
3. Domain compatibility - Entity space property domains must be compatible with experiment requirements (subdomain or equal)

Property Domain Guidelines

Discrete (categorical):

See skill-property-domain-discrete-categorical.yaml.

Discrete (numeric):

See skill-property-domain-discrete-numeric.yaml.

Continuous:

See skill-property-domain-continuous.yaml.

Validation Checklist

Before finalizing, verify:

• All required experiment properties are in entity space
• Entity space domains are compatible with experiment requirements
• No redundant entity space dimensions
• Optional properties only added if necessary (with explanation)
• Default values only changed if necessary (with explanation)
• Domain refinements explained
• DiscoverySpace YAML validates (--dry-run)
• Operation YAML validates (--dry-run)
• All ado CLI commands and options are valid (uv run ado [COMMAND] --help)

Common Issues and Solutions

Issue: Validation error "required property not in entity space"

• Solution: Add the missing property to entitySpace with appropriate domain

Issue: Validation error "domain incompatible"

• Solution: Ensure entity space domain is a subdomain of experiment's required domain

Issue: Validation error "redundant dimension"

• Solution: Remove properties from entity space that aren't required by any experiment

Issue: Operation validation fails

• Solution: Check operator parameters match schema. Use --include-schema flag with template command.

Additional Resources

• For detailed schema information, see reference.md
• For example workflows, see examples.md
• For Pydantic model details, examine:
  • orchestrator/schema/experiment.py
  • orchestrator/schema/measurementspace.py
  • orchestrator/schema/entityspace.py
  • orchestrator/core/discoveryspace/config.py
  • orchestrator/core/operation/operation.py

References

When modifying or creating code while using this skill, follow:

• AGENTS.md
• plugin-development.mdc (if working with plugins)