rajshah4/skills/vm_install_analysis

VM Installation Analysis Skill

Overview

This skill analyzes VM installation processes to track time investment, identify bottlenecks, acknowledge customer issues, and generate actionable insights for process improvement.

Purpose

When deploying OpenHands Enterprise (or any VM-based installation), understanding the actual time and effort required helps:

1. Set realistic expectations for future customers
2. Identify process bottlenecks that delay installations
3. Acknowledge customer pain points and improve support
4. Track team effort for resource planning
5. Generate data-driven improvements to streamline the process

Use Cases

• Post-Installation Review: After completing a customer installation, analyze what went well and what didn't
• Process Benchmarking: Compare installation times across different customers
• Support Planning: Estimate support resources needed for upcoming installations
• Documentation Improvement: Identify areas where documentation is lacking
• Product Development: Feed issues back to engineering for product improvements

Key Features

📊 Timeline Analysis

• Automatically extracts timeline from Slack channels
• Calculates total calendar days and active working days
• Identifies holidays and breaks that affected timeline

👥 Effort Tracking

• Estimates customer time investment
• Tracks internal team effort by person
• Calculates total hours and working days

🐛 Issue Identification

• Extracts issues from Slack conversations
• Classifies by severity (Critical, High, Medium, Low)
• Calculates customer impact in days
• Tracks resolution status

💡 Recommendations

• Generates process improvement suggestions
• Identifies positive outcomes
• Creates actionable next steps

📝 Reporting

• Comprehensive markdown reports
• Executive summary with key metrics
• Detailed timeline breakdown
• Issue tracking and resolution
• Process improvement recommendations

Installation

This skill is part of the my-agent-skills repository. No additional dependencies required beyond Python 3.7+.

Usage

Method 1: Use the Analysis Workflow (Recommended)

For analyzing a customer installation from Slack data:

cd skills/vm_install_analysis
python workflow_vm_install_tracker.py --customer "Customer Name" --channel "channel-name"

Method 2: Use the Python Library Directly

from analyze_vm_install import VMInstallAnalyzer
from datetime import datetime

# Create analyzer
analyzer = VMInstallAnalyzer(
    customer_name="Build One",
    channel_name="#e-buildone"
)

# Add timeline events
analyzer.add_event(
    date=datetime(2026, 3, 9),
    event_type='kickoff',
    description='Slack channel created, project kickoff',
    actor='internal',
    effort_hours=2
)

analyzer.add_event(
    date=datetime(2026, 4, 2),
    event_type='deployment',
    description='AWS infrastructure deployed via Terraform',
    actor='customer',
    effort_hours=16
)

# Add issues
analyzer.add_issue(
    title="GitHub App Authentication Failed",
    description="OAuth configuration vs App ID confusion",
    severity='critical',
    raised_date=datetime(2026, 4, 3),
    resolved_date=datetime(2026, 4, 14),
    customer_impact_days=11,
    resolution="Used GitHub App creation script, manual Keycloak fix"
)

# Add holidays/breaks
analyzer.add_holiday_break(
    start=datetime(2026, 4, 11),
    end=datetime(2026, 4, 14),
    reason="Orthodox Easter holiday"
)

# Set final status
analyzer.analysis.success = True
analyzer.analysis.final_status = "Successfully deployed after resolving authentication issues"

# Generate report
report = analyzer.generate_report("output/buildone_analysis.md")
print("Analysis complete!")

Method 3: Integration with Agent Workflow

This skill can be invoked by an AI agent to automatically analyze Slack channels:

1. Search for the customer Slack channel
2. Read all messages from the channel
3. Read relevant thread replies
4. Extract timeline events and issues
5. Generate comprehensive analysis report

Example prompt for the agent:

Analyze the VM installation for [Customer Name] from the #[channel-name] Slack channel.
I need:
- Total days from start to finish
- Customer time investment
- Internal team effort
- All issues raised
- Process improvement recommendations

Data Sources

The skill can extract data from:

1. Slack Channels (Primary)

   • Enterprise channels (e-*): Customer-facing channels for direct support
   • Internal channels (i-*): Internal team coordination channels (NEW)
   • Channel creation date = Start date
   • Message timestamps = Timeline events
   • Thread lengths = Effort indicators
   • Error messages = Issues
   • Resolution messages = Completions

2. Multi-Channel Analysis (NEW)

   • Searches BOTH e-* and i-* channels for the same customer
   • Merges timeline events from multiple sources
   • De-duplicates overlapping information
   • Gets fuller picture of install process

3. Notion Pages (Optional)

   • Customer notes
   • Meeting summaries
   • Technical documentation
   • Issue tracking

4. Manual Entry (Fallback)

   • Direct Python API usage
   • CSV/JSON imports
   • Command-line input

Output Formats

1. Markdown Reports (for reference)

Generated in output/ directory with detailed analysis

2. Notion Pages (PRIMARY OUTPUT)

Sub-pages created under main tracking page: https://www.notion.so/Self-Hosted-Install-Recaps-3447be798a178003b493ce6a21adfd7c

Each customer gets a dedicated Notion page with:

• Timeline metrics
• Critical blockers (with severity levels)
• Technical stack
• Effort estimates
• Customer context
• Data sources and last updated date

Output Example

Executive Summary

Total Installation Timeline: 36 days
Active Working Days: ~8-10 days
Customer Effort: ~8-10 working days (64-80 hours)
Internal Effort: ~6-8 working days (48-64 hours)
Critical Issues: 1
Status: ✅ Successfully Deployed

Key Metrics Table

| Metric | Value | |--------|-------| | Calendar Days | 36 | | Active Working Days | 8-10 | | Critical Issues | 1 | | Customer Time | 64-80 hrs | | Internal Time | 48-64 hrs |

Timeline Phases

• Phase 1: Kickoff & Planning (23 days)
• Phase 2: Infrastructure Setup (2 days)
• Phase 3: Configuration & Deployment (11 days)
• Phase 4: Resolution & Handoff (0 days)

Issues Summary

• Critical: GitHub App Authentication (11-day impact)
• High: Keycloak Settings Sync (1-day impact)
• Medium: Field Naming Confusion (0.5-day impact)

Process Improvements

1. Add preflight check for GitHub App installation
2. Improve admin console → Keycloak sync
3. Rename confusing field labels
4. Create installation checklist

Metrics Tracked

Time Metrics

• Total Calendar Days: Start to finish including all delays
• Active Working Days: Excluding holidays and waiting periods
• Customer Effort Hours: Time spent by customer team
• Internal Effort Hours: Time spent by our team
• Per-Person Breakdown: Individual effort tracking

Issue Metrics

• Issue Count by Severity: Critical, High, Medium, Low
• Average Resolution Time: Days to resolve issues
• Customer Impact Days: Actual delay caused by issues
• Open vs Closed: Current status of issues

Process Metrics

• Phases Completed: Installation progress tracking
• Bottlenecks Identified: Where process got stuck
• Best Practices Followed: Checklist compliance
• Improvement Opportunities: Areas for enhancement

Configuration

Effort Estimation Heuristics

The skill uses these heuristics to estimate effort:

# Base effort for reading/responding to a message
base_hours = 0.5

# Additional time per message in thread
per_message_hours = 0.25  # 15 minutes

# Complexity multiplier
simple_task = 1.0
complex_issue = 2.0

# Total effort formula
effort = (base_hours + thread_length * per_message_hours) * complexity

Severity Classification

Issues are classified based on:

• Critical: Contains words like "blocked", "critical", "deployment fail"
• High: Thread length > 15 messages OR contains "major", "urgent"
• Medium: Thread length > 5 messages
• Low: Everything else

Actor Classification

Messages are classified as customer or internal based on:

1. User ID mapping (if available)
2. Name detection in message content
3. Email domain patterns
4. Default to "customer" if uncertain

Best Practices

When to Use This Skill

✅ DO use when:

• Completing a customer installation
• Preparing quarterly reviews
• Benchmarking installation times
• Planning resource allocation
• Improving documentation
• Training new support team members

❌ DON'T use when:

• Installation is still in progress (wait for completion)
• Insufficient data available (< 5 messages in channel)
• Customer has requested confidentiality
• Data is more than 6 months old (outdated patterns)

How to Get Accurate Results

1. Include ALL relevant Slack channels - Don't miss side conversations
2. Read thread replies - Critical context is often in threads
3. Mark holidays explicitly - Prevents skewing effort calculations
4. Validate effort estimates - Auto-estimates are approximate
5. Add manual context - Include information not in Slack
6. Review before sharing - Ensure accuracy and appropriate framing

Presenting Results

Internal Use:

• Include all details, warts and all
• Be honest about failures
• Focus on actionable improvements

Customer Sharing:

• Focus on positive outcomes
• Frame issues as "learnings"
• Highlight responsiveness
• Emphasize future improvements

Executive Summary:

• Lead with metrics
• Visualize timeline
• Top 3 improvements
• ROI of changes

Extending the Skill

Adding New Data Sources

To add support for new data sources (e.g., Jira, Linear):

def analyze_jira_issues(self, jira_data):
    """Extract issues from Jira tickets"""
    for ticket in jira_data:
        self.analyzer.add_issue(
            title=ticket['summary'],
            description=ticket['description'],
            severity=self.map_jira_priority(ticket['priority']),
            raised_date=ticket['created'],
            resolved_date=ticket['resolved'],
            ...
        )

Custom Report Templates

Create custom report formats:

def generate_executive_summary(self) -> str:
    """One-page executive summary"""
    # Custom formatting logic
    pass

def generate_csv_export(self) -> str:
    """CSV format for spreadsheet analysis"""
    # CSV generation logic
    pass

Integration with Dashboards

Export metrics for dashboards:

def export_metrics_json(self) -> dict:
    """JSON format for dashboard ingestion"""
    return {
        'customer': self.customer_name,
        'total_days': self.analysis.total_calendar_days,
        'customer_hours': self.analysis.customer_effort_hours,
        'internal_hours': self.analysis.internal_effort_hours,
        'issues': len(self.analysis.issues),
        ...
    }

Troubleshooting

"No events found"

• Check that Slack channel name is correct
• Verify message timestamps are being parsed
• Ensure date range covers installation period

"Effort estimates seem wrong"

• Review effort estimation heuristics
• Add manual overrides for known events
• Validate thread lengths are accurate

"Missing critical issues"

• Check issue detection keywords
• Manually add known issues
• Review thread replies for hidden context

"Report formatting broken"

• Ensure all dates are datetime objects
• Check for None values in required fields
• Validate markdown syntax

Examples

See output/buildone_vm_install_analysis.md for a complete real-world example.

Contributing

To improve this skill:

1. Add new data source integrations
2. Enhance effort estimation algorithms
3. Improve issue classification accuracy
4. Create additional report templates
5. Add visualization capabilities

License

MIT License - Feel free to use and modify for your needs.

Support

For questions or issues with this skill:

• Open an issue in the repository
• Contact: [email protected]
• Slack: #skills-development

---

Version: 1.0.0
Last Updated: April 16, 2026
Author: Rajiv Shah