wenmin-wu/cv-pairwise-tracking-feature-merge

Pairwise Tracking Feature Merge

Overview

Interaction detection tasks (contact, collision, handoff) require features for both entities in each candidate pair. When tracking data has one row per entity per timestep, a double left-join merges entity 1's features first, then entity 2's features, producing a single row with both sets of kinematics. The resulting pairwise features (positions, speeds, accelerations, relative distance) serve as tabular input alongside visual features.

Quick Start

import numpy as np
import pandas as pd

def merge_pairwise_features(pairs_df, tracking_df, merge_col='step',
                            feature_cols=None):
    """Merge tracking features for both entities in each pair."""
    if feature_cols is None:
        feature_cols = ['x_position', 'y_position', 'speed', 'direction']

    track = tracking_df[['game_play', merge_col, 'nfl_player_id'] + feature_cols]

    df = (pairs_df
        .merge(track, left_on=['game_play', merge_col, 'player_id_1'],
               right_on=['game_play', merge_col, 'nfl_player_id'], how='left')
        .rename(columns={c: f'{c}_1' for c in feature_cols})
        .drop('nfl_player_id', axis=1)
        .merge(track, left_on=['game_play', merge_col, 'player_id_2'],
               right_on=['game_play', merge_col, 'nfl_player_id'], how='left')
        .rename(columns={c: f'{c}_2' for c in feature_cols})
        .drop('nfl_player_id', axis=1))

    df['distance'] = np.sqrt(
        (df['x_position_1'] - df['x_position_2'])**2
        + (df['y_position_1'] - df['y_position_2'])**2)
    return df

df_pairs = merge_pairwise_features(contact_pairs, tracking, merge_col='step')

Workflow

1. Start with a pairs DataFrame containing (game_play, timestep, entity_1, entity_2)
2. Left-join tracking data on entity_1 → suffix columns with _1
3. Left-join tracking data on entity_2 → suffix columns with _2
4. Compute derived features: Euclidean distance, relative speed, heading difference
5. Feed as tabular features to the model alongside visual features

Key Decisions

• Left join: preserves all pairs even when tracking is missing for one entity
• Feature columns: position, speed, direction, acceleration — all get _1/_2 suffixes
• Derived features: distance is most predictive; relative velocity and heading angle add marginal gain
• String casting: ensure player IDs match type between pairs and tracking DataFrames

References

• NFL 2.5D CNN
• NFL Player Contact Detection - Getting Started