OVERVIEW
Production-shaped computer-vision pipeline for soccer video. YOLOv8n (stock COCO) handles detection, ByteTrack with Kalman filtering persists player identities, and a spatial-temporal graph feeds a GraphSAGE position-classifier scaffold. Wrapped in real MLOps: a DVC-declared pipeline, MLflow tracking, FastAPI endpoints with a live RTSP path, ONNX and TensorRT export, and Docker. Detection and tracking run on real La Liga footage at 22 FPS; the tactical GNN is wired and waiting on trained weights.
INPUT FRAME → PIPELINE OUTPUT · drag
PIPELINE OUTPUT
RAWARRIVED AS
Take raw soccer video (YouTube highlights, FIFA gameplay, or a live RTSP stream) and turn it into tracked players and tactical interaction graphs, wrapped in real MLOps for experiment tracking, data versioning, and deployment.
Commercial sports-analytics platforms lean on manual tagging teams that spend hundreds of hours per match, so only elite leagues get full coverage and lower-tier, youth, and amateur footage go unanalyzed. This project is an honest attempt at the automation path: get the production-grade plumbing right (detection, tracking, graph construction, MLOps) on real broadcast footage first, then layer trained tactical models on top. The detection and tracking stages work today on real La Liga clips; the tactical GNN is a wired scaffold waiting on labelled training data.
WHAT I BUILT
- 01Modular detect to track to graph pipeline: YOLOv8n detection, ByteTrack persistence with Kalman filtering, and a spatial-temporal graph built from tracklets.
- 02FastAPI service plus a live RTSP path, with ONNX and TensorRT export for deployment.
- 03MLflow experiment tracking and a DVC-versioned pipeline so runs are reproducible end to end.
WHAT CHANGED
- Detection and tracking run on real La Liga footage at 22 FPS on an RTX 3070-class 8GB GPU.
- GraphSAGE role classifier is scaffolded and inference-wired (trained weights pending), feeding off the tracklet graph.
- Each stage logs to MLflow and runs standalone or as one unified pipeline; Dockerized.
Data flow
click a stage
Pull frames from a YouTube highlight, a FIFA gameplay clip, or a live RTSP stream; preprocess to a common size and FPS.
COMPONENT
No component mapped to this stage.
Decisions, with the cost of each.
A decision without its trade-off is marketing. Each row says what was chosen, why, and what it gave up.
Stock YOLOv8n (COCO person + ball) instead of waiting on a fine-tuned model
COCO's person and sports-ball classes already cover the core of player and ball detection, so the rest of the pipeline (tracking, graph, serving) could be built and validated on real footage immediately rather than blocking on annotation and training.
No goalkeeper, referee, or team classes yet, and accuracy is generic rather than soccer-tuned. The soccer checkpoint is a defined next step, with the training pipeline already in the repo.
ByteTrack (via supervision) over DeepSORT
ByteTrack's second association pass over low-confidence detections recovers players through brief occlusions without a separate appearance re-identification model, which keeps the tracker light and dependency-free.
Less reliable under long occlusions than appearance-based re-ID; tuned here for mostly static broadcast camera angles.
GraphSAGE over flat positional features for tactics
Soccer tactics are relational: who is near whom, and how that changes over time. A graph where players are nodes and proximity plus time are edges encodes that directly, and an inductive GNN generalizes across variable player counts (substitutions, missed detections).
Needs labelled tactical data to actually train; the classifier currently ships as an inference-ready scaffold rather than trained weights.
MLflow + DVC instead of a managed MLOps platform
As a solo build, a lightweight stack gave experiment tracking, a model registry, and git-like data versioning that run locally or on any cloud, without standing up Kubernetes or paying for a managed service.
More manual wiring than a turnkey platform, and no hosted UI by default.
FastAPI with async endpoints and a live path
Video jobs take seconds to minutes and a live stream never stops, so async endpoints plus a dedicated RTSP path keep the service responsive instead of blocking on a single long request.
Real-time overlay rendering is GPU-bound, so live throughput is tied to the same 22 FPS detection budget.
The part that mattered.
The numbers behind the work, and the code that produced them.
- real-time inference
- 22 FPS
- RTX 3070-class 8GB GPU
- detect to track to graph
- 3-stage
- modular; runs unified or per stage
- occlusion recovery
- max_age 20
- ByteTrack recovers 3 to 5 frame gaps
- export targets
- ONNX + TRT
- deployment paths in src/detect
@dataclass(slots=True)
class InferenceConfig:
weights: str = "yolov8n.pt" # stock COCO weights
device: str = "cuda_if_available"
confidence: float = 0.4
max_frames: int = 30
def extract_detections(result) -> list[dict]:
"""Convert a YOLO result into serialisable detections, backend-agnostic."""
boxes = getattr(result, "boxes", None)
if boxes is None:
return []
xyxy = _tensor_to_list(getattr(boxes, "xyxy", None))
confs = _tensor_to_list(getattr(boxes, "conf", None))
classes = _tensor_to_list(getattr(boxes, "cls", None))
names = getattr(result, "names", {})
dets = []
for i, bbox in enumerate(xyxy):
cls_id = classes[i] if i < len(classes) else None
dets.append({
"bbox": bbox,
"confidence": confs[i] if i < len(confs) else None,
"class_id": cls_id,
"class_name": names.get(int(cls_id)) if cls_id is not None else None,
})
return dets
The real detection path: stock YOLOv8n through ultralytics, with detection extraction that tolerates torch tensors, numpy arrays, or plain lists so the rest of the pipeline gets clean JSON regardless of backend.
def build_track_graph(track_windows, window=30, distance_threshold=120.0,
include_temporal_edges=True, max_spatial_edges=1000):
active = track_windows[-window:]
# nodes = each tracked player in each frame of the window (bbox -> tensor)
edges = []
# temporal edges: link a track to itself across consecutive frames
if include_temporal_edges:
temporal = {}
for frame in active:
for track in frame.items:
cur = node_lookup[(frame.frame_id, track.track_id)]
if track.track_id in temporal:
prev_frame, prev = temporal[track.track_id]
if frame.frame_id - prev_frame <= window:
edges += [(prev, cur), (cur, prev)]
temporal[track.track_id] = (frame.frame_id, cur)
# spatial edges: link players within distance_threshold px
for node_indices in frame_to_nodes.values():
for a, b in combinations(node_indices, 2):
if dist(center(nodes[a]), center(nodes[b])) <= distance_threshold:
edges += [(a, b), (b, a)]
The interaction graph is what makes the tactical model possible. Temporal edges give each player continuity across the window; spatial edges connect players who are close on the pitch. Dense-frame and edge-count caps keep graph construction bounded on busy frames.
from torch_geometric.nn import SAGEConv
class PositionClassifier(nn.Module):
"""GraphSAGE classifier producing role logits per tracked player."""
def __init__(self, in_channels, hidden_channels=64, num_layers=2,
num_classes=3, dropout=0.2):
super().__init__()
self.convs = nn.ModuleList([SAGEConv(in_channels, hidden_channels)])
for _ in range(num_layers - 1):
self.convs.append(SAGEConv(hidden_channels, hidden_channels))
self.dropout = nn.Dropout(dropout)
self.classifier = nn.Linear(hidden_channels, num_classes)
def forward(self, data):
x, edge_index = data.x, data.edge_index
for conv in self.convs:
x = self.dropout(F.relu(conv(x, edge_index)))
return self.classifier(x)
The tactical head: a GraphSAGE stack that aggregates each player's graph neighbourhood into role logits. It is inference-ready and wired into the full pipeline, but ships untrained, training it on labelled tactical data is the open milestone.
✓ LEARNED
Build the pipeline around stock models first. Wiring detection, tracking, graph construction, and serving on COCO YOLOv8n meant the whole system worked end to end on real footage before spending a single GPU-hour on annotation or training.
ByteTrack's second pass over low-confidence detections is the part that matters. Keeping, not discarding, the 0.3 to 0.6 confidence boxes is what recovers players through brief occlusions in crowded frames.
Graphs are the right abstraction for tactics. Players are not independent; proximity and time are edges. Spending the effort on a clean spatial-temporal graph builder is what makes a tactical model tractable later.
MLflow plus DVC was enough. A lightweight, reproducible stack beat reaching for a managed MLOps platform on a solo project, with no vendor lock-in and a local-first workflow.
Production CV is mostly plumbing. Video IO, backend-agnostic tensor handling, async serving, RTSP capture, and export paths took far more effort than the model code, and that is where a notebook prototype would have fallen over.
◔ NOT DONE YET
- Fine-tune and ship the soccer-specific YOLO checkpoint (player, goalkeeper, ball, referee) using the existing training pipeline and DVC stage.
- Label tactical data and train the GraphSAGE role classifier so the scaffold produces real formation and role outputs.
- Homography and pitch calibration to map image coordinates to real pitch coordinates for distance-true graphs (calibration code exists in src/calib).
- Team classification by jersey colour to split tracks into home and away before graph construction.
- Edge deployment with INT8 quantization and ONNX Runtime for sideline analysis on tablets.