Capture
Process a short side-view walking sequence while preserving source and segment identifiers.
Computer VisionKeypoint EstimationBiomechanicsQuality AssuranceEdge/Cloud
Cattle Lameness Detection System
Video-based gait and back-shape analysis for automated locomotion scoring
An end-to-end system that analyzes walking videos of dairy cows, validates keypoint quality, extracts interpretable biomechanical features and produces locomotion assessments suitable for research and field deployment.
Problem & Context
Manual locomotion scoring is useful but time-consuming and affected by inter-rater variability. Practical recordings also vary in lighting, occlusion, background and walking path.
The project therefore focuses on an objective and traceable workflow that can operate with standard side-view video while explicitly reporting input quality and uncertainty-related indicators.
Design goals
- Objective measurements from ordinary video
- Robust handling of missing or low-confidence keypoints
- Traceable processing from video to prediction
- Support for both cloud and edge deployment
System Overview
The system avoids opaque scoring by exposing intermediate artifacts and quality flags alongside the final assessment.
Analyze
Estimate landmarks, evaluate data quality and calculate interpretable kinematic descriptors.
Report
Return locomotion estimates together with quality indicators and optional visual overlays.
Pipeline Architecture
1. Video ingestion and segmentation 2. Animal detection and keypoint estimation 3. Frame- and segment-level quality assurance 4. Back-shape and temporal feature engineering 5. Model inference and validation 6. Reporting, visualization and deployment logging
Data & Labeling
Veterinary locomotion scores provide the reference labels used for model development and evaluation.
Raw video references, per-frame landmarks, confidence values, quality metrics, engineered features and predictions remain linked for reproducibility.
Stored artifacts
- Video and segment identifiers
- Per-frame keypoints and confidence values
- Missing, outlier and completeness metrics
- Biomechanical and temporal features
- Model outputs and processing logs
Keypoint Estimation & QC
Segments are assigned to high, medium or low quality tiers. Strict training and validation can be limited to reliable samples, while lower-quality data can be retained for robustness analysis.
Quality metrics
- Missing-keypoint ratio
- Outlier ratio before and after interpolation
- Complete-frame ratio
- Confidence-threshold filtering
Feature Engineering
Features are computed per frame and aggregated at segment level so that posture and movement patterns can be evaluated without losing temporal context.
Back-shape features
- Regional angles along the back
- Curvature and straightness descriptors
- Body-relative geometry for position invariance
Temporal features
- Mean, standard deviation, minimum, maximum and range
- Window-based movement summaries
- Consistent feature scaling and normalization
Modeling & Validation
Modeling combines interpretable features with careful train-test separation and cross-validation. Outputs are reviewed together with quality indicators to avoid overconfident predictions on unreliable samples.
Modeling principles
- Cow-level separation between training and testing
- Stratified evaluation and class-imbalance handling
- Joint reporting of model performance and data reliability
- Qualitative review of disagreements with expert scores
Deployment & Operations
The modular implementation supports server-based inference as well as resource-aware processing on farm-side edge hardware.
Cloud mode
- Web upload and asynchronous analysis
- Containerized inference services
- Versioned models and reproducible environments
Edge mode
- GStreamer camera pipeline
- Detection- or RFID-triggered recording
- CPU/GPU-aware operation on NVIDIA Jetson
Results Snapshot
Validation reports combine predictive results with input-quality summaries and visual examples so that performance can be interpreted in context.
Validation artifacts
- Quality-tier distribution
- Prediction summaries by locomotion-score group
- Confusion matrices and class-wise metrics
- Example landmark and back-geometry overlays
Limitations & Next Steps
Important limitations include domain shift between farms and reduced landmark reliability under severe occlusion, poor lighting or non-standard camera geometry.
Next steps
- Broader multi-farm validation
- Uncertainty estimation and low-confidence flagging
- Active learning for difficult recordings
- Improved detector robustness and augmentation
- Richer explainability in the reporting layer