Game-Based Motion Capture for Rehabilitation Assessment

Virtual rehabilitation, particularly through engaging game-based therapies, has emerged as a highly effective and preferred alternative to traditional care for enhancing upper limb function and improving daily living activities. These innovative systems, often leveraging low-cost motion capture technology like the Microsoft Kinect™, generate enormous datasets of therapeutic movements. However, a significant hurdle has been the lack of a robust infrastructure to efficiently process and analyze this vast quantity of movement data collected outside controlled laboratory settings, such as homes and clinics. The challenge is compounded by the inherent noise, extraneous movements, and potential capture of other individuals or inanimate objects in naturalistic environments.

This pivotal paper, “A Novel Methodology for Extracting and Evaluating Therapeutic Movements in Game-Based Motion Capture Rehabilitation Systems,” presents a groundbreaking, comprehensive framework designed to overcome these analytical challenges. The methodology is built upon the adroit integration of signal processing, kinematic analysis, clustering algorithms, and probability density functions to accurately extract and evaluate task-relevant movements over time. While developed using Microsoft Kinect™ data from the “Recovery Rapids™” gaming system, its principles are adaptable to any motion capture system producing time-series data of joint angles or 3D joint positions.

Key features and steps of this innovative methodology include:

• Efficient Data Processing: It establishes an infrastructure for the efficient processing of large quantities of motion capture data from uncontrolled clinical or home settings, enabling meaningful kinematic analysis within individuals.
• Robust Denoising (Step 1): Initial preprocessing involves removing duplicate records and entries with missing data. A median filter is then applied to effectively remove spikes and noise, ensuring data quality without flattening natural movement peaks. This process typically removes less than 1% of total data while maintaining accuracy.
• Precise Session Segmentation (Step 2): The methodology identifies isolated sessions of consecutive game play, defined as periods with no more than a 5-minute lapse in therapeutic movements. This step is crucial because features used to detect extraneous data can vary between sessions.
• Extraneous Data Removal with DBSCAN (Step 3): Addressing the challenge of uncontrolled settings, this step uses the DBSCAN (Density-based spatial clustering of applications with noise) algorithm to identify and remove data unrelated to therapeutic game play. It leverages specific kinematic features, such as trunk angles (identifying non-upright posture or inanimate objects), torso position (identifying movement around the room), and body segment lengths (identifying different individuals or incorrect sensor facing), which exhibit distinct profiles for task-relevant vs. extraneous data. This step can remove data from other individuals, inanimate objects, or non-therapeutic patient movements like gesturing or walking away from the system.
• Accurate Gesture Retrieval (Step 4): Therapeutic gestures, which are specific movement patterns mapped to game actions (e.g., shoulder abduction for steering in Recovery Rapids™), are precisely parsed from the processed data. This step tackles the difficulty of identifying accurate start and end points for gestures, especially in patients with abnormal or non-fluid movement patterns, by employing strategies like thresholding local minima and analyzing slope changes.
• Normalcy Shape Index (Nshape) (Step 5): To evaluate how well each therapeutic movement matches an expected reference, an ideal movement trajectory is generated from control participants. The Nshape index quantifies the goodness of fit by calculating Euclidean distances between the produced movement and the ideal shape, as well as the least compatible shape. Movements with an Nshape below a predefined threshold (e.g., 0.73 for “bats” movements) are filtered out, effectively removing improperly parsed or combined movements.
• Kinematic Normalcy Indices (Nquality of movement) (Step 6): For movements passing the Nshape threshold, the quality of movement is calculated. This second application of the normalcy index incorporates multiple kinematic features (e.g., speed, range of motion, compensatory movements) into a single, objective Nquality of movement score, providing a more powerful assessment tool than examining individual features [34, 35, Table 2].

Validation and Insights:

The methodology was validated using a retrospective database of skeletal data from 24 individuals with chronic stroke and mild to moderate upper extremity impairment who played Recovery Rapids™ independently at home for 15 hours over three weeks [9, 36, 37, Table 4]. The study focused on capturing spontaneous movements during sustained practice, rather than maximal effort.

• Data Retention: On average, about 71% of the original raw data was retained for final analysis, demonstrating the feasibility of capturing ample suitable data despite the initial volume [49, Table 5].
• Validity and Reliability: Significant correlations were observed between Nquality of movement and established motor measures like the Wolf Motor Function Test (r = −.62) and the Motor Activity Log (r = 0.27), establishing convergent validity. Divergent validity was confirmed by the absence of a significant relationship with cognition. The methodology also demonstrated good test-retest reliability with a Cronbach’s alpha of .82.
• Personalized Rehabilitation Insights: Analysis of dose-response plots revealed two crucial observations:
  1. The majority of patients continued to show improvement through the end of treatment, suggesting that current rehabilitation trials may under-dose participants.
  2. Patterns of recovery are specific to each individual, highlighting the value of this data-driven approach for personalized treatment planning and predicting rehabilitation potential.

This framework represents a crucial advancement in utilizing objective kinematic data from real-world rehabilitation settings. It provides the foundation for generating personalized, data-driven insights from game play, not only for stroke rehabilitation but also with potential generalizability to other rehabilitation applications and even athletic performance quantification.

Reference: Yang, Z., Rafiei, M. H., Hall, A., Thomas, C., Midtlien, H. A., Hasselbach, A., Adeli, H., & Gauthier, L. V. (2018). A Novel Methodology for Extracting and Evaluating Therapeutic Movements in Game-Based Motion Capture Rehabilitation Systems. Journal of Medical Systems, 42(255). https://doi.org/10.1007/s10916-018-1113-4