CD-STGCN: A Novel Image-to-Graph Dual-Stream Spatio-Temporal Network for Interpretable Clinical Movement Quality Assessment
DOI:
https://doi.org/10.62411/faith.3048-3719-355Keywords:
Clinical movement assessment, Deep learning, Graph convolutional networks, Medical image processing, Rehabilitation, Skeleton-based action recognition, Spatio-temporal modeling, TelerehabilitationAbstract
: In medical imaging, automated assessment of human movement quality remains challenging because many existing computer vision approaches rely on rigid anatomical priors that fail to capture complex spatio-temporal coordination patterns. This limitation is particularly relevant for neurological rehabilitation, where objective motion analysis is needed to monitor patient progress and support remote telerehabilitation. To address this problem, we propose Clustering-based Dual-Stream Spatio-Temporal Graph Convolutional Network (CD-STGCN), a novel image-to-graph framework that dynamically discovers latent coordination patterns through unsupervised joint clustering. Unlike conventional graph-based approaches with fixed anatomical structures, the proposed method learns adaptive functional joint relationships directly from movement data. The framework integrates 3D skeleton extraction from RGB/depth clinical recordings, followed by a dual-stream architecture that captures both spatial posture and temporal motion dynamics from skeletal sequences. Latent Motion Clustering, implemented using a Student’s t-distribution kernel, enables data-driven functional joint grouping without explicit anatomical supervision, while dynamic graph construction generates sample-adaptive adjacency matrices. A multi-task learning strategy is further employed for simultaneous clinical score regression and exercise classification. Evaluation on the KIMORE dataset (n = 78 stroke survivors) demonstrates the effectiveness of the proposed framework. CD-STGCN achieved a mean absolute error of 0.098 (95% CI: 0.089–0.107), outperforming CTR-GCN (MAE = 0.121), corresponding to a 19% improvement (Cohen’s d = 1.24, p < 0.001). The proposed method also achieved real-time performance at 83 FPS compared with 35 FPS reported by recent rehabilitation-oriented approaches. Ablation analysis showed that removing the Latent Motion Clustering module reduced performance by 20.4% (MAE increased from 0.098 to 0.118, p < 0.001). These findings provide proof-of-concept evidence that adaptive, data-driven coordination modeling may offer advantages over static anatomical priors for clinical movement assessment, although larger and more diverse studies are still required before clinical deployment.
Downloads
References
W. M. Carroll, “The global burden of neurological disorders,” Lancet Neurol., vol. 18, no. 5, pp. 418–419, May 2019, doi: 10.1016/S1474-4422(19)30029-8.
F. S. Sarfo, U. Ulasavets, O. K. Opare-Sem, and B. Ovbiagele, “Tele-Rehabilitation after Stroke: An Updated Systematic Review of the Literature,” J. Stroke Cerebrovasc. Dis., vol. 27, no. 9, pp. 2306–2318, Sep. 2018, doi: 10.1016/j.jstrokecerebrovasdis.2018.05.013.
S. C. Cramer, B. M. Young, A. Schwarz, T. Y. Chang, and M. Su, “Telerehabilitation Following Stroke,” Phys. Med. Rehabil. Clin. N. Am., vol. 35, no. 2, pp. 305–318, May 2024, doi: 10.1016/j.pmr.2023.06.005.
A. Fugl-Meyer, L. Jääskö, I. Leyman, S. Olsson, and S. Steglind, “The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance,” J. Rehabil. Med., vol. 7, no. 1, pp. 13–31, Jan. 1975, doi: 10.2340/1650197771331.
E. Cippitelli, S. Gasparrini, E. Gambi, and S. Spinsante, “A Human Activity Recognition System Using Skeleton Data from RGBD Sensors,” Comput. Intell. Neurosci., vol. 2016, pp. 1–14, 2016, doi: 10.1155/2016/4351435.
C. G. Goetz et al., “Movement Disorder Society‐sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS‐UPDRS): Scale presentation and clinimetric testing results,” Mov. Disord., vol. 23, no. 15, pp. 2129–2170, Nov. 2008, doi: 10.1002/mds.22340.
S. Yan, Y. Xiong, and D. Lin, “Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition,” Proc. AAAI Conf. Artif. Intell., vol. 32, no. 1, Apr. 2018, doi: 10.1609/aaai.v32i1.12328.
R. Dickstein and N. Abulaffio, “Postural sway of the affected and nonaffected pelvis and leg in stance of hemiparetic patients,” Arch. Phys. Med. Rehabil., vol. 81, no. 3, pp. 364–367, Mar. 2000, doi: 10.1016/S0003-9993(00)90085-6.
L. P. Cobb et al., “Movement System Diagnosis: A Consensus-Based Position for Pediatric Movement Specialists,” Pediatr. Phys. Ther., vol. 37, no. 2, pp. 257–264, Apr. 2025, doi: 10.1097/PEP.0000000000001186.
Z. Zhang, “Microsoft Kinect Sensor and Its Effect,” IEEE Multimed., vol. 19, no. 2, pp. 4–10, Feb. 2012, doi: 10.1109/MMUL.2012.24.
Z. Liu, H. Zhang, Z. Chen, Z. Wang, and W. Ouyang, “Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2020, pp. 140–149. doi: 10.1109/CVPR42600.2020.00022.
U. Pawar, D. O’Shea, S. Rea, and R. O’Reilly, “Explainable AI in Healthcare,” in 2020 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), Jun. 2020, pp. 1–2. doi: 10.1109/CyberSA49311.2020.9139655.
X. Hu, “A Clinical Theory-Driven Deep Learning Model for Interpretable Autism Severity Prediction,” medRxiv. Jan. 26, 2026. doi: 10.64898/2026.01.25.26344792.
R. Ying, J. You, C. Morris, X. Ren, W. L. Hamilton, and J. Leskovec, “Hierarchical Graph Representation Learning with Differentiable Pooling,” in 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), 2019. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2018/file/e77dbaf6759253c7c6d0efc5690369c7-Paper.pdf
R. Vemulapalli, F. Arrate, and R. Chellappa, “Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp. 588–595. doi: 10.1109/CVPR.2014.82.
S. Patel, H. Park, P. Bonato, L. Chan, and M. Rodgers, “A review of wearable sensors and systems with application in rehabilitation,” J. Neuroeng. Rehabil., vol. 9, no. 1, p. 21, Dec. 2012, doi: 10.1186/1743-0003-9-21.
L. Shi, Y. Zhang, J. Cheng, and H. Lu, “Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2019, pp. 12018–12027. doi: 10.1109/CVPR.2019.01230.
Y. Chen, Z. Zhang, C. Yuan, B. Li, Y. Deng, and W. Hu, “Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 13339–13348. doi: 10.1109/ICCV48922.2021.01311.
S. Parisot et al., “Spectral Graph Convolutions for Population-Based Disease Prediction,” in Lecture Notes in Computer Science, 2017, pp. 177–185. doi: 10.1007/978-3-319-66179-7_21.
M. Y. Lu et al., “AI-based pathology predicts origins for cancers of unknown primary,” Nature, vol. 594, no. 7861, pp. 106–110, Jun. 2021, doi: 10.1038/s41586-021-03512-4.
C. Lindner, S. Thiagarajah, J. Wilkinson, T. Consortium, G. Wallis, and T. Cootes, “Fully Automatic Segmentation of the Proximal Femur Using Random Forest Regression Voting,” IEEE Trans. Med. Imaging, vol. 32, no. 8, pp. 1462–1472, Aug. 2013, doi: 10.1109/TMI.2013.2258030.
K. A. Thomas et al., “Automated Classification of Radiographic Knee Osteoarthritis Severity Using Deep Neural Networks,” Radiol. Artif. Intell., vol. 2, no. 2, p. e190065, Mar. 2020, doi: 10.1148/ryai.2020190065.
J. Zhu, Z. Ye, M. Ren, and G. Ma, “Transformative skeletal motion analysis: optimization of exercise training and injury prevention through graph neural networks,” Front. Neurosci., vol. 18, Mar. 2024, doi: 10.3389/fnins.2024.1353257.
F. M. Bianchi, D. Grattarola, and C. Alippi, “Spectral Clustering with Graph Neural Networks for Graph Pooling,” Proceedings of the 37 th International Conference on Machine Learning. Dec. 29, 2020. [Online]. Available: http://arxiv.org/abs/1907.00481
B. H. Dobkin, “Rehabilitation after Stroke,” N. Engl. J. Med., vol. 352, no. 16, pp. 1677–1684, Apr. 2005, doi: 10.1056/NEJMcp043511.
G. Kwakkel, R. C. Wagenaar, B. J. Kollen, and G. J. Lankhorst, “Predicting Disability in Stroke—A Critical Review of the Literature,” Age Ageing, vol. 25, no. 6, pp. 479–489, 1996, doi: 10.1093/ageing/25.6.479.
R. A. Clark et al., “Validity of the Microsoft Kinect for assessment of postural control,” Gait Posture, vol. 36, no. 3, pp. 372–377, Jul. 2012, doi: 10.1016/j.gaitpost.2012.03.033.
B. Galna, G. Barry, D. Jackson, D. Mhiripiri, P. Olivier, and L. Rochester, “Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease,” Gait Posture, vol. 39, no. 4, pp. 1062–1068, Apr. 2014, doi: 10.1016/j.gaitpost.2014.01.008.
W. Tao, T. Liu, R. Zheng, and H. Feng, “Gait Analysis Using Wearable Sensors,” Sensors, vol. 12, no. 2, pp. 2255–2283, Feb. 2012, doi: 10.3390/s120202255.
Y. Mourchid and R. Slama, “D-STGCNT: A Dense Spatio-Temporal Graph Conv-GRU Network based on transformer for assessment of patient physical rehabilitation,” Comput. Biol. Med., vol. 165, p. 107420, Oct. 2023, doi: 10.1016/j.compbiomed.2023.107420.
Y. Zhou, X. Yan, Z.-Q. Cheng, Y. Yan, Q. Dai, and X.-S. Hua, “BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2024, pp. 2049–2058. doi: 10.1109/CVPR52733.2024.00200.
W. Myung, N. Su, J.-H. Xue, and G. Wang, “DeGCN: Deformable Graph Convolutional Networks for Skeleton-Based Action Recognition,” IEEE Trans. Image Process., vol. 33, pp. 2477–2490, 2024, doi: 10.1109/TIP.2024.3378886.
Y.-W. Ma, J.-L. Chen, Y.-J. Chen, and Y.-H. Lai, “Explainable deep learning architecture for early diagnosis of Parkinson’s disease,” Soft Comput., vol. 27, no. 5, pp. 2729–2738, Mar. 2023, doi: 10.1007/s00500-021-06170-w.
S. Rani, S. Masood, and D. R. Rizvi, “A Bi-LSTM based approach for Compensation detection during robotic stroke rehabilitation therapy,” Procedia Comput. Sci., vol. 258, pp. 3145–3154, 2025, doi: 10.1016/j.procs.2025.04.572.
P. E. Shrout and J. L. Fleiss, “Intraclass correlations: Uses in assessing rater reliability.,” Psychol. Bull., vol. 86, no. 2, pp. 420–428, 1979, doi: 10.1037/0033-2909.86.2.420.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Diyar M. Witefee, Aseel Hussein Zahi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
