Small Movements Detection In Video Using Video-EEG Data

Valerii Zuev, Elena Salmagambetova, Stepan Djakov, Vadim Pak

Abstract


Long-term video-EEG monitoring with sleep recording is the gold standard for diagnosing epilepsy. Automated methods for video-EEG analysis primarily focus on EEG, neglecting the video modality. This paper addresses the underexplored problem of detecting small motions, such as eye blinks and eyelid fluttering, from video recordings. We propose a novel pipeline that uses EEG eye movement and myographic artifacts as a source of ground-truth labels to train a video-based classifier. Our video-based classifier achieves 94.4% test accuracy and 87.4% accuracy when tested on a different subject.

Full Text:

PDF

References


Fisher, Robert S., et al. "Epileptic seizures and epilepsy: definitions proposed by the International League Against Epilepsy (ILAE) and the International Bureau for Epilepsy (IBE)." Epilepsia 46.4 (2005): 470-472. Available: https://doi.org/10.1111/j.0013-9580.2005.66104.x

V. A. Zuev, E. G. Salmagambetova, S. N. Djakov, and L. V. Utkin, “Automated Video-EEG Analysis in Epilepsy Studies: A Narrative Review of Advances and Challenges,” Journal of Medical Systems, vol. 49, no. 1, p. 142, Oct. 2025, doi: 10.1007/s10916-025-02274-0. Available: https://doi.org/10.1007/s10916-025-02274-0

Hamed, S. A. Ocular dysfunctions and toxicities induced by antiepileptic medications: Types, pathogenic mechanisms, and treatment strategies. Expert Rev Clin Pharmacol. 2019 Apr;12(4):309-328. doi: 10.1080/17512433.2019.1591274. PMID: 30840840. Available: https://doi.org/10.1080/17512433.2019.1591274

Bajwa, R., Jay, W. M., & Asconapé, J. (2006). Neuro-Ophthalmologic Manifestations of Epilepsy. Seminars in Ophthalmology, 21(4), 255–261. doi: 10.1080/08820530601006742. Available: https://doi.org/10.1080/08820530601006742

Saporito M. A. N., Vitaliti G., Pavone P., Di Stefano G, Striano P, Caraballo R. H., Falsaperla R. Ictal blinking, an under-recognized phenomenon: our experience and literature review. Neuropsychiatr Dis Treat. 2017 May 31;13:1435-1439. doi: 10.2147/NDT.S135979. PMID: 28615945; PMCID: PMC5460649. Available: https://doi.org/10.2147/NDT.S135979

Giangrande, Alessandra, et al. "Motion artifacts in dynamic EEG recordings: Experimental observations, electrical modelling, and design considerations." Sensors 24.19 (2024): 6363. Available: https://doi.org/10.3390/s24196363

Britton, Jeffrey W., et al. "Electroencephalography (EEG): An introductory text and atlas of normal and abnormal findings in adults, children, and infants." Chicago: American Epilepsy Society (2016). ISBN 978-0-9979756-0-4. Available: https://www.ncbi.nlm.nih.gov/books/NBK390354/

M. K. Islam, A. Rastegarnia, and Z. Yang, “Methods for artifact detection and removal from scalp EEG: A review,” Neurophysiologie Clinique/Clinical Neurophysiology, vol. 46, no. 4, pp. 287–305, Nov. 2016, doi: 10.1016/j.neucli.2016.07.002. Available: https://doi.org/10.1016/j.neucli.2016.07.002

I. Obeid and J. Picone, “The Temple University Hospital EEG Data Corpus,” Frontiers in Neuroscience, vol. 10, May 2016, doi: 10.3389/fnins.2016.00196.

H. Zhang, M. Zhao, C. Wei, D. Mantini, Z. Li, and Q. Liu, “EEGdenoiseNet: A benchmark dataset for deep learning solutions of EEG denoising,” Journal of Neural Engineering, vol. 18, no. 5, p. 056057, Oct. 2021, doi: 10.1088/1741-2552/ac2bf8.

J. Chen, D. Pi, X. Jiang, Y. Xu, Y. Chen, and X. Wang, “Denosieformer: A Transformer-Based Approach for Single-Channel EEG Artifact Removal,” IEEE Transactions on Instrumentation and Measurement, vol. 73, pp. 1–16, 2024, doi: 10.1109/TIM.2023.3341114.

C.-H. Chuang, K.-Y. Chang, C.-S. Huang, and A.-M. Bessas, “Augmenting brain-computer interfaces with ART: An artifact removal transformer for reconstructing multichannel EEG signals,” NeuroImage, vol. 310, p. 121123, Apr. 2025, doi: 10.1016/j.neuroimage.2025.121123.

W. Jiang, L. Zhao, and B. Lu, “Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI,” Oct. 2023. Accessed: Jun. 11, 2025. [Online]. Available: https://openreview.net/forum?id=QzTpTRVtrP

G. Wang, W. Liu, Y. He, C. Xu, L. Ma, and H. Li, “EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG Signals,” Advances in Neural Information Processing Systems, vol. 37, pp. 39249–39280, Dec. 2024, Accessed: Jun. 11, 2025. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/hash/4540d267eeec4e5dbd9dae9448f0b739-Abstract-Conference.html

A. Tegon, T. M. Ingolfsson, X. Wang, L. Benini, and Y. Li, “FEMBA: Efficient and Scalable EEG Analysis with a Bidirectional Mamba Foundation Model.” arXiv, Oct. 2025. doi: 10.48550/arXiv.2502.06438.

S. Sathyanarayana, R. K. Satzoda, S. Sathyanarayana, and S. Thambipillai, “Vision-based patient monitoring: A comprehensive review of algorithms and technologies,” Journal of Ambient Intelligence and Humanized Computing, vol. 9, no. 2, pp. 225–251, Apr. 2018, doi: 10.1007/s12652-015-0328-1.

H. Mu, R. Sun, G. Yuan, and Y. Wang, “Abnormal Human Behavior Detection in Videos: A Review,” Information Technology and Control, vol. 50, no. 3, pp. 522–545, Sep. 2021, doi: 10.5755/j01.itc.50.3.27864.

Z. Li, A. M. da Silva, and J. P. S. Cunha, “Movement quantification in epileptic seizures: A new approach to video-EEG analysis,” IEEE Transactions on Biomedical Engineering, vol. 49, no. 6, pp. 565–573, Jun. 2002, doi: 10.1109/TBME.2002.1001971.

J. Cao, Y. Fang, X. Cui, R. Zheng, T. Jiang, and F. Gao, “Synchronized Video and EEG Based Childhood Epilepsy Seizure Detection,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 8, no. 6, pp. 3742–3753, Dec. 2024, doi: 10.1109/TETCI.2024.3372387.

A. Morozov, O. Sushkova, I. Okuneva, and M. Sinkin, “Development of software tools for video-EEG monitoring based on the Actor Prolog logic programming system,” Radioelectronics. Nanosystems. Information Technologies., vol. 17, pp. 383–394, Jun. 2025, doi: 10.17725/j.rensit.2025.17.383.

R. Bajwa, W. M. Jay, and J. Asconapé, “Neuro-ophthalmologic manifestations of epilepsy,” Seminars in Ophthalmology, vol. 21, no. 4, pp. 255–261, 2006, doi: 10.1080/08820530601006742.

M. A. N. Saporito et al., “Ictal blinking, an under-recognized phenomenon: Our experience and literature review,” Neuropsychiatric Disease and Treatment, vol. 13, pp. 1435–1439, May 2017, doi: 10.2147/NDT.S135979.

T. Mitsuhashi, M. Sonoda, H. Iwaki, K. Sakakura, and E. Asano, “Detection of absence seizures using a glasses-type eye tracker,” Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology, vol. 132, no. 3, pp. 720–722, Mar. 2021, doi: 10.1016/j.clinph.2020.12.015.

N. Barqan, A. Almarri, F. AlSheeb, S. Badawy, K. Abualsaud, and E. Yaacoub, “Early Epilepsy Seizure Monitoring Through Integrated Eye Tracking, Heart Rate and EEG Sensors,” in 2024 IEEE 21st International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and IoT (HONET), Dec. 2024, pp. 91–96. doi: 10.1109/HONET63146.2024.10822941.

D. Ahmedt-Aristizabal et al., “Deep learning approaches for seizure video analysis: A review,” Epilepsy & Behavior, vol. 154, p. 109735, May 2024, doi: 10.1016/j.yebeh.2024.109735.

Królak, A., & Strumiłło, P. (2012). Eye-blink detection system for human–computer interaction. Universal Access in the Information Society, 11(4), 409-419. Available: https://doi.org/10.1007/s10209-011-0256-6

Hong, J., Shin, J., Choi, J., & Ko, M. (2024). Robust eye blink detection using dual embedding video vision transformer. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 6374-6384). Available: https://doi.org/10.1109/WACV57701.2024.00625

Hu, Guilei, et al. "Towards real-time eyeblink detection in the wild: Dataset, theory and practices." IEEE Transactions on Information Forensics and Security 15 (2019): 2194-2208. Available: https://doi.org/10.1109/TIFS.2019.2959978

Xiong, J., Dai, W., Wang, Q., Dong, X., Ye, B., & Yang, J. (2025). A review of deep learning in blink detection. PeerJ Computer Science, 11, e2594. Available: https://doi.org/10.7717/peerj-cs.2594

Sangamithra, B., Sanmitha, V., & Sathvika, S. V. (2025, August). Smart IOT Patient Alert System for Paralysis Care Using Eye Blink Detection. In 2025 11th International Conference on Electrical Energy Systems (ICEES) (pp. 808-813). IEEE. Available: https://doi.org/10.1109/ICEES67011.2025.11213164

Aireddy, H., Sheshadri, K., Kumar, J. P., Harshavardhan, M., Shravani, N., Nanditha, D. N., & Papishetty, N. (2025, December). A Smart Eyewear-Based Solution for Non-Verbal Patient Communication Using Blink Detection. In 2025 IEEE 4th International Conference on Smart Technologies for Power, Energy and Control (STPEC) (pp. 1-5). IEEE. Available: https://doi.org/10.1109/STPEC66316.2025.11490470

C. Liu, A. Torralba, W. T. Freeman, F. Durand, and E. H. Adelson, “Motion magnification,” ACM Trans. Graph., vol. 24, no. 3, pp. 519–526, 2005, doi: 10.1145/1073204.1073223.

V. Mattioli, D. Alinovi, G. Ferrari, F. Pisani, and R. Raheli, “Motion magnification algorithms for video-based breathing monitoring,” Biomedical Signal Processing and Control, vol. 86, p. 105148, Sep. 2023, doi: 10.1016/j.bspc.2023.105148.

P. Agarwal, N. Mishra, A. Deshmukh, and S. Patil, “Contactless Heart Rate Detection Using Eulerian Video Magnification,” in 2024 International Conference on Emerging Smart Computing and Informatics (ESCI), Mar. 2024, pp. 1–5. doi: 10.1109/ESCI59607.2024.10497207.

A. C. Le Ngo and R. C.-W. Phan, “Seeing the Invisible: Survey of Video Motion Magnification and Small Motion Analysis,” ACM Comput. Surv., vol. 52, no. 6, pp. 114:1–114:20, 2019, doi: 10.1145/3355389.

T.-H. Oh et al., “Learning-based Video Motion Magnification.” arXiv, Aug. 2018. doi: 10.48550/arXiv.1804.02684.

Nyström, Marcus, Richard Andersson, Diederick C. Niehorster, Roy S. Hessels, and Ignace TC Hooge. "What is a blink? Classifying and characterizing blinks in eye openness signals." Behavior research methods 56, no. 4 (2024): 3280-3299. Available: https://doi.org/10.3758/s13428-023-02333-9

Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, Goj R, Jas M, Brooks T, Parkkonen L, Hämäläinen M. MEG and EEG data analysis with MNE-Python. Front Neurosci. 2013 Dec 26;7:267. doi: 10.3389/fnins.2013.00267. PMID: 24431986; PMCID: PMC3872725. Available: https://doi.org/10.3389/fnins.2013.00267

Khanam R, Hussain M. What is YOLOv5: A deep look into the internal features of the popular object detector. arXiv preprint arXiv:2407.20892. 2024 Jul 30. Available: https://doi.org/10.48550/arXiv.2407.20892

Tran D, Bourdev LD, Fergus R, Torresani L, Paluri M. Learning spatiotemporal features with 3d convolutional networks. arXiv preprint. arXiv preprint arXiv:1412.0767. 2014. Available: https://doi.org/10.48550/arXiv.1412.0767

Cui, Z., Chen, W., & Chen, Y. (2016). Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995. Available: https://doi.org/10.48550/arXiv.1603.06995

He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. 2016 IEEE Conf. In Comput. Vis. Pattern Recognit (pp. 770-778). Available: https://doi.org/10.1109/CVPR.2016.90

Woo, S., Park, J., Lee, J. Y., & Kweon, I. S. (2018). Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV) (pp. 3-19). Available: https://doi.org/10.1007/978-3-030-01234-2_1

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. Available: http://arxiv.org/abs/1412.6980

Paszke, Adam, et al. "Pytorch: An imperative style, high-performance deep learning library." Advances in neural information processing systems 32 (2019). Available: https://arxiv.org/abs/1912.01703


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ-КОНГРЕСС ВМК МГУ 2026 СНЭ

ISSN: 2307-8162