Robotic Learning & AI Reasoning
Multimodal learning from gaze, muscle activity and motion, with a research agenda in predictive models and grounded reasoning.
Robotic Vision and Intelligence Group
Kenan Niu
University of Twente · Robotics and Mechatronics
We investigate how robots perceive, learn, predict and reason through interaction with the physical world. Our research connects robotic imaging, multimodal learning, spatial understanding and human motion.
Research
Robotic Vision & Imaging
Robotic ultrasound, computer vision, anatomical segmentation and image-guided interventions.
3D Spatial Intelligence
3D reconstruction, spatial sensing, navigation and AR/VR interfaces for robotic and medical systems.
Motion Intelligence
Human movement estimation and biomechanical understanding using ultrasound, sEMG, gaze and kinematic signals.
Selected research outcomes
- Self-derived Motion Features from sEMG for Inferring 3D Forearm Trajectories
Derives motion-relevant features from surface EMG to infer continuous three-dimensional forearm trajectories.
- Hierarchical Transformer Fusion of Gaze Attention and Muscle Activity for Forearm Movement Estimation
Fuses gaze attention and muscle activity hierarchically to estimate forearm movement from complementary human signals.
- Automatic robotic ultrasound for 3d musculoskeletal reconstruction: A comprehensive framework
Integrates robotic scanning, contact control, segmentation and reconstruction into an automatic 3D musculoskeletal imaging workflow.
- HCCE-CUNet Based Multi-Class Musculoskeletal Segmentation for Robotic Ultrasound System
Segments multiple musculoskeletal structures in ultrasound to support automated robotic scanning and reconstruction.
- An autonomous fluoroscopic imaging system for catheter insertions by bilateral control scheme: A numerical simulation study
Studies bilateral control for autonomous fluoroscopic viewpoint management during simulated catheter insertion.
Publications
- Self-derived Motion Features from sEMG for Inferring 3D Forearm Trajectories
B. Lan, D. Sun, I. Tamadon, and K. Niu
IEEE Transactions on Biomedical Engineering · 2026
- HCCE-CUNet Based Multi-Class Musculoskeletal Segmentation for Robotic Ultrasound System
D. Sun, S. Stramigioli, and K. Niu
IEEE Transactions on Medical Robotics and Bionics, 7(4), 1728-1738 · 2025
- Endoscopic sleeve gastroplasty video assessment: do technical features influence ESG integrity and weight loss at 6 and 12 months follow-up?
M Vannucci, D Sun, B Lan, K Niu, A Riba, N Heikens, M Fehervari, ...
Surgical endoscopy 39 (10), 6819-6827 · 2025
- Sex-based differences in musculoskeletal pain among surgeons: an international survey
M Vannucci, MR Rodríguez-Luna, B Lan, K Niu, A Lapergola, E Reitano, ...
Surgical endoscopy 39 (7), 4217-4224 · 2025
- An autonomous fluoroscopic imaging system for catheter insertions by bilateral control scheme: A numerical simulation study
G. Y. Ward, D. Sun, and K. Niu
Machines, 13(6), 498 · 2025
- Hierarchical Transformer Fusion of Gaze Attention and Muscle Activity for Forearm Movement Estimation
B. Lan, S. Stramigioli, and K. Niu
IEEE Transactions on Biomedical Engineering, 72(12), 3674-3681 · 2025
- Comparable performance between automatic and manual laryngeal and hypopharyngeal gross tumor volume delineations validated with pathology
KM Kuijer, HJG Smits, PAH Doornaert, K Niu, MHF Savenije, EJ Smid, ...
International Journal of Radiation Oncology* Biology* Physics 122 (1), 186-193 · 2025
- Deciphering Muscular Dynamics: A Dual-Attention Framework for Predicting Muscle Contraction from Activation Patterns
B Lan, G Krijnen, S Stramigioli, K Niu
IEEE journal of biomedical and health informatics · 2025
- Automatic robotic ultrasound for 3d musculoskeletal reconstruction: A comprehensive framework
D. Sun, A. Cappellari, B. Lan, M. Abayazid, S. Stramigioli, and K. Niu
Technologies, 13(2), 70 · 2025
- The Study of Relationship Between Muscle Activation and Deformation During Arm Motion
B Lan, K Niu
10th Dutch Biomedical Engineering Conference, BME 2025 · 2025
- Automatic Robotic Ultrasound Scanning for Muscle Segmentation and Reconstruction
D. Sun, A. Cappellari, B. Lan, and K. Niu
10th Dutch Biomedical Engineering Conference · 2025
- Muscle Activation–Deformation Correlation in Dynamic Arm Movements
B. Lan, D. Sun, and K. Niu
J: Multidisciplinary Scientific Journal · 2025
- Advanced Robotic Ultrasound Imaging for Spine Surgery
R Li, E Vander Poorten, K Niu
2024
- Predicting muscle thickness deformation from muscle activation patterns: A dual-attention framework
B Lan, K Niu
arXiv preprint arXiv:2409.18266 · 2024
- Controlling an MR safe robot arm with six pneumatic stepper motors using eight tubes
V Groenhuis, M Bac, S Stramigioli, K Niu
ICRA@ 40 special conference 2024: 40th Anniversary of the IEEE International… · 2024
- SIRC-UNet: Improving bone tracking precision of a-mode ultrasound by decoding hierarchical resolution features
B Lan, M Abayazid, N Verdonschot, S Stramigioli, K Niu
IEEE Sensors Journal 24 (22), 38174-38184 · 2024
- Augmented-reality based digital twin to control an MR safe robot for breast biopsy: A benchmark study
L Marx, V Groenhuis, S Stramigioli, K Niu
2024 10th IEEE RAS/EMBS International Conference for Biomedical Robotics and… · 2024
- Anatomical region recognition and real-time bone tracking methods by dynamically decoding a-mode ultrasound signals
B Lan, S Stramigioli, K Niu
2024 10th IEEE RAS/EMBS International Conference for Biomedical Robotics and… · 2024
- Bilateral Control Scheme for Surgical Robotic System Collaboration
GY Ward
University of Twente · 2024
- A pathologically validated deep learning model for laryngeal and hypopharyngeal tumour segmentation on MRI
K Kuijer, H Smits, P Doornaert, K Niu, M Savenije, E Smid, C Terhaard, ...
EUROPEAN JOURNAL OF CLINICAL INVESTIGATION 54 · 2024
- Deep Learning based acoustic measurement approach for robotic applications on orthopedics
B Lan, M Abayazid, N Verdonschot, S Stramigioli, K Niu
2024 IEEE International Conference on Robotics and Automation (ICRA), 921-927 · 2024
- Pathologically validated deep learning model for laryngeal and hypopharyngeal GTV delineation on MRI
KM Kuijer, HJG Smits, PAH Doornaert, K Niu, EJ Smid, CHJ Terhaard, ...
RADIOTHERAPY AND ONCOLOGY 194, S3102-S3104 · 2024
- 2411: Pathologically validated deep learning model for laryngeal and hypopharyngeal GTV delineation on MRI
KM Kuijer, HJG Smits, PAH Doornaert, K Niu, EJ Smid, CHJ Terhaard, ...
Radiotherapy and Oncology 194, S3101-S3104 · 2024
- A method to track 3d knee kinematics by multi-channel 3d-tracked a-mode ultrasound
K Niu, V Sluiter, B Lan, J Homminga, A Sprengers, N Verdonschot
Sensors 24 (8), 2439 · 2024
- State-of-the-art of non-radiative, non-visual spine sensing with a focus on sensing forces, vibrations and bioelectrical properties: A systematic review
M Timmermans, A Massalimova, R Li, A Davoodi, Q Goossens, K Niu, ...
Sensors 23 (19), 8094 · 2023
- Robot-assisted ultrasound reconstruction for spine surgery: from bench-top to pre-clinical study
R Li, A Davoodi, Y Cai, K Niu, G Borghesan, N Cavalcanti, A Massalimova, ...
International journal of computer assisted radiology and surgery 18 (9… · 2023
- Physiological motion compensation for neuroscience research based on electrical bio-impedance sensing
Y Zhang, E Verschooten, M Ourak, K Van Assche, G Borghesan, D Wu, ...
IEEE Sensors Journal 23 (20), 25377-25389 · 2023
- Development of robot-assisted ultrasound system for fetoscopic tracking in twin to twin transfusion syndrome surgery
Y Cai, A Davoodi, R Li, M Ourak, K Niu, J Deprest, E Vander Poorten
2023 International Symposium on Medical Robotics (ISMR), 1-7 · 2023
- Robot-assisted Ultrasound Reconstruction for Spine Surgery: from Bench-top to Pre-clinical Study
R Li, A Davoodi, Y Cai, K Niu, G Borghesan, N Cavalcanti, A Massalimova, ...
TechRxiv 2023 (0109) · 2023
- A Comparative Study for Control of Semi-Automatic Robotic-assisted Ultrasound System in Spine Surgery
A Davoodi, R Li, Y Cai, K Niu, G Borghesan, E Vander Poorten
the Proceedings of 2023 International Conference on Advanced Robotics (ICAR… · 2023
- RL-Based Guidance in Outpatient Hysteroscopy Training: A Feasibility Study
V Poliakov, K Niu, E Vander Poorten, D Tsetserukou
2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2171… · 2022
- Feasibility of using an Extended Kalman Filter for Motion Estimation based on Electrical Bioimpedance Sensing
Y Zhang, K Van Assche, M Ourak, E Verschooten, G Borghesan, N Kenan, ...
IROS 2022 workshop: A Panacea or an Alchemy?——Benefits and Risks of Robot… · 2022
- An augmented reality-based interaction scheme for robotic pedicle screw placement
V Vörös, R Li, A Davoodi, G Wybaillie, E Vander Poorten, K Niu
Journal of imaging 8 (10), 273 · 2022
- An in-office hysteroscopy vr/haptic simulation platform for training in spatial navigation and passage of the cervical canal
V Poliakov, K Niu, D Tsetserukou, E Vander Poorten
IEEE Transactions on Medical Robotics and Bionics 4 (3), 608-620 · 2022
- Deep-learning-based compliant motion control of a pneumatically-driven robotic catheter
D Wu, XT Ha, Y Zhang, M Ourak, G Borghesan, K Niu, F Trauzettel, ...
IEEE Robotics and Automation Letters 7 (4), 8853-8860 · 2022
- 3d reconstruction of local anatomy based on 2d ultrasound for fetal laser surgery
Y Cai, MA Ahmad, M Ourak, R Li, K Niu, T Vercauteren, S Ourselin, ...
11th Conference on New Technologies for Computer and Robot Assisted Surgery… · 2022
- Automatic robotic scanning for real-time 3d ultrasound reconstruction in spine surgery
R Li, A Davoodi, Y Cai, E Vander Poorten
11th Conference on New Technologies for Computer and Robot Assisted Surgery… · 2022
- Controlling of a micropositioning piezoelectric actuator using an LSTM network for robotic eye surgery
Y Zhang, M Ourak, E Verschooten, K Niu, D Wu, G Borghesan, P Joris, ...
Conference on New Technologies for Computer and Robot Assisted Surgery… · 2022
- Quantitative assessment of calibration motion profiles in robotic-assisted ultrasound system
M Schoovaerts, R Li, K Niu, E Vander Poorten
2022 International Symposium on Medical Robotics (ISMR), 1-7 · 2022
- Deep-learning-based position control of a robotic catheter under environmental contact
D Wu, Y Zhang, M Ourak, XT Ha, K Niu, J Dankelman, E Vander Poorten
2022 International Symposium on Medical Robotics (ISMR), 1-7 · 2022
- Comparative Quantitative Analysis of Robotic Ultrasound Image Calibration Methods
R Li, Y Cai, K Niu, E Vander Poorten
20th International Conference on Advanced Robotics (ICAR) · 2022
- A Framework for Fast Automatic Robot Ultrasound Calibration
R Li, K Niu, E Vander Poorten
2021 International Symposium on Medical Robotics (ISMR) · 2022
- A method based on 3D shape analysis towards the design of flexible instruments for endoscopic maxillary sinus surgery
J Legrand, K Niu, Z Qian, K Denis, V Vander Poorten, L Van Gerven, ...
Annals of biomedical engineering 49 (6), 1534-1550 · 2021
- Hysteresis modeling of robotic catheters based on long short-term memory network for improved environment reconstruction
D Wu, Y Zhang, M Ourak, K Niu, J Dankelman, E Vander Poorten
IEEE Robotics and Automation Letters 6 (2), 2106-2113 · 2021
- A uniaxial force and stiffness model of the vagina during laparoscopic sacrocolpopexy
J De Smet, A Page, J Deprest, S Housmans, K Niu, E Vander Poorten
Clinical Biomechanics 81, 105204 · 2021
- Towards modeling of hysteresis in robotic catheters based on LSTM
D Wu, M Ourak, K Niu, Y Zhang, MA Ahmad, J Dankelman, ...
32nd Conference of the International Society for Medical Innovation and… · 2020
- Characterizing vaginal vault interaction during laparoscopic sacrocolpopexy
A Page, J De Smet, J Deprest, S Housmans, K Niu, E Vander Poorten
International Urogynecology Journal 31 (SUPPL 1), S241-S242 · 2020
- A virtual reality surgical training system for office hysteroscopy with haptic feedback: A feasibility study
V Poliakov, K Niu, BP De Vree, D Tsetserukou, E Vander Poorten
International Conference on Virtual Reality and Augmented Reality, 115-127 · 2020
- VR Simulation System for In-Office Hysteroscopy: A Pilot Study
D Tsetserukou, K Niu, E Vander Poorten
10 th Conference on New Technologies for Computer and Robot Assisted Surgery… · 2020
- A framework of real-time freehand ultrasound reconstruction based on deep learning for spine surgery
R Li, K Niu, D Wu, E Vander Poorten
10 th Conference on New Technologies for Computer and Robot Assisted Surgery… · 2020
- A Framework of Real-time Freehand Ultrasound Reconstruction based on Deep Learning for Spine Surgery
L Ruixuan, K Niu, D Wu, E Vander Poorten
Conference on New Technologies for Computer and Robot Assisted Surgery · 2020
- Feasibility of using a Long Short-Term Memory Network for Robotic Catheter Control
D Wu, M Ourak, MA Ahmad, N Kenan, G Borghesan, J Dankelman, ...
10 th Conference on New Technologies for Computer and Robot Assisted Surgery… · 2020
- Statistical shape modelling of the human nasal cavity and maxillary sinus for minimally invasive surgery
K Niu, J Legrand, L Van Gerven, E Vander Poorten
the 10th Conference on New Technologies for Computer/Robot Assisted Surgery… · 2020
- Evaluating the potential benefit of autostereoscopy in laparoscopic sacrocolpopexy through VR simulation
J De Smet, V Poliakov, K Niu, F Chesterman, J Fornier, MA Ahmad, ...
2019 19th International Conference on Advanced Robotics (ICAR), 566-571 · 2019
- A novel ultrasound-based lower extremity motion tracking system
K Niu, V Sluiter, J Homminga, A Sprengers, N Verdonschot
Intelligent Orthopaedics: Artificial Intelligence and Smart Image-guided… · 2018
- Towards functional pre-planning in orthopaedic surgery
N Verdonschot, A Sprengers, V Mazzoli, M Marra, K Gijsbertse, H Chen, ...
Virtual Physiological Human Conference 2018 · 2018
- Measuring relative positions and orientations of the tibia with respect to the femur using one-channel 3D-tracked A-mode ultrasound tracking system: A cadaveric study
K Niu, J Homminga, V Sluiter, A Sprengers, N Verdonschot
Medical engineering & physics 57 (1), 61-68 · 2018
- Feasibility of a-mode ultrasound based intraoperative registration in computer-aided orthopedic surgery: a simulation and experimental study
K Niu, J Homminga, VI Sluiter, A Sprengers, N Verdonschot
Plos One · 2018
- In situ comparison of A-mode ultrasound tracking system and skin-mounted markers for measuring kinematics of the lower extremity
K Niu, T Anijs, V Sluiter, J Homminga, A Sprengers, MA Marra, ...
Journal of biomechanics 72, 134-143 · 2018
- Ultrasound based skeletal motion capture: the development and validation of a non-invasive knee joint motion tracking method
K Niu
University of Twente · 2018
- A novel tibiafemoral kinematics measurement system based on multi-channel a-mode ultrasound system
K Niu, V Sluiter, A Sprengers, J Homminga, N Verdonschot
17th Annual Meeting of the International Society for Computer Assisted… · 2017
- The relation between the number of a-mode ultrasound sampled points and the registration accuracy and robustness on computer aided orthopedic surgery
K Niu, JJ Homminga, VIJ Sluiter, AMJ Sprengers, NJJ Verdonschot
5th Dutch Bio-Medical Engineering Conference, BME 2015 · 2015
- 基于空间定位的经皮肺穿刺图像引导关键技术研究与实现
牛珂楠
东北大学 · 2013
- A medical image navigating system with iPad
K Niu, N Bao, Y Kang
2012 International Conference on Computerized Healthcare (ICCH), 5-8 · 2012
- Address Mapping Algorithm and the Design of the LYSO Detector in PET
K Niu
Beijing Biomedical Engineering 30 (5), 484-489 · 2011
- LYSO 闪烁晶体 PET 探测器设计及位置映射算法研究
牛珂楠， 燕辉， 项士海， 王军博， 孙航， 刘军强， 梁国栋， 吴国城， 尚艺童， ...
北京生物医学工程 30 (5), 484-489 · 2011
- A Preliminary Study of the Pulse Oximetry for Early Breast Cancer Detection
B. Lan, E. Juffermans, I. Tamadon, and K. Niu
10th Dutch Biomedical Engineering Conference
- Deep-Learning-Based Compliant Motion Control of a Pneumatically-Driven Robotic Catheter
M Ourak, G Borghesan, K Niu, F Trauzettel, J Dankelman, A Menciassi, ...
- Hysteresis Modeling of Robotic Catheters based on Long Short-Term Memory Network for Improved Environment Reconstruction
E Vander Poorten
- A-MODE ULTRASOUND-BASED INTRAOPERATIVE REGISTRATION ACCURACY ANALYSIS BASED ON SIMULATION ENVIRONMENT
K Niu, J Homminga, N Verdonschot
Contact & supervision
Research collaborations and BSc, MSc and doctoral projects in robotic intelligence, imaging and motion.
k.niu@utwente.nl · Education & student projects