Seeing the unseen: Augmented reality for stroke rehabilitation

After a stroke, some people experience spatial neglect, a condition in which the brain has difficulty paying attention to one side of the body and the space around it. A person may leave food untouched on one side of a plate, bump into objects, miss words on part of a page, or fail to notice one side of their own body. This is not simply a problem with eyesight; it is caused by a disruption in how the brain directs attention. Spatial neglect can make everyday activities such as eating, dressing, reading and moving around much harder, which reduces independence and slows recovery. Led by Associate Professor Alan Wang at the Auckland Bioengineering Institute, University of Auckland, this project is exploring how augmented reality and intelligent technologies could provide a more engaging and personalised approach to rehabilitation. The work is being carried out in collaboration with Professor Monika Harvey and colleagues at the University of Glasgow, Scotland bringing together expertise in stroke rehabilitation, cognitive neuroscience, artificial intelligence and immersive technology.

Augmented reality,(AR) adds digital information and interactive objects to the real world. In rehabilitation, AR can encourage users to look, reach and respond towards the side they tend to overlook while completing game-like activities in a familiar environment. During the Catalyst: Seeding project, the team first developed an early prototype called awARe. Feedback from public demonstrations helped improve the design and usability, and the work later progressed into ARBingo, a more advanced platform combining AR with intelligent technologies. ARBingo can track how a user responds during an activity, monitor patterns of visual attention and adjust aspects of the task according to performance. It also incorporates speech interaction, eye-movement monitoring and real-time behavioural tracking. Together, these features provide new ways to understand how people explore their surrounding space, where difficulties occur and how attention may change during rehabilitation.

Project team members Associate Professor Alan Wang (right), Professor Monika Harvey (centre) and PhD candidate Arthur (Shaojun) Li (left) at a collaborative research meeting during Professor Harvey’s visit to the University of Auckland. Photo: Supplied.

An initial pilot test showed that the platform was engaging, easy to use and technically reliable. Users successfully completed the activities, and their feedback helped the team refine the system further. These encouraging results have provided a strong foundation for the next stage of evaluation in stroke rehabilitation settings. Catalyst: Seeding funding has also supported reciprocal research visits, workshops, joint research activities, student training and conference demonstrations, helping to establish a lasting international partnership between the Universities of Auckland and Glasgow.

The next step is to work closely with clinicians and stroke rehabilitation teams at Auckland City Hospital and Middlemore Hospital. This co-design process will help ensure that the technology is practical, culturally responsive and relevant to the needs of stroke survivors, whānau and clinicians. The long-term goal is to create a personalised rehabilitation tool that can support therapy in clinics and, eventually, in people’s homes. By combining AR, intelligent data analysis and clinical expertise, the team hopes to make spatial neglect rehabilitation more engaging, measurable and accessible, while giving stroke survivors more opportunities to practise safely and independently.