AR (Augmented Reality) and VR (Virtual Reality) are still emerging technologies as we move into a new age of computing. As TSSG have been involved from the early stages of development on the latest platforms in this technological area, we have noticed a growing need in the field for developing new innovative technologies to interact with this new mixed reality world. Sensing and understanding the real world to map digital elements and functionality onto the projected virtual world requires the use of the latest state of the art sensing technologies and, as such, the technology fits quite well under the TSSG group ‘MEPS’ (Mobile Ecosystem and Pervasive Sensing).
With the current industry focus and demands, we are driven towards creating exciting and novel approaches to the latest challenges within the health sector, digital manufacturing, digital twin technologies for construction, intelligent assistive technologies and training. We strive to create the leading approaches to technical challenges in these fields using our talented group of developers to find solutions. A key aspect to drive the growth of our development expertise is our focus on research in the AR/VR area. This research leads us to develop new methodologies and techniques to aid the development of commercial solutions.
The core aim of our research is to utilize the technology present within the AR/VR ecosystem. The focus within the AR/VR group currently follows two research strands:
Research Focus Area 1: AR/VR Perception
How the user perceives and interacts with the virtual/augmented world is key to understanding how we can adapt the contents of this environment to allow for more immersive experiences and unique interaction techniques while in the environment. By monitoring the user with the virtual world using EEG and other physiological measures, we can understand how they are currently perceiving the simulated world. By understanding how their brain network reacts to this in both emotional/mental states as well as in the form of graph theory we can begin to recognize distinct brain pattern activity within AR/VR. By utilising deep learning to predict and make inferences we can use these predictions to monitor how the brain will react to certain stimulus. This allows us to create very specific content that is adaptable and modifiable to the user themselves and allows us to alter the way in which we experience virtual/augmented reality by causing subtle brain network changes. This technology builds on top of research within the neurorehabilitation field ultimately allowing for treatment of patients of neurological disorders such as stroke, depression and Parkinson’s.
This focus area includes the following research elements:
- Neural Stimulation
- Virtual Neural Rehabilitation
- Visual Processing
- Brain perceives emotional/mental states
- Graph Theory of brain networks
- Dynamic Adaptive content
Research Focus Area 2: Social Cognition Training using AR/VR
Impaired social communication represents a key obstacle facing people with ASD (Autism Spectrum Disorder); this includes response to social stimuli as well as appropriate navigation of natural social settings. Deficiencies in social interactive skills – which are prevalent in people with ASD from early childhood – greatly impair a person’s social development, leading to increased difficulties such as employability, progress in education, access to social services and mental health issues such as depression. There have been several initiatives for people with ASD which have targeted the area of enhancing social cognisance and social interactions with the envisaged outcome of improving their quality of life as well as their employability prospects to help them navigate real life situations.
The primary application of this research would be the use of AR/VR as a basis for social skills training – to help people with ASD to develop communication skills and social cognisance. The user is presented with virtual content which represents a real-life scenario, including virtual avatars of other people. Their response to this content can then be measured by the AR/VR application including the ability to make and retain eye contact, time taken to respond to greetings as well as level of anxiety in social situations.
Future Research Areas of Interest.
There is the potential to leverage AR/VR technology with the other current thematic research areas in TSSG including:
- Gait Analysis/ Gait Tracking
- Kinetic Pose Estimation
- Brain research (occipital function / motor cortex function)
- AR Communications – 3D Mesh Reconstruction
- Computer Vision Fundamentals