Sept. 28, 2016
Computer science researchers create augmented reality education tool
Christian Jacob and Markus Santoso developed HoloCell.
Markus Santoso on YouTube
In the film Fantastic Voyage, a brilliant scientist develops a way to shrink objects 鈥 and people 鈥 to the size of a micrometer. In an effort to save his life, a group of intrepid CIA agents and medical experts are shrunk and injected into the scientist, travelling through his body to locate and destroy a blood clot in the scientist鈥檚 brain.
When the movie was released in 1966, this concept existed in science fiction alone. Researchers in the Department of Computer Science at the 91快色, however, are pushing the limits of augmented reality (AR) technology and exploring its practical uses in health care and education.
and Markus Santoso are trying to re-create the experience of the aforementioned agents in Fantastic Voyage. Working with 3D modelling company , they and recent MSc graduate Douglas Yuen have created , an educational software. Using Microsoft鈥檚 revolutionary HoloLens AR glasses, HoloCell provides a mixed reality experience allowing users to explore a 3D simulation of the inner workings, organelles, and molecules of a healthy human cell.
HoloCell allows users to explore a 3D simulation of a human cell using Microsoft's HoloLens glasses.
Jacob has plenty of experience in bioinformatics as the head of the . By combining forces with Santoso, an Eyes High Postdoctoral Fellow with an extensive background in AR research, the pair are taking interactive education to the next level. 鈥淲e鈥檙e finessing the cell right now, but eventually we will expand this software to include the entire body,鈥 explains Jacob, who says that upscaling the project will be relatively straightforward and will eventually become a widely-used educational tool.
Jacob cites oncology as an example of one of the practical purposes of AR technology, a field into which he and Santoso 鈥 together with Wellness Computational, a 91快色-based company that develops computational medical illustrations and health analytics solutions 鈥 are currently expanding their work through . 鈥淟et鈥檚 say a doctor and a patient are talking about a cancer diagnosis. The doctor can have a more informative conversation with the patient when they can both put on the AR glasses and see the same hologram of a tumour,鈥 he says.
鈥淲e are so used to a screen, a flat surface. But we can put a lot more information into a 3D space than into a 2D space. It changes the whole paradigm of interacting with information and interacting with computers as well,鈥 explains Jacob. 鈥淏eing in computer science, we always try to push the limits. There鈥檚 always new technology and we try to get as much out of it as we can, or more out of it than what it was built for.鈥
Arash Minhas experiences the HoloLens and HoloCell during Beakerhead's Campus Collisions.
This past weekend, Santoso touted the AR research environment at the 91快色 as one of the keynote speakers at the Augmented World Expo (AWE) Asia 2016, the world鈥檚 largest AR conference and expo, in Xi'an, China.
鈥淟ately, it seems like our work is getting a lot of recognition,鈥 he says. 鈥淚鈥檓 so happy to be conducting my research here. Although I am still a fairly junior academic, I am getting the opportunity to speak at such a major event.鈥
Santoso, who has been researching AR through some of the world鈥檚 most innovative institutions since 2011, says that Jacob鈥檚 lab at the 91快色 is already making strides toward becoming a leader in AR software development. 鈥淚t takes time to really send this technology forward for commercial purposes,鈥 he says, 鈥渂ut we鈥檙e trying to use future technology to get ahead so that by the time everyone gets used to it, we鈥檒l already be prepared with the next steps forward.鈥
Markus Santoso uses a smartphone to turn a regular medical textbook into a 3D learning tool.