A team of UCF researchers has won an industry award for using robotics and digital twins to blend human intuition with technological innovation.
Their work, “Be There Without Being There: Humanoid Robot Immersive Telepresence,” won an Auggie Award for the Best Interaction Product at this year’s Augmented World Expo in Long Beach, California. The team is comprised of computer science student researchers Judah Rowe ’23 ’25MS, junior Liam Abramov, and seniors SeNiah McField and Megan Bailey; Assistant Professor Mohsen Rakhshan; and led by UCF Institute for Simulation and Training Director and Agere Chair Professor of Computer Science Carolina Cruz-Neira.
The group developed a system that integrates a humanoid robot and a remote immersive environment that enables a human to interact as if it were in the robot’s location, creating a powerful immersive experience that gives a person the ability to interact as the robot. The human, digitally mapped to the robot, is able to engage in distant surroundings as the robot relays a digital twin rendering, in real time, to an immersive environment.
“From a technical significance, this is the perhaps the first working product or system that actually puts a human to be physically functional in a remote location, as if they were really there,” Cruz-Neira says. “We map the body of the human to the robot. So the human walks, the robot walks. The human moves, and the robot moves. The human turns the head, the robot turns the head. So it’s almost like the concept of the movie Avatar.”
She adds that the award highlights how talented and amazing UCF students are for their successful implementation of physical telepresence.
“This project was done entirely by UCF students,” she says. “Dr. Rakhshan and I provided direction of course but they are the ones that solved all the problems, and they brought it to this level of sophistication to win the award.”
The ability to interact in another environment entirely while physically being located elsewhere opens doors to infinite possibilities. Robots can physically be present in areas or situations that may not be safe for people, such as in the aftermath of a natural disaster, to assess the damage or aid in locating survivors. Using this technology, a healthcare provider can offer services to patients who may not be able to visit their office. A trip to Paris for those physically unable to make the journey can spend the afternoon visiting the Champs-Élysées.
Cruz-Neira emphasizes that the most important aspect of the work is its ability to blend human intuition with technological innovation.
“At the end of the day, we have a human element, and robots can also help us to extend our human capabilities through this amazing technology,” she says. “This project is about how we can use technology to augment or extend our human capabilities and help us to do more things or to do things safer or to do things more effectively. So it’s kind of the right balance of human and technology kind of working together.”
The team will continue to refine the technology, adding features to enhance the human experience. They plan to incorporate artificial intelligence to give the robot a level of autonomy. The group will also continue working with Rakhshan to give the robot a sense of touch that can be experienced by the human, giving people the ability to experience temperatures, sense textures and manipulate objects remotely.
Cruz-Neira says with this project, the students have demonstrated an impressive level of robustness in a working prototype, carrying the project from the lab to the real world in an exceptional display of its value to the community.
“You have a university, UCF, that is making a mark in the world as the as the technology university,” she says. “And this project demonstrates that as a university, we are part of the future that we like to talk about, on how the universities can have real impact.”
- Written by Bel Huston