NASA and Microsoft will team up to develop the new project, named Sidekick, with a pair of HoloLenses scheduled to launch on SpaceX’s seventh commercial resupply mission to the station on June 28.
Sam Scimemi, director of the ISS program at NASA Headquarters, said:
“HoloLens and other virtual and mixed reality devices are cutting edge technologies that could help drive future exploration and provide new capabilities to the men and women conducting critical science on the International Space Station. This new technology could also empower future explorers requiring greater autonomy on the journey to Mars.”
This new capability could reduce crew training requirements and increase the efficiency at which astronauts can work in space.
HoloLens, Microsoft’s answer to Google Glass, is a pair of goggles that projects holographic images into the wearer’s line of vision. The new capability could reduce space station crew training requirements and increase the efficiency at which astronauts can work in space.
“Microsoft HoloLens is about transforming the ways you create, connect, and explore,” said Alex Kipman, technical fellow, Windows and Devices Group at Microsoft. “Sidekick is a prime example of an application for which we envisioned HoloLens being used – unlocking new potential for astronauts and giving us all a new perspective on what is possible with holographic computing.”
Project Sidekick includes two modes of operation.
The first is “Remote Expert Mode.” It uses Microsoft-owned Skype, to enable a ground operator to see what a crew member sees, provide real-time guidance, and draw annotations into the crew member’s environment to coach him or her through a task.
Until now, crew members have relied on written and voice instructions when performing complex repair tasks or experiments.
The second mode is “Procedure Mode.” It augments standalone procedures with animated holographic illustrations displayed on top of the objects with which the crew is interacting, a sort of virtual blueprint or Ikea instruction set as it were.
This capability could lessen the amount of training that future crews will require and could be an invaluable resource for missions deep into our solar system, where communication delays complicate difficult operations.
Project Sidekick and the Microsoft HoloLens were tested by NASA and Microsoft engineers onboard NASA’s Weightless Wonder C9 jet to ensure they function as expected in free-fall in advance of their delivery to the microgravity environment of the space station.
The Sidekick project is part of a bigger partnership formed by NASA and Microsoft to investigate applications of holographic computing in space exploration.