What you need to know
- Microsoft Mesh is a new mixed reality platform that has been years in the making.
- Microsoft Mesh uses Azure, mixed reality, and 3D capture technology to beam a person's lifelike image into a virtual scene.
- Microsoft Technical Fellow Alex Kipman demoed the tech live at Microsoft Ignite.
- James Cameron, Niantic, and Guy Laliberté of Cirque du Soleil are working with Microsoft on the tech.
Microsoft Mesh: 'Holoportation' is getting real
In 2016, Microsoft broke the ice on its new HoloLens mixed reality platform, often demoing potential features that could arrive someday. One of those features was 'holoportation', the ability to be in a shared space with someone else virtually. The concept seemed straight out of Star Trek. Although only a rough demo, it was just two years later Microsoft showed advancements in the idea at Ignite 2019.
Today, holoportation becomes even more real as the Redmond software giant announces Microsoft Mesh – a new mixed-reality platform powered by its Azure cloud service. Like its previous holoportation concepts, Microsoft Mesh relies on "3D capture technology to beam a lifelike image of a person into a virtual scene."
Microsoft Mesh is the result of years of Microsoft research and development in creating persistent holograms.
The announcement came via a live demonstration of the platform at Microsoft Ignite delivered by Microsoft Technical Fellow Alex Kipman in real-time along with its partners Lune Rouge (Guy Laliberté), Niantic (John Hanke), and OceanX OceanX (Ray Dalio).
On stage, Microsoft and its partners demoed how Microsoft Mesh can enable new abilities and remote collaboration. James Cameron, the famous Hollywood filmmaker, discussed Ray Dalio's OceanX non-profit that "merges cutting edge science with compelling storytelling" to support ocean education and awareness. Collaborating with Microsoft, OceanX is able "to create a Mesh-enabled holographic laboratory" on its research vessel, OceanXplorer. Scientists on the OceanXplorer ship will be able to gather – either in-person or virtually- from labs around the world to see 3D holograms of the areas they're exploring."
Other demos from the CEO and founder of Niantic included a proof-of-concept around Pokémon GO that runs on HoloLens 2. Additionally, Guy Laliberté, founder of Lune Rouge and Cirque du Soleil, spoke about how Microsoft Mesh helped his company bring to life a project called Hanai World that connects "live and digital entertainment experiences into single events."
Turning back to Microsoft's primary mission of productivity, Mesh is also expected to play a significant role in digitizing future workspaces with the benefit of Azure's enterprise-grade security and privacy features:
Microsoft Mesh will also enable geographically distributed teams to have more collaborative meetings, conduct virtual design sessions, assist others, learn together and host virtual social meetups. People will initially be able to express themselves as avatars in these shared virtual experiences and over time use holoportation to project themselves as their most lifelike, photorealistic selves.Architects and engineers could physically walk through a holographic model of a factory floor under construction, seeing how all the pieces of equipment fit together in three dimensions, potentially avoiding costly mistakes.Engineering or medical students learning about electric car engines or human anatomy could gather as avatars around a holographic model and remove parts of the engine or peel back muscles to see what's underneath. Colleagues could simply get together and chat in a shared virtual space, or companies could use Microsoft Mesh-enabled apps to offer virtual all-hands meetings or trainings to employees around the world.
Of course, the future of actual holograms is not quite here yet, as the company noted people "will initially be able to express themselves as avatars" at first, and then, over time, "use holoportation to project themselves as their most lifelike, photorealistic selves."
Microsoft Mesh: Coming soon (and open standards)
Microsoft Mesh is not just another concept. The new platform brings many of these new features – including "a full suite of AI-powered tools for avatars, session management, spatial rendering, synchronization across multiple users and holoportation" – in the coming months.
Although Microsoft's own HoloLens 2 mixed-reality headset is mainly featured with holoportation, the company noted the platform has "open standards," It will work across many devices, including other virtual reality headsets, smartphones, tablets, and even PCs.
Additionally, Microsoft is announcing the availability of the new Microsoft Mesh app for HoloLens and a new Mesh-enabled AltspaceVR (which Microsoft acquired in late 2017). AltspaceVR with Microsoft Mesh "will allow companies to hold meetings and work gatherings in virtual reality with enterprise-grade security features including secure sign-ins, session management, and privacy compliance."
Microsoft is also letting developers create new applications for the platform and plans to integrate the technology into Microsoft Teams and Dynamics 365.
HoloLens, but better
HoloLens 2 takes everything that made the original great and turns it up to 11. With a new carbon-fiber body, extra padding, eye tracking, and a wider field of view, the headset should have no problem finding success among developers and first-line workers.
