Hands-on: Varjo Teleport is a comprehensive tool for scanning environments, and it is out now

Today Varjo has officially launched Teleport, its comprehensive solution for environment scanning and reconstruction using Gaussian Splats. I had the pretty unique occasion of trying it, so in this article, I will not only tell you the news of the day but also give you my hands-on analysis of this product. There’s a lot to unfold, so let’s start!

[Disclaimer: VR journalists love to write disclaimers lately, so let me tell you that Varjo didn’t pay my trip to Helsinki because I was already there thanks to the kind invite of the Helsinki XR Center for MatchXR. But the Varjo people offered me some chocolate. And I love chocolate… so you know, if they offered me money I wouldn’t have cared, but chocolate… this article will surely be biased because of this]

Varjo Teleport

Varjo is one of the most famous manufacturers of professional VR headsets. Its headsets have always been characterized by very high resolution and, for instance, the XR-4 Focal Edition that I tried lately has an unprecedented definition of the passthrough.

Today it has just launched a new product, that is not a device, but a service called Varjo Teleport. The purpose of Teleport is to make you scan an environment with your phone so that you can later enter into it with whatever device you have, including VR headsets for full immersion. The idea is to digitalize an environment so that later you or someone you know can teleport into it (hence the name). Varjo Teleport uses Gaussian Splatting for the rendering of the places: that is a technology that is getting a lot of attention lately.

I already had a preview of the service at AWE US this year and I found it promising, but still a bit rough around the edges. Today Varjo is launching this service for everyone to use and it’s time for me to tell you everything you need to know about it.

How to scan the environments

Let me start the description of Varjo Teleport with the first step of the pipeline, which is the creation of the content. Speaking with Varjo engineers, I understood that they put a lot of attention on this part because they wanted to make Teleport accessible for everyone. This means that the ability to scan an environment shouldn’t be possible only for a professional, but that anyone can scan any place (of a moderate size) in 5-10 minutes. Ease of use is of paramount importance and this is one of the main selling points of Varjo Teleport.

To scan a place you currently must have an iPhone (because of the LiDAR). You launch the Varjo Teleport app and you start moving around the place you are in, recording the environment from different points of view. After you have done that, you upload your scan to the cloud and you let Varjo servers do the magic of converting your video scan into an environment people can enter into.

The number of scans you can perform depends on the subscription tier you are paying to Varjo.

Hands-on environment scanning

Scanning a place with the Teleport app (Image by Varjo)

People at Varjo were telling me how easy it is to make a scan with Teleport, so I said: “ok, cool… if anyone can make a scan, let me do a scan now! I’m not a photogrammetry expert, so I can be a good test”. After a few seconds of panic on their faces, one of the employees (a super kind guy, btw) handed me his phone and said “Sure”. I so proceeded to scan the room with all the Varjo employees in: they had to stay still for five minutes like statues to not ruin my results. It was fun (for me), but I guess I will never be invited to Varjo again.

By testing the app, I can confirm that Varjo is doing its best to make the app user-friendly. The app contains a short guide with images and short texts giving you instructions about how to take the scan, with clear suggestions on how to make sure that the resulting environment comes up with good quality. I skimmed through this guide pretty quickly, then I proceeded to move around the room with the phone in my hands.

I’m not used to scanning places, so it was a good test for me. It’s interesting that if you have rarely done this kind of operation, you truly do not know where to point your phone… there are too many degrees of freedom. I so started to move in circles around the room, putting the phone at a different height for every tour I was making (this was one of the suggestions of the app). I also tried to spend more time around relevant objects (like the flight simulator chair in the room), because the short guide suggested framing objects from many points of view so that they had a better quality especially when seen in VR. I noticed I was not very good at scanning, though: the app constantly warned me that I was moving too fast; then the Varjo employee told me that I should not frame the wall close to me, because there are not enough reference points, but it is better to frame always the wall that is on the opposite side, so the system has a wider view of the elements in the room; I also forgot to properly scan the floor and the ceiling and I just did this roughly.

The interface I could see on the phone resembled the one of the camera app when you record a vertical video. The only difference was that on the top of the screen, there was a counter of the frames that the app was capturing. The system does not use the whole video for the reconstruction, but it just uses some sort of key-frames that are taken while you move. When I arrived at around 600 frames, I stopped the recording. The system allowed me to watch the taken video, and the pose in the room in which the whole keyframes had been taken. At this point, I could decide to shoot a new video to integrate the existing one or just upload the capture to the Varjo servers. Since Varjo people had been still like statues already for five minutes and I know that in Finland there are lots of places where my corpse could be hidden under the snow without anyone finding it for years, I decided that it was time to free them and I uploaded just the capture I did.

A screenshot from my scan: I have spent some time around the simulator chair, so you can see that it is pretty ok in its reconstruction, but for instance, I have not framed the keyboard on the small table well, so it has not been reconstructed properly. The bottom right corner also shows many artifacts.

Yesterday I received the results of my capture and the kind Finnish people of Varjo defined it as “Not bad as a first capture”, while as a direct Italian guy, I would say “It kinda sucks”. There are some parts which are reconstructed well, others like the ceiling and the floor are really bad and the people seem coming from an impressionist painting. The thing is: for everything there is a learning curve. Even shooting a video with a smartphone is easy to do, but to shoot a GOOD video, you need to learn how to properly do it. The same is true for Teleport: the app is truly user-friendly as it is marketed, but there is a learning curve to follow to be able to do a very good scanning of a place. I guess I need to do a few scans, maybe starting from smaller places, before being able to do a very good scan for this system. Varjo on its side has also to work to make the system more tolerant to scanning mistakes.

How to view the environments

Visualizing a Teleport on a laptop (Image by Varjo)

After you have uploaded your scan, Varjo servers have to crunch it for some hours (2-4 hours, usually, but can arrive to 24 hours), then the scanned environment is available for entering inside it.

The pretty smart thing that Varjo did is to not focus only on VR people but to make the environments available for everyone through the web. The preferred way to share an environment is to obtain a link to it inside the Teleport portal and open it with your browser on whatever device you want: mobile phone, tablet, PC, etc… You will have a camera inside the reconstructed place and you can use the input scheme of your device (e.g. mouse and keyboard on a PC, your fingers on your mobile/laptop) to move and rotate the camera to explore the space. It’s cool that the reconstruction is performed at different quality levels and you can choose which one you want to see (a bit like in YouTube): mobile devices can use SD quality, PC can use HD, and PC with VR the ULTRA-HD. Different quality levels correspond to a reconstruction with a different number of Gaussian splats. This way, it is possible to have a decent framerate no matter what device you are using. Rendering happens locally after the download of the reconstructed space, there is no pixel streaming involved.

VR requires ULTRA-HD quality because being literally inside the place, you are more susceptible to every imperfection, so you need the highest reconstruction quality possible. This comes at the expense of having a powerful computer: the recommended setup involves having an NVIDIA RTX 3090 with 24GB VRAM, or higher equivalent. To see the teleports in VR you are also requested to install the Varjo Teleport desktop client. The application is compatible with all OpenXR compatible headsets, so you can use your Quest+Link, Varjo headsets, Vive headsets, etc… Notice that while the viewing experience is web-based, Teleport is not compatible with WebXR and this is why you need the desktop app. Also, notice that currently, Teleport has not a client on standalone headsets like the Quest 3 or Pico 4 Ultra.

The gallery of some pre-scanned environments viewable on Teleport

I personally like the approach of using the web to make Teleport available for everyone, everywhere. This way if someone has a VR headset, he/she can enter into the spaces in the most immersive way possible, but if he/she has not it, can still somehow access the environments with whatever device. If there could be a way to add WebXR to the mix, this would be perfect. Sharing a link with everyone you want is very easy. Varjo explained to me that these links that creators can generate are like YouTube unlisted links: only the people you give the links to can access your scans.

Hands-on viewing of the environments

I was able to enter into a few scans that Varjo and its partners have done. The commercial launch was preceded by a closed beta period during which Varjo worked with 2,000 professionals and enterprises to develop and optimize the solution. These beta-testers have created pretty good scans and Varjo made me jump into some of them.

I have been in a car garage and looked at a racing car. The racing car was fantastically scanned and looked almost real when seen through the Varjo XR-4. The environment around it had been scanned with less attention, though, so the magic was broken as soon as I looked around me. I have been on some stairs full of graffiti in a city, and the walls were very well made, too: I had to look very close to them to find imperfections. These scenes were very well made and a clear highlight for me. There was another one they showed me that did not have the same quality, though: it was a forest, a pretty challenging environment for a scan, and in fact the results were not great (or maybe I should use the PR language and say “Not bad as a first capture”…). The quality of the reconstructions depended a lot on the quality of the scans.

Trying Teleport, like when I tried similar solutions, I had my moments when I internally thought “Oh wow, this looks real”, but also other moments when the reconstruction artifacts ruined the magic. When there is not enough data to reconstruct something, Gaussian Splats appear as brush strokes or as small needles and if the person making the capture focuses too much on some part of the space while giving less attention to other parts, the result will be that some parts of the room will show these artifacts. When seen in VR, the artifacts are always going to be an immersion-breaker. That’s why I suggested Varjo people do some magic to either use AI to improve the reconstruction of poorly scanned portions or to do some visual tricks (e.g. fading to black) to hide these parts of the place.

Movement inside the environments in VR can happen via teleporting or via fly-locomotion so that you can move up-down-left-right and also rotate your view. Flying allows to see everything in a place, so I prefer it, but it can also cause sickness to people less used to VR.

The interface of Varjo Teleport on the web, with the ability to switch between the different qualities of a capture

I tried to see environments both on a laptop, tablet, and VR. Rendering and navigation work pretty well on all of them. But I’ve noticed that when the scenes are heavy, the rendering starts becoming a bit less fluid. Even on my laptop, with an RTX 20x, I have some stutterings when entering into some scenes, even when I’m not in VR. So you had better have a powerful PC or tablet to use these splat reconstructions. I think that also Varjo has to do more optimizations to make everything run smoother on all devices.

I had already tried Varjo Teleport as a preview in June at AWE US, and I have to say that the quality of the environments that are available now is much higher. Varjo has done a great job in improving its product in these 5 months and now it can clearly compete with other applications in the same space like Gracia and Horizon Hyperscape.

Varjo Teleport vs Horizon Hyperscape vs Gracia

A wonderful burger (scanned by Tipatat) and visualized on Gracia

Having tried all the most famous solutions to scan and visualize environments in XR with Gaussian Splats, let me do a quick comparison between them.

Meta Horizon Hyperscape is the best app if you want to see some amazingly reconstructed environment. I’m pretty sure that some professionals have been hired to use all their best tools to scan some rooms in the best way possible. It runs on Quest, but only thanks to pixel streaming from Meta servers using the Meta Avalanche service. Mark Zuckerberg announced that Meta will release an app to scan places to publish there, but at the moment, only these showcase environments are available. But still, these showcase places are damn cool.

Gracia is the best app to see reconstructed moving scenes. It is a platform that can visualize Gaussian Splats both on PC VR and standalone VR; on MR headsets, it can show elements both in VR and in MR in your room. Rendering is fully local. The PC version allows also to see the scannings made by the community using the scan tools they prefer. The most impressive feature is the visualization of volumetric videos, which literally put me in awe when I tried them. Gracia seems to be the most versatile of the solutions.

Teleport is the best to have a complete reconstruction solution. It features both the scanning (via the companion app) and the visualization of the environments (via the web portal), and also easy sharing of the scans. But it lacks a proper VR client on standalone headsets. The scanning is thought to be as user-friendly as possible. Visualization happens locally. Teleport seems to be the most comprehensive of the solutions for static scenes. But this comes also with a price, as we’ll see in the next paragraph.

As for the reconstruction quality, it seems that all of them are pretty good at showing reconstructed elements. Gracia seems to perform better on objects, while Hyperscape and Teleport on environments. Hyperscape seems a bit better, but I would like to wait for a community-made scan before making any judgment on its real quality.

Pricing and availability

Varjo Teleport is available now at teleport.varjo.com.

Viewing scans is free, and possible also without an account on the web. You have to register (for free) to download the desktop client for VR, though.

Performing scans requires a paid subscription. You need to pay €30/month to have available 15 scans per month, with the ability to share them via links, export the PLY splats, and also shoot fly-through videos. There is also a free trial available.

I have come out with a positive impression of Teleport and I appreciated especially the fact that it is a solution encompassing all the aspects of environment scanning and reproduction (sharing included). I think that if someone is a professional in this field, it may be worth it for him/her to pay the subscription price to have it. Hobbyists maybe could look for cheaper solutions, instead.

A glimpse of the future

I’m very bullish on this kind of application. There are some clear enterprise use cases where Varjo Teleport can become very relevant. A big one of them is virtual tours of houses, for instance: 360 photos are nice, but what about truly entering a room and moving freely inside it? Or I can imagine training scenarios or keeping people updated about the installation of something in a specific place. Varjo told me that it is possible to keep a history of all the scans of a place, so that a remote manager can follow the evolution of a place that is being built over time without flying there every week, for instance.

But where I see the long-term value is for consumers. I think that long-term scans will be the new pictures. My mind definitely clicked when Varjo’s PR person and I were at an event and we both saw a cool thing. I immediately shot a picture with my phone. She instead said “Oh, let me make a Teleport” and proceeded to scan it. Yes, with the current technology, I needed one second for the photo, while she took two minutes to make the scan. But she will have a reconstruction through which she can feel there again in the future, while I have just a flat memory from a single point of view. I remember thinking “This girl already lives in 3024”.

When the scanning tech is as quick as taking a short video, I believe that it will become a new way to save our memories and share them with our friends so that we can truly re-live these moments. Now Teleport, Gracia, and the others are all tech demos or professional tools, but long term they can become the media production and consumption of the future. What a time to be alive.

(PS Huge thanks to Annaleena, Valeria, Jobin, and the 4th secret guy for the kindness in assisting me in the demo and explaining everything about the Teleport!)

Disclaimer: this blog contains advertisement and affiliate links to sustain itself. If you click on an affiliate link, I’ll be very happy because I’ll earn a small commission on your purchase. You can find my boring full disclosure here.

This article was originally published on skarredghost.com