MDE#02 Metaverse Data Privacy 1

Let´s dance in VR! While enjoying data harvesting…

⏳ 6 min read

Guarantee the privacy of metaverse users, MDE#01’s first commandment.

Let´s dig into user privacy in the metaverse (yay!). This topic may seem boring, but I assure you that marketers will make it damn attractive. So, you´d rather be informed! Here, I refer to the metaverse as a vision for the next wave of computing that is ubiquitous, seamless, and immersive—making use of Augmented Reality (AR) and Virtual Reality (VR).

Today, I analyze the paper “Exploring the Unprecedented Privacy Risks of the Metaverse” [1] from UC Berkeley (US) and the Technical University of Munich (Germany) by researchers Vivek Nair, Gonzalo Munilla Garrido, and Dawn Song.

The article presents an experiment where 30 participants navigated a VR escape room while simultaneously playing little games. Let´s check what happened behind the scenes.

Researchers created an adversarial program to collect personal data attributes. In other words, they created a program to collect participants’ data from 4 data sources: the VR device, the application (the game), the server, and other potential users. They managed to collect 25 unique data attributes from each participant: from geospatial telemetry, device specifications, network data, and behavioral observations. Wanna know which ones? Check their amazing data attribute taxonomy. (I can´t publish it here. Damn!).

Getting into details, what is interesting is that participants´ data is classified as primary, secondary, and inferred. Primary data attributes are collected from sensor data. For example, data from the left and right arm length. Secondary data attributes are derived from primary data. For example, you can determine the wingspan (secondary data attribute) by knowing the left and right arm lengths. Inferred data attributes are derived using machine learning from primary and secondary data attributes. In this case, it is possible to infer the gender by knowing the wingspan and other data points such as height and interpupillary distance.

So, a pleasant and innocent activity such as dancing in VR and opening your arms to the rhythm of the music could lead to inferring your gender, wealth, ethnicity, age, and even disabilities! 😲

So, let´s dance in VR and enjoy? The fun doesn´t end here.

How about biometric data harvested right off your face?

This is the next frontier. And it is here.

Biometric data harvesting

Advanced headsets include analytics that track facial expressions and the gaze (eye-tracking). Examples are Varjo´s headsets; HP Reverb G2 Omnicept with a set of monitoring tools that include the heart rate, pupil dilation, and facial tracking; and the freshest, Meta Quest Pro.

So, why is biometric data harvesting the next frontier? The main reason is to create realistic avatars. Have you ever used a VR social platform? Interacting with other avatars feels clunky: the interaction is limited. When chatting with another avatar, it seems like you were talking to a puppet or a robot (depending on the avatar's look!). It’s very impersonal: it lacks facial expressions.

To make the interactions more natural and feel that you are in front of a real person, incorporating users´ data from their face is crucial. Even the promise of including avatars’ legs is a step towards making avatars more realistic.

Hats off to the advances from tech companies, their research and product teams that are making it possible!

As a technologist, I am impressed.

As a person, I am concerned about where society is heading.

Dr. Luke Stark, subject expert and assistant professor at Western University (Canada), explains in this article: “It’s been clear for some years that animated avatars are acting as privacy loss leaders,” he said. “This data is far more granular and far more personal than an image of a face in the photograph.”

In other words, data from your face reveals much more information than a simple picture. Why? Because you get a stream of information on how your face moves and reacts: it’s a digital blueprint of yourself.

Here is where privacy notices and agreements between users and companies matter.

Checking privacy notices

Let´s take the example of Meta´s new Quest Pro privacy notice.

Raw data from eye-tracking and facial expressions is processed locally on the headset and deleted after the processing. Kudos to Meta for that!!

However, using third-party services such as the new Movement SDK would allow developers to access biometric data and is subject to third-party terms and privacy policies. ☠💥💥

Isn´t this familiar? My intuition tells me that it will not be necessary to “hack” AR/VR devices to collect data from users and exploit it for commercial purposes. It’s as simple as concealing the consent somewhere in a privacy policy agreement that the user accepts without even noticing it. Same as with third-party cookies in mobile apps…

The main difference and concern about AR/VR is the amount of data collected quickly and the accuracy of inferring users´ data attributes. AR/VR sensor data is enough to create a digital and behavioral blueprint of a person. We should be aware of that.

On that note, AR/VR pioneer Dr. Rosenberg, states in his paper “Regulation of the Metaverse: A RoadMap” [2]: The platform providers controlling the metaverse will not just know how their users physically act and interact, but how they emotionally react as they traverse real and virtual spaces, profiling their responses at far deeper levels than has been possible in traditional media platforms. Of course, the danger is not merely that these personal parameters can be monitored in real-time, but that advertisers and other paying third parties can use such invasive data to manipulate the wants and needs of consumers more effectively than ever before.

It will be very simple for data harvesting machines powered by machine learning and AI to predict the type of content that you like and present it in a delightful manner, without you even noticing that you clicked on an ad. Oh! 💥

So, what can we do?

There are 3 ways to prevent this. (Let me know if I should add more. Ha!)

• Create awareness on the topic. Talk to your colleagues, family, and kids. For kids (and teens), here are some guidelines for having healthy relationships with tech.

• Awaken policymakers. Yes, these are still early times… but it´s the right moment to act knowing the pace of technology vs. the pace of regulations. If you know any policymakers working in the digital space: share this newsletter, now.

• Spread the word out and loud with people you care about. What are you waiting for? It´s time to share this newsletter. ;-)

Thanks for reading until the end.

Bye for now!

Scientific references

[1] Nair, V., Garrido, G.M., & Song, D.X. (2022). Exploring the Unprecedented Privacy Risks of the Metaverse. ArXiv, abs/2207.13176.

[2] Rosenberg, Louis. (2022). Regulation of the Metaverse: A Roadmap. 10.1145/3546607.3546611.

---

Just click below 👇 to keep updated on the next ones. I am looking forward to having you as part of this expanding community.