âł 5 min read
I knowâŚI know⌠I am lagging behind, haha. I think I am the last one to write about Appleâs new headset. Anyway, MDE is not a burning news newsletter, but rather a space of reflection around XR ethics and the metaverse ecosystem. So, articles here do not have the urgency of a news feed. đ
Why talk about Vision Pro? Here, I donât talk specifically about products, but this one deserves special attention. The release of Vision Pro is a statement for the XR and metaverse field: Apple is legitimizing XR as the next wave of computingâyeah, they call it spatial computing to get away from the metaverse and VR buzz, but we know itâs more of the same.
Its soon-to-be-released headset was well-known to the XR community for the past years. We knew that Apple was going to set the bar for excellence in the XR space. And as important, to make appealing the device to mainstream users so that XR reaches the consumer.
You will disagree with me now: $3,499 is that mainstream? Of course not. But this device is just the first of a series of Vision Pros that will become lighter, cheaper, and appealing to diverse budgets. Just envision a beautiful sleek Vision Air⌠This is just the start of mainstream adoption. â¨
Now letâs dig in!
Biofeedback. What is that?
According to Wikipedia, âBiofeedback is the process of gaining greater awareness of many physiological functions of one's own body by using electronic or other instruments, and with a goal of being able to manipulate the body's systems at will.â
For biofeedback to happen you need a sensor that measures data from your body and a system to process such data. Once the system has processed your bodily data, it can learn from it and program certain responses according to your body. This is the case with most wearables.
For example, Apple Watch has a Breathe App that detects your breathing rate via sensors on the watch and adjusts the training accordingly.
Biofeedback or âbiological feedbackâ just means that the device is adapting to the information itâs receiving from your body.
Of course, thereâs the dark side of the force too⌠Biofeedback is the most accurate manipulation tool. When a device gets to know about your body and physiology, you are more exposed than ever to apps that can manipulate you.
Biofeedback and Vision Pro
Vision Pro is an advanced headset that has accumulated more than 5000 patents over its more than 10 years of development.
What is fascinating is how biofeedback is integrated into the headset: anticipating the interaction from the user. This is pioneering in the field. đ
Besides using hand-gesture and voice-command controls to interact, the headset presents several eye-gazing patents that facilitate the seemingly ânatural interactionsâ that make the device âjust workâ.
Disclaimer: I was not in the privileged pool of testers of the device. What I write here is after seeing the launch and some reviews from testers of the device.
So instead of testing the device, I dug into some of its neurotechnology patents.
Three very interesting patents were co-developed by researcher Sterling Crispin while at Apple. These patents help predict usersâ behavior and adjust the mixed-reality visuals accordingly. Letâs check them!
Eye-gaze based biofeedback. Evaluates whether users are attentive to an experience to adjust the UI accordingly.
Biofeedback method of modulating digital content to invoke greater pupil radius response. Analyzes usersâ pupil dilation to certain visual features. The system learns from usersâ responses and then adjusts the content to enhance the experience for the user. Â
Utilization of luminance changes to determine user characteristics. Identifies whether users are paying attention, are distracted, or are even mind-wandering by analyzing the pupillary response to luminance changes in an experience.
In Sterlingâs words: âThere were a lot of tricks involved to make specific predictions possible, which the handful of patents Iâm named on go into detail about. One of the coolest results involved predicting a user was going to click on something before they actually did. That was a ton of work and something Iâm proud of. Your pupil reacts before you click in part because you expect something will happen after you click. So you can create biofeedback with a user's brain by monitoring their eye behavior, and redesigning the UI in real time to create more of this anticipatory pupil response. Itâs a crude brain computer interface via the eyes, but very cool.â
So, we could even talk about BCIs (Brain-Computer Interfaces). Itâs still early to talk about this type of technology for mainstream adoption and consumer usesâjust witness the controversy of Neuralinkâs human trials. However, Apple masterfully introduces physiological sensors in its headset pointing toward that direction. Biofeedback is the future.
So what can we do?
Well, that was more of an informative post.
What matters is being aware of neurotechnology being embedded in advanced headsets to create the âmagicâ weâre all looking for.
Itâs not magic, haha⌠itâs the headset reading the user ;-)
Now, you know.
Bye for today!
News of the past month: I literally packed up my life. My cycle in Denmark has come to an end and June was about piling stuff into boxes and coordinating the logistics for my international move. Where to? Paris! New adventures await me there. Canât wait to share more soon đ
Thanks for reading until the end! I am looking forward to having you as part of this expanding community. Just click below đ to keep updated on the next ones.