UploadVR covers a Facebook Reality Labs prototype that tracks facial expressions using cameras mounted in a VR headset. The updated version captures subtle micro-expressions for realistic avatars.
Why Hacker News points people here: Pointed to in metaverse and VR threads as proof that realtime photorealistic face tracking from headset cameras is coming.
“seriously jawdropping demo of facial tracking using a few cameras in a headset”HN comment, 2021
“seriously jawdropping demo of facial tracking using a few cameras in a headset”
Jawdropping Oculus demo of facial tracking from headset cameras plus AI.
Full comment
> The author mentions lip syncing and hand tracking. That goes a long way, but it doesn't recreate the video experience to me. The oculus team put together a seriously jawdropping demo of facial tracking using a few cameras in a headset, combined with a lot of AI. The prototype was from a couple years ago. I can't wait until it makes it into commercial headsets. I wonder what the hold up is? Well worth a look if you haven't seen it: this video
Cites itjp555Aug 2020
Shows prototype VR face tracking conveying micro-expressions.
Full comment
It will ( this video ), and VR already conveys MUCH more of the micro-interactions of being with people that we are unknowingly blind to in Zoom meetings, which is one reason Zoom meetings are so draining.
Cites itbabyAug 2021
“Whatever you're talking about is coming”
Shows face tracking is coming to VR.
Full comment
I don't think they don't understand it, they understand incremental improvements. Whatever you're talking about is coming: this video
Cites itroveoNov 2021
“There are algos that reconstruct photorealistic 3D facial expressions from headset cameras in realtime.”
Demonstrates realtime photorealistic facial expression reconstruction from headset cameras.
Full comment
There are algos that reconstruct photorealistic 3D facial expressions from headset cameras in realtime. this video
Cites itmicrojimMar 2022
“See what Facebook have been doing with cameras mounted in VR headsets”
Example of Facebook's headset-camera face tracking for remote presence.
Full comment
Title should be amended to something like 'Face-to-face, rather than shoulder-to-shoulder, enhances learning, innovation.' For a high quality face-to-face remote experience, I expect we'll start to see really impressive consumer ready technology announced soon. See what Facebook have been doing with cameras mounted in VR headsets [1] or Google's Project Starline [2]. That said, when we're doing this much transformation and synthesis of the raw data to achieve high-fidelity telepresence, are we going to have issues trusting it? [1] this video [2] link
