Face-Mic motion-sensor speech eavesdropping

1. Summary

Face-Mic is a side-channel eavesdropping technique against face-mounted AR/VR headsets in which a zero-permission application samples the headset’s built-in three-axis accelerometer and gyroscope while the wearer speaks [2]. Because the headset is rigidly seated against the face, speech-associated facial muscle movements and bone-borne vocal-fold vibrations mechanically couple into the motion sensors, which require no permission to read even though the microphone does [2][4]. A deep-learning pipeline reconstructs speaker gender, speaker identity, and simple speech content (digits and words) from these vibrations without ever touching the microphone [2]. An attacker thereby gains a covert, permissionless microphone-equivalent capable of recovering spoken PINs, card numbers, and persistent biometric attributes [4].

Under the strongest attack scenario the technique achieves over 99% speaker-gender detection and over 99% top-1 speech recognition on standalone headsets [2]. Even without any labeled data from the target, top-1 digit recognition on the Oculus Quest remains 54% under adaptive domain adaptation (Scenario-2) and 44% with a purely pre-trained model (Scenario-3) - roughly 5x random guessing [2].

2. Affected products & versions

Note: this is not versioned as a software CVE. The flaw is a device/platform-class design issue; any face-mounted headset exposing ungated high-rate motion sensors is in scope [4].

3. The vulnerability in detail

The flaw does not live in a single source file, network protocol, or memory-management routine. It lives in the sensor-governance layer of the AR/VR platform - specifically in the asymmetry between how the microphone and the inertial motion sensors are permission-gated [2][4]. On AR/VR operating systems (and on the smartphone OSes underpinning Cardboard headsets), the microphone is a rigorously protected resource: an app must request and be granted explicit user permission before it can capture audio. The built-in inertial measurement sensors - the three-axis accelerometer and the three-axis gyroscope - carry no such gate [4][2]. Any installed app can read them in the background, silently, because motion sensing is treated as a benign, non-sensitive capability required by essentially every AR/VR application for head/hand tracking. That assumption is the root cause: the platform classifies motion data as non-privacy-relevant, but on a device strapped to the wearer’s face, motion data is not benign. It becomes a covert acoustic channel that leaks exactly the class of information - spoken content and voice biometrics - that the microphone permission was designed to protect.

The physical mechanism is what turns this governance gap into an acoustic side channel. When a person speaks while wearing a headset, three vibration effects reach the sensors [2]. First, facial muscle movements - principally of the perioral muscles that encircle the mouth and adjacent muscles in contact with the headset - dominate the sensor readings at lower frequencies (below roughly 100 Hz). Second, bone-borne vibrations generated by the vocal folds propagate through the skull and dominate the readings at higher frequencies; the paper notes the skull responds to sound vibrations above roughly 250 Hz [2]. Third, weaker airborne vibrations share characteristics with the bone-borne component but contribute less energy [2][6]. The accelerometer and gyroscope, rigidly coupled to the face by the headset’s foam cover and headband, together act as a low-fidelity contact microphone for these vibrations. Vendors cap sampling rates for power reasons - 227 Hz on the Nexus 6, 203 Hz on the Galaxy S6, and 1000 Hz on both the Oculus Quest and HTC Vive Pro [2] - all far below the 85 Hz-20 kHz span of human voice [2], yet the residual low-fidelity signal still carries enough structure for machine-learning inference. Notably the split between the two dominant regimes (~100 Hz) is what lets the pipeline separate a facial-movement channel from a bone-borne channel and model each independently.

The Face-Mic pipeline exploits that residual signal [2]. A malicious app - disguised as a game, browser, or other ordinary AR/VR app - reads the accelerometer and gyroscope in the background. The authors demonstrate this against both mainstream AR/VR programming platforms [2]. On an app built with the Oculus SDK (v23), they confirmed the function ovr_GetTrackingState records the Oculus Quest’s accelerometer/gyroscope data in the background without user permission. On the OpenVR platform - which supports a broader range of headsets from HTC, Valve, and most Windows Mixed Reality manufacturers - the equivalent read is GetRawTrackedDevicePoses, likewise available without user permission [2]. (These are two distinct platform APIs; GetRawTrackedDevicePoses is an OpenVR call tied to HTC/Valve/Windows MR headsets, not an Oculus Quest API.) The raw x/y/z streams are first passed through a deep-regression Body Motion Artifact Removal (BMAR) stage - a signal-source-separation step that strips large-scale head- and body-movement artifacts (concentrated at low frequencies, below ~60 Hz) that would otherwise swamp the speech-associated micro-vibrations. The cleaned signal is transformed into spectrograms via Short-Time FFT, and the per-axis accelerometer/gyroscope channels feed two CNNs - one specialized for the low-frequency facial-movement component and one for the high-frequency bone-borne component. The resulting feature representations drive three classifiers: speaker gender, speaker identity, and speech content (digits and words). To generalize to a victim the model has never seen, Face-Mic applies unsupervised domain adaptation with a domain discriminator trained by adversarial loss, so a model pre-trained on other people transfers to a new victim without needing that victim’s labeled data [2].

The exploitation chain is: (1) build an app that reads motion sensors in the background with no permission gate and disguise it as benign; (2) get the victim to install it via social engineering or an app store; (3) wait for the victim to speak while wearing the headset - voice dictation, multiplayer chat, or a virtual meeting; (4) capture and exfiltrate the raw accelerometer/gyroscope readings to a remote adversary; (5) run BMAR + STFFT + the two CNNs to infer gender, identity, and speech content; (6) depending on scenario, either train on prior victim-labeled data (Scenario-1, highest accuracy), adapt a pre-trained model via unsupervised domain adaptation (Scenario-2), or apply a pre-trained model directly with no victim data (Scenario-3); (7) reconstruct sensitive strings - SSNs, credit-card numbers, PINs, phone numbers, passwords - and persistent biometrics [2][4]. This is a LOCAL attack in the classic sense: it requires code running on the device but no network adjacency, no RF proximity, no authentication bypass, and crucially no microphone permission [7].

Reported accuracy is striking. Under Scenario-1 the technique reaches over 99% gender detection and over 99% top-1 speech recognition on the standalone headsets, with speaker identification of over 97% (Oculus Quest) and over 93% (HTC Vive Pro) [2]. It degrades gracefully without victim labels: on the Oculus Quest, top-1 digit recognition is 54% under Scenario-2 and 44% under Scenario-3 (versus a 9.1% random-guess baseline for the 11-way digit set), and HTC Vive Pro reaches 44% (S2) and 33% (S3) top-1 for digits [2]. Even the low-sampling-rate Cardboard headsets exceed 93% top-1 speech recognition under Scenario-1 [2].

The proof-of-concept is a fully implemented academic demonstration, empirically validated across four commercial headsets and 45 participants, with the background zero-permission sensor read confirmed on both the Oculus SDK (Oculus Quest) and OpenVR platforms [2].

In CWE terms this is best read as CWE-200, Exposure of Sensitive Information to an Unauthorized Actor, realized through a physical/hardware side channel (CWE-1300-adjacent): an unregulated, zero-permission sensor leaks exactly the class of sensitive information - spoken content and voice biometrics - that the microphone permission was designed to protect. No CWE is assigned in the available sources; this mapping is the analyst’s inference, not a documented or formal assignment, and must not be read as one.

4. Discovery & timeline

Face-Mic was developed by researchers at Rutgers University’s WINLAB / DAISY Lab, led by Yingying “Jennifer” Chen, with collaborators Nitesh Saxena (Texas A&M) and Jian Liu (University of Tennessee, Knoxville), and co-authors Cong Shi, Xiangyu Xu, Tianfang Zhang, Payton Walker, Yi Wu, and Jiadi Yu (Shanghai Jiao Tong University) [2][3][4]. The work was published at the 27th ACM MobiCom 2021 in New Orleans (proceedings dated January 31-February 4, 2022), DOI 10.1145/3447993.3483272 [1][2][3]. A Rutgers institutional press announcement and public reporting appeared around February 2022 [4][7]. No coordinated-disclosure timeline, vendor-notification date, or official Meta/HTC/Sony vendor response is documented in the available sources [4].

5. Technical reference

• CVE(s): none assigned in the available sources.
• CWE: inferred CWE-200 (Exposure of Sensitive Information to an Unauthorized Actor) via a physical/hardware side channel (CWE-1300-adjacent). Not documented by any source; analyst-inferred mapping only, not a formal assignment.
• CVSS: no CVSS vector is published for this vulnerability; the harm is surveillance-class privacy leakage that CVSS’s confidentiality/integrity/availability model does not cleanly capture (there is no data written, no service disrupted - only a covert read of information the platform never modeled as sensor-accessible). This is why it is treated as UNSCORABLE under CVSS (see Section 6).
• Key hardware refs: sensor modules - Invensense MPU-6500 (HTC Vive Pro, Galaxy S6), Bosch Sensortec BMI160 (Nexus 6), Facebook-designed original motion sensor board for the Oculus Quest. The board’s silkscreen/part string appears in the paper as 330-0019303 1PASF8K; the exact hyphenation is not confidently established from the public source (it is plausibly 330-00193-03 1PASF8K, but only the un-hyphenated form is present in the paper text) [2]. Sampling rates: 227 Hz (Nexus 6), 203 Hz (Galaxy S6), 1000 Hz (Quest and Vive Pro) [2].
• API refs: ovr_GetTrackingState - Oculus SDK v23, records Oculus Quest accelerometer/gyroscope in the background without permission; GetRawTrackedDevicePoses - OpenVR (HTC / Valve / Windows Mixed Reality headsets), collects motion-sensor data without permission [2]. These are distinct platform APIs; the OpenVR call is not an Oculus Quest API.
• Code/commit refs: none.

6. Consequence & CFSE path analysis

Real-world consequence: an attacker who gets a single innocuous-looking app onto a victim’s headset can covertly turn that headset into a microphone the victim never authorized and never perceives as a microphone - recovering spoken PINs, SSNs, card numbers, and passwords, and persistently fingerprinting who is speaking (identity, gender) [4][2]. The victim has no permission prompt, no recording indicator, and no realistic way to detect it; recovery is not a software update but a hardware/platform change.

Why the CFSE Path Score diverges from CVSS. CVSS has no baseline score here - the harm is a pure side-channel privacy leak that its metric set cannot represent, so the record is UNSCORABLE by CVSS. CFSE instead scores the consequence paths the bug enables, and decomposes this vulnerability into two:

• PERCEPTION_PRIVACY (dominant path) - banded CRITICAL. A zero-permission app samples built-in motion sensors to reconstruct live speech content from sensors the user never perceives as a microphone. The harm reuses across the entire fleet (the design flaw is class-wide, not device-specific), and recovery is largely unpatchable in software - it requires hardware or platform-model change (damping materials, sensor rate governance, permission redesign). This dominant path drives the overall verdict.
• DATA_PRIVACY (supporting path) - banded HIGH. The same motion-sensor inference reveals speaker identity, gender, and other persistent biometric attributes - durable, non-revocable personal data.

Both paths share the same access surface: a read-only sensor with no permission gate, attack complexity low (AT low), PH:0 (a pure side-channel privacy leak with no perception/observability by the victim), EV:3 (reproduced across four devices and 45 participants), and a recovery posture treated as fleet-wide / recall-class. The overall path_verdict is CRITICAL, driven by the dominant perception-privacy path. The record explicitly notes that no public baseline (CVSS) score exists for this surveillance-class harm - the CFSE banding, not CVSS, is what carries the severity signal here.

Scoring-token key (for this section only): AT = CVSS-style attack-complexity/requirements axis (low = no special conditions); PH = perceptibility-to-victim on a 0-3 scale (0 = imperceptible); EV = evidence-strength on a 0-3 scale (3 = multi-device, multi-subject empirical reproduction); “recall-class recovery” = remediation requiring hardware/platform change rather than a shippable software patch. These tokens are CFSE scoring shorthand, not values drawn from any source.

7. Remediation & mitigations

There is no vendor patch in the available sources. All mitigations below are researcher proposals, not shipped fixes [2][4]:

1. Rate-limit accelerometer/gyroscope sampling in the AR/VR OS - the paper proposes constraining sensor sampling rates as a defense, but flags limited efficacy: the facial-muscle component persists below 100 Hz, the attack still works down to ~200 Hz on Cardboard headsets, and aggressive limits degrade legitimate head/hand-tracking functionality and usability [2].
2. Programmatically inject sensory noise into motion-sensor readings - add controlled noise to break the correlation between sensor output and facial dynamics while preserving usable motion tracking [2].
3. Add ductile/damping materials to the foam face cover and headband - mechanically attenuate speech-associated facial and bone-borne vibrations before they reach the sensors; the paper notes the thicker HTC Vive mask already yields lower attack success than the thin Oculus foam cover, evidence that added damping directly suppresses the side channel [4][2].
4. Gate background motion-sensor access behind a permission - conceptually the direct fix (it closes the microphone/motion-sensor asymmetry), but noted as impractical because nearly all AR/VR apps legitimately need motion sensing and users habitually grant such prompts [2].

Detection note specific to this bug: because the leak is a passive background sensor read with no microphone access, conventional microphone-permission auditing and recording indicators will not surface it. Any monitoring would have to target anomalous high-rate background motion-sensor polling and exfiltration by apps that are not tracking-critical - a platform-level capability that does not exist in the tested devices.

8. Sources

[1] Face-Mic: Inferring Live Speech and Speaker Identity via Subtle Facial Dynamics Captured by AR/VR Motion Sensors - ACM MobiCom 2021 - https://dl.acm.org/doi/10.1145/3447993.3483272 - canonical peer-reviewed publication of record; confirms DOI, venue, author list, core method. Credibility: researcher-primary.

[2] Face-Mic (full PDF) - Rutgers WINLAB / DAISY Lab - http://eceweb1.rutgers.edu/~daisylab/data/papers/FaceMic.pdf - author-hosted full text; load-bearing for methodology, ML pipeline, sampling rates, hardware modules, API names/platform attribution, user study, and all accuracy figures. Credibility: researcher-primary.

[3] Face-Mic publication page (co-author Xiangyu Xu homepage) - Xiangyu Xu - https://shawnnxu.github.io/publication/2021-10-conference-FaceMic-MobiCom - corroborates title, author list, venue, and year from a primary researcher. Credibility: researcher-primary.

[4] Rutgers Researchers Discover Security Vulnerabilities in Virtual Reality Headsets - Rutgers University - https://research.rutgers.edu/news/rutgers-researchers-discover-security-vulnerabilities-virtual-reality-headsets - official institutional advisory; root-cause permission asymmetry, affected devices (including Quest 2, PSVR), sensitive-data categories, lead researcher, mitigation recommendations. Credibility: primary-advisory.

[5] Face-Mic… (paper record) - Semantic Scholar - https://www.semanticscholar.org/paper/Face-Mic:-inferring-live-speech-and-speaker-via-by-Shi-Xu/f2c3dfa7ad07ebe251f5e8280108f7946770bca9 - independent bibliographic index confirming canonical metadata and author list. Credibility: reputable-press.

[6] Face-Mic Turns Your Virtual Reality Headset Into a Permissionless Speech-Monitoring Microphone - Hackster.io - https://www.hackster.io/news/face-mic-turns-your-virtual-reality-headset-into-a-permissionless-speech-monitoring-microphone-9332fa8cb2c5 - technical press framing the attack as a zero-permission microphone bypass; describes the three vibration types and affected headsets. Credibility: reputable-press.

[7] How hackers could use popular virtual reality headsets to steal sensitive information - Help Net Security - https://www.helpnetsecurity.com/2022/02/18/virtual-reality-headsets-privacy/ - security-industry press (Feb 2022) corroborating the permissionless-sensor root cause, the local malicious/sandboxed-app vector, and the extractable sensitive-data categories. Credibility: reputable-press.