James Babcock 60c87521b0 Fix a bunch of microphones app reliability issues
<details>
<summary>Prompts Used</summary>
User: Review the Microphones app, focusing on error handling and reliability.

Opus 5.5:
> I've finished reading the Microphones code and its native pieces. The 2026-10-14 commit `b50898e` fixed a lot, and the main problem left is the same in most places: the app reports "Listening" or "captions on" but never checks that audio is actually arriving. When something fails partway, the session usually stops working with no visible sign.
>
> ## High: silent loss of function or data
>
> **1. In stock mode, other mic users can take the mic away for good.**
> - `stopRawCapture()` tears down the single shared audio tap no matter who started it (`app/native/voice-control.ts:257`).
> - The EvenHub mic router calls it whenever a mic-using EvenHub app goes to the background or closes (`app/apps/evenhub/mic-router.ts:88`). That kills Microphones' audio.
> - Transcribe takes the mic without setting `voiceActivity` (`voice-control.ts:269`), so Microphones is never told to get it back.
> - If capture starts while the session isn't ready, the native controller only records the request, so no audio ever flows.
> - `checkExtendedCapture` (`mic-session.ts:456`) only covers 4-mic mode. Its comment says the stock path "is resumed by the voice-control bridge", but that only covers the session-dropped-while-active case.
> - In 4-mic mode the problem runs the other way: Transcribe takes over the shared Kotlin listener (`GlassesSessionCore.kt:470`) while the glasses keep sending 4-mic frames, so Transcribe gets audio in the wrong format.
> - **Fix:** track who holds the raw tap (like `captureHolders`), add a stock-mode branch to the 3 s check, and have Transcribe set the same "mic in use" signal as the assistant.
>
> **2. One bad utterance ends captions for the rest of the session.**
> - The caption worker has no per-utterance error handling. Any exception from `recognize()` or `embed()` stops the worker for good (`CaptionEngineCore.kt:334`).
> - `started` stays true, so TypeScript still thinks captions are running and recording carries on. The only sign is a "Captions failed:" status string.
> - **Fix:** catch errors per utterance in `onUtterance` (`:367`). Report a fatal worker exit through its own callback so the session can restart the engine or mark captions inactive.
>
> **3. A crash or low-memory kill during recording leaves files that are never shown or deleted.**
> - The WAV header is written with a data size of 0 (`FaceclawWavRecorder.kt:26`) and only corrected in `finish()`.
> - The file is only linked to its conversation in `stopRecording` (`mic-session.ts:1200`), and the conversation row is never closed.
> - Nothing cleans up `mic-recordings/`, so after a crash you get the same never-expired orphan files the last commit fixed for the captions-off case.
> - **Fix:** link the file to its conversation when recording starts. In the boot-time cleanup: close conversations left open (`ended_at IS NULL`), repair the WAV header from the file length, and delete files no conversation points to.
>
> **4. When the transcoder's success callback fails, it deletes both copies of the recording.**
> - `FaceclawAudioTranscoder.kt:66-83` deletes the WAV before calling `onDone`. If `onDone` throws, the catch block deletes the M4A too, and the database still points at the deleted WAV.
> - Separately, the JS listener is referenced only from Java (`mic-session.ts:1208`). Under `markingMode: 'none'` that's the situation `mic-control.ts:74` warns about.
> - I haven't confirmed the garbage-collection risk. `asr-model.ts` does the same thing and its downloads seem to work. But the delete order is wrong either way.
> - `rediarize.ts` and `mic-models.ts` hold their listeners the same way.
>
> ## Medium
>
> 5. **The last utterance is lost when you close the app or turn captions off.** When the caption engine stops, it flushes the utterance in progress and posts it to the main thread. By the time it arrives, `stop()` has already run `finishSessionRow()` (`mic-session.ts:328`), so the line is never saved and is missing from the session insights. If decoding takes longer than the 2 s wait, the line can land in the *next* session with wrong times. Also, `stop()` can block the UI for up to 2 s.
> 6. **The beam filter can mute 4-mic captions permanently.** The direction and signal strength only update on frames with a non-zero angle or strength (`:685`), and they never go stale. If you turn off "Direction detection", the last direction stays frozen. If it was outside the beam, `deliverExtendedAudio` drops all audio from then on (`:751-757`). Stock mode avoids this because it updates the strength on every frame. **Fix:** timestamp the fused direction and treat it as unknown after `TEMPLE_STALE_MS`.
> 7. **The 4-mic health check looks at a phone-side flag, not at whether audio is arriving.**
>    - The check only re-arms when `isAudioCaptureActive()` is false.
>    - A configure command can be skipped while the session isn't ready even though `send()` returns true (`mic-control.ts:155`, `GlassesSessionCore.kt:890`).
>    - A temple can reboot on its own, or `apply()` can refuse to arm when the developer setting is turned off mid-session.
>    - Each of these leaves forwarding "active" with no frames arriving.
>    - **Fix:** you already track `templeFrameAtMs`. Re-send the config when an enabled temple has gone quiet, and show that in the status instead of "Listening (4-mic array)".
> 8. **A failed start is permanent.** `start()` returns before installing any timers (`:280`). A failed mode switch mid-session (`:376`) leaves `forwardingProxy` null, so the health check exits at `:457` every time. **Fix:** stay in a "waiting for glasses" state and let the timer retry.
> 9. **The caption feed can flip between left and right on every frame.** The choice at `:744` doesn't take the current side into account, and `strongerChanged` is just a `!==` check. Near the 1.5× threshold, each flip splices about 25 ms from the other temple (which runs on its own clock) into captions and the recording, either repeating or skipping audio.
>
> ## Lower
> - **Unguarded speaker call (`:1034`):** the speaker rename for a pending introduction runs before the line is added. A database error drops the line and leaves the introduction pending, so the next 2 minutes of lines from unnamed speakers fail the same way. Add the line first, then do the extra steps each with its own error handling.
> - **Speaker list grows forever:** every unknown voice adds a permanent "Speaker N" with a voiceprint. Retention never prunes them. `assign()` reloads and decodes every profile on every utterance, on the main thread.
> - **Translation:** there's no timeout, so lines can wait indefinitely while a model download is stuck offline. The detected language is only saved after translation finishes (`:1132`). The "hysteresis" comment at `:1120` describes logic that doesn't exist: `conversationLang` is written but never read.
> - **Frame fields ignored:** the sequence number (but `TempleAudioPipeline` assumes consecutive frames), the truncated flag, and `sampleRateHz`. The UI offers 8–48 kHz while the code assumes 16 kHz.
> - **Duplicated model path:** `moonshineModelDir()` (`:1337`) copies `asr-model.ts`'s directory name. A model version bump there would quietly drop captions to "no ASR model".
> - **Silent recording gaps:** `WavRecorder.append` swallows each write error, so a full disk means log spam and silent gaps in the recording.
>
> The startup order with cleanup on failure, the 90 s lease that recovers on its own from a lost STOP, and keeping the last good speaker cache all hold up well.
>
> I worked from the code only; nothing was tested on hardware. #6 relies on the protocol comment saying the angle is 0 when direction detection is off.
>
> I'd fix #1, #2, #5 and #6 first, since those are the ones you'd hit in normal use. Want me to start on them?

<hr>

User: Thanks. Let's fix all of the high and medium issues you found now.

Opus 5.5:
> All nine high and medium issues are fixed. TS typecheck, lint, 813 TS tests, the Kotlin host tests (4 new), the iOS Kotlin compile and `build.sh` all pass, and I confirmed the new code is in the APK. Nothing has been tested on the glasses yet, and nothing is committed.
>
> **High**
> 1. **Mic taken by other apps:**
>    - The shared audio tap now tracks who's using it, so an EvenHub app backgrounding no longer cuts off Microphones.
>    - Transcribe now marks the mic as in use, like the assistant does, so Microphones and EvenHub apps step aside while it runs.
>    - A check every 3 s now covers both modes. It restarts a dead stock tap and switches back to stock mode if the developer setting is turned off mid-session.
> 2. **One bad utterance ending captions:** a decode error now skips that utterance instead of stopping the caption engine (`CaptionEngineCore.kt`). If the engine dies anyway, it's restarted on the same conversation up to 3 times. After that, the Captions toggle shows off and the view says why; turning it on retries.
> 3. **Recordings orphaned by a crash:** the recording is linked to its conversation as soon as it starts. At boot, the app closes conversations a crash left open, repairs their WAV headers (`FaceclawWavRecorder.repairHeader`), transcodes them, and deletes files no conversation points to. That cleanup matches by file name, so moving the app to other storage can't delete live recordings.
> 4. **Transcoder deleting both copies:** a failure while recording the new file no longer deletes both files. The app now deletes the WAV itself, only after the conversation points at the M4A. The transcoder, re-diarize and model-download callbacks are now held by the app until they fire.
>
> **Medium**
> 5. **Last utterance lost:** stopping captions no longer blocks the UI. The engine finishes the audio still queued, then sends a new `onStopped` callback, and the conversation stays open until that arrives (20 s fallback). A late line from a previous session won't show up in a newly reopened one.
> 6. **Beam filter muting captions:** the direction of arrival only filters audio while it's under 1 s old, and it resets when "Direction detection" changes.
> 7. **4-mic health check:** a temple that sends no audio for 5 s gets its config re-sent, to that temple only. Mic-control sends now report when they weren't actually queued. The status line says when no audio is arriving, but not after routine restarts like the voice dialog closing.
> 8. **Start failures being permanent:** the session now waits ("Waiting for the glasses microphone...") and the 3 s check retries. Captions start only once the mic is up, so waiting doesn't create empty conversations.
> 9. **Left/right flapping:** the caption feed only switches temples when the other one is clearly louder for 500 ms.
>
> Behaviour changes you'll notice:
> - The watch shows "Listening…" during Transcribe.
> - EvenHub mic apps get no audio while Transcribe runs.
> - In 4-mic mode, an EvenHub mic app still takes over the single native audio listener. Fixing that needs a change on the Kotlin side, so I left it.
>
> On hardware, I'd test:
> - The assistant, Transcribe and an EvenHub mic app taking the mic while Microphones is open, then giving it back.
> - Reconnecting in both stock and 4-mic modes.
> - Killing the app mid-recording, then checking the conversation is recovered at the next start.
>
> I saved these notes to memory as `microphones-reliability.md`.
</details>
2026-10-14 02:17:05 -07:00
2026-04-26 23:07:54 -07:00
2026-04-26 23:07:54 -07:00
2026-10-14 01:16:37 -07:00
2026-10-04 17:57:26 -07:00
2026-08-24 13:17:09 -07:00
2026-08-28 12:02:59 -07:00
2026-08-28 12:02:59 -07:00
2026-08-28 12:02:59 -07:00
2026-04-26 23:07:54 -07:00
2026-09-29 14:33:35 -07:00
2026-10-03 17:01:33 -07:00
2026-04-26 23:07:54 -07:00

Faceclaw - An unofficial user interface for the Even Realities G2 smart glasses

This is an unofficial user interface for the Even Realities G2 smart glasses. It is entirely unofficial, and comes with no support or warranty from Even Realities or from anyone.

This app runs on Android, and requires installing custom firmware on the glasses. The Android app itself can install and uninstall the custom firmware; it downloads the stock firmware from Even and applies patches generated from https://github.com/jimrandomh/g2flash.

User-facing documentation lives at https://faceclaw.org/.

Screenshots

App launcher Music player Customization settings Assistant settings Compass Blocks

Installation

Download faceclaw-.apk from the GitHub releases section on the phone that will be paired with the glasses and install it. On first run, the app will walk you through pairing with the glasses and installing custom firmware.

Compiling

For the NativeScript iOS port, see iOS development. Use scripts/pull_config_ios.sh and scripts/push_config_ios.sh to transfer iPhone settings; see iOS config transfer for options and Android XML support. The instructions below build the Android app.

If you want to customize faceclaw, the best way to do it is to download the source code and using a coding agent such as Codex or Claude Code.

To compile, you will need an Android SDK environment with SDK version 35 installed and licenses accepted, and the ANDROID_HOME pointing to the install. You will need Nativescript installed, and a JDK environment (v21), with the JAVA_HOME environment variable pointed at it. You will also need a reasonably up to date npm and nodejs installed. You can put environment variables in build_paths.sh and they will be used by all build commands.

To install a version that you compiled, you will need developer mode enabled on your phone, and adb connected. To enable developer mode, go to Settings>About phone and tap the "Build number" field seven times. Then plug the phone's USB-C port into your computer, authorize access on the phone, and run ./build_and_run.sh to install.

Note that if you have installed a signed release build, you will have to uninstall it before installing a customized version. Use scripts/pull_config.sh to export a copy of your configuration first, and scripts/push_config.sh to import it into the new version.

Features

  • A voice assistant that wakes up when you say "Hey Even", transcribes text with an on-device model, OpenAI Whisper, ElevenLabs, or Soniox API, and responds to queries and commands using an onboard model (Qwen3 4B, slow) or with an Anthropic or OpenAI model (requires an API key), or using your own long-running OpenClaw agent.
  • Multitasking, with an app-switcher sidebar and app launcher.
  • Glanceboard, a dashboard with widgets that you can quickly view with a
  • Mostly-compatible with EvenHub. Install and runapps from the Even app store.
  • A lock screen; glasses lock automatically when you take them off and unlock when you unlock your phone.
  • The full display. Full-screen apps can use the full 640x480 display area, rather than the 576x288 that EvenHub apps can use.
  • Integration with Android notifications: shows the same notification icons your phone does, and popups when notifications arrive.
  • Terminal mirroring. Mirror terminal apps such as Claude Code or Codex CLI with g2mirror, view them on the glasses, and send them inputs with the voice assistant.
  • Media player controls including playlist and media library navigation, compatible with most Android media players.
  • Turn-by-turn directions (requires a Mapbox API token).
  • Nightscout, an app for viewing blood-glucose data (requires a cloud server and API token).
  • Power management: the glasses go to sleep properly when the screen is off, and wake when you double-tap the ring or speak the wakeword, allowing battery life similar to the stock Even app.
  • A Wear OS watch app that replaces (and outdoes) the R1 ring: tap, swipe, hold and crown gestures, side buttons, app launching and window switching, voice or keyboard queries to the assistant with the reply on your wrist, typing into apps, and glasses status/lock/display control.
  • On-phone screen mirroring: Your phone screen shows the same contents as the glasses, and you can use the phone touchscreen as an input device..

Watch remote (Wear OS)

The wear/ directory holds a companion app for Wear OS 3+ watches that turns the watch into a remote for the glasses. The whole watch face is a touchpad that mirrors the ring (tap, double-tap, hold, swipe, crown scrolling, side buttons), and beyond the ring it can launch apps, switch and close windows, wake/sleep/lock the display, send spoken or typed queries straight to the assistant (using the watch's own microphone) and show the reply on the wrist, and type text into apps such as the terminal. Its swipes are a spatial control scheme the ring never had, with no mode to enter first: the launcher highlights one app and moves it by rows and columns, right goes into things (sidebar → window, settings section → items, next track in Music), left backs out, tap selects, hold opens the context menu, double-tap or a two-finger tap is back everywhere, and two wrist twists go back; up/down scroll and right/left select/back everywhere else. The ring's own scheme is unchanged — watch input is tagged as a separate source. Faceclaw's Settings > Watch section has the phone-side switches. See wear/README.md for building and installing it; it is a separate Gradle build, sideloaded over adb, and needs no Play Store. The phone side requires Google Play services.

Connecting an external agent (OpenClaw)

Instead of calling an LLM API directly from the phone, the voice assistant can route queries to your own long-running agent, which also gets access to the glasses' tools (show alerts, read notifications, control media, type into apps including terminals) both during conversations and proactively. This works through the faceclaw-agent-bridge OpenClaw plugin (https://github.com/jimrandomh/faceclaw-agent-bridge): the phone dials out to it over a websocket (typically across a tailnet).

Setup lives in the faceclaw-agent-bridge repository's README. It covers both the OpenClaw-host side (plugin install and configuration) and the phone side, which can be configured either by hand in Settings > Assistant on the glasses, or over adb using scripts/pull_config.sh and scripts/push_config.sh from this repo; the instructions are written so an OpenClaw agent with the phone plugged into its host can perform the whole setup itself. Note that a pulled settings file contains your API keys, so treat it as a secret (the default output path is gitignored here).

EvenHub App Notes

Faceclaw is mostly compatible with EvenHub apps. If you are developing an app or using an open source app, you can package it into an EHPK file, send it to your phone, and open it in the file browser to install. Or, you can log into EvenHub and download apps there.

Faceclaw runs EvenHub apps through an emulation layer; you may run into bugs and differences in behavior. If you run into bugs, please try tunning them in the stock Android app before you report them to the app's creator. If you're developing your own app and plan to submit it to the Hub, be sure to test it in the stock Android app before submitting.

Additional Caveats

Some permissions are used, but may not reliably prompted for. Go to Android Settings > Apps > Faceclaw to ensure permissions are available or things may not work.

Being a background app on a locked Android phone is fraught, and there may be issues with manufacturer-psecific battery optimization software that lead to it getting paused, throttled to low CPU usage, etc.

Contributing

Be bold. Modify Faceclaw into the app that you want it to be for yourself, without worrying about whether other people will like your version. Then if you think your changes might be useful to others, make a pull request at https://github.com/jimrandomh/faceclaw. Consider joining the Discord server to talk to other developer: https://discord.gg/XVFDFSwek3

The Typescript and Java code in this repository runs on your phone, not on the glasses themselves, and (with the narrow exception of the firmware-updating tool), can't hurt your hardware. For changes to the glasses firmware, refer to g2flash; changes there require more caution.

Faceclaw is Free Software (GPLv3). Please only contribute code that you wrote (or prompted an agent to write), hold the copyright to, and have tested on physical hardware under real-world conditions. If you make any changes that involve integration with a third party's API or services, modify PRIVACY to mention them and link to that provider's privacy policy. For services that involve a user-provided API key, we assume that the user agreed to any terms associated with that service when they generated the key. For services that don't involve API keys, more caution may be required.

S
Description
No description provided
Readme GPL-3.0
41 MiB
0 Stars 1 Watchers 0 Forks
2026-11-18 21:49:32 +00:00
Languages
TypeScript 51%
Kotlin 23.9%
JavaScript 13.4%
C 5.9%
Objective-C 2%
Other 3.8%