94 Commits
Author SHA1 Message Date
James Babcock 30de2c4dd7 Move recording setting from Developer menu to Transcribe app; add prominent indicator 2026-11-11 15:55:52 -08:00
James Babcock 8b70585da8 Metadata updates for Play Store distribution 2026-11-09 12:08:37 -08:00
James Babcock 479308f4a6 Firmware uninstall/revert-to-stock for G1s 2026-11-08 00:11:13 -08:00
James Babcock d1f9f76abb Better rendering of color emojis 2026-11-07 20:20:34 -08:00
James Babcock b353483ac2 G1 support: Preserve connections after flashing, debug 2026-11-07 04:16:15 -08:00
James Babcock 6e4515afa5 Add ANCS support to G1 2026-11-06 21:47:23 -08:00
James Babcock 38c52ebc31 G1 support: Flashing within the Android app 2026-11-06 12:55:24 -08:00
James Babcock 345ba36c33 G1: Microphone, light sensor, accelerometer, bandwidth benchmark, wear detection 2026-11-06 01:44:37 -08:00
James Babcock 67aff19923 Faceclaw support for G1s
Relies on the glasses having the firmware from
https://github.com/jimrandomh/scouter_kit, based on Mafik's Scouter Kit.
2026-11-06 00:12:23 -08:00
James Babcock f803f4212c Brightness control 2026-11-05 11:18:17 -08:00
James Babcock b2b6c5c8d8 Merge branch 'main' into rayneo-io-support 2026-11-03 14:06:19 -08:00
Om Alve 5e729d1236 Case battery: read the glasses battery soon after they dock
Only the settings reply says the glasses are charging. Outside charging
mode it is polled every five minutes, and not at all while the display
is suspended, which it is once the glasses come off the face. So after
docking, the phone kept showing the glasses as "Not charging", and
charging mode (which pauses display traffic) didn't start until the
display woke or the link reconnected.

The case push that reports the glasses going from out of the case to
docked arrives within a second. It now schedules a battery read about
three seconds later. The drive loop queues it as a pending message,
which still goes out while the display is suspended.
2026-11-02 12:30:39 +05:30
Om Alve 046454e8c0 Merge remote-tracking branch 'origin/main' into g2-case-battery
# Conflicts:
#	native/kotlin/shared/src/commonMain/kotlin/com/faceclaw/app/g2protocol/session/GlassesSessionCore.kt
2026-11-02 12:02:53 +05:30
Om Alve fa7f33bfe8 Case battery: stop showing stale charging; read after the layout
Review fixes for the charging-case battery:

- The case reports whether the glasses are docked. An empty caseInfo
  (the reply out of the case) now parses as "not docked, no level"
  instead of nothing, and undocking is reported even when the level and
  charge state are unchanged. TS keeps the last level but clears
  caseCharging while the glasses are out of the case.
- The phone line shows the charging mark only while it is live: G2 and
  Case only with a link to the glasses (connected or charging), and Case
  only while docked. Levels stay as the last known ones.
- The session-start case read waits until the layout is up, so a
  firmware that ignored it could no longer hold up the first frame. When
  the glasses are already charging at connect, the charging-mode read is
  the only one; the old duplicate read and its "late ACK" log are gone.
- Case dedup state resets on every connect, as wearState does, so each
  session's read reaches TS.
- caseCharging uses > 0 like the headset, and is normalized to 0/1.
- iOS now feeds the reading into the shell battery state, and the
  assistant's describeState includes the case level.
- The batteryLine accessors move above connectionStatusLabel's doc
  comment.
2026-11-02 12:01:12 +05:30
James Babcock 9ddd4c4d03 Add microphone support on RayNeo iO 2026-11-01 21:08:25 -08:00
James Babcock fde847a31a Add support for using RayNeo iO together with an R1 ring 2026-11-01 19:37:55 -08:00
James Babcock 4d101eefe1 Partial support for Rayneo iO 2026-11-01 17:19:55 -08:00
James Babcock 7289496c09 R1 ring: Debug health-data sync 2026-11-01 12:19:46 -08:00
James Babcock fcea0b369a Merge branch 'main' of https://github.com/jimrandomh/faceclaw into ring-health-fixes 2026-11-01 11:00:38 -08:00
Om Alve acfcf79787 Read the charging case battery (sid 0x81 GlassesCaseInfo)
The glasses relay the charging case's state on UX_GLASSES_CASE_APP_ID
(sid 0x81): GlassesCaseDataPackage{commandId=CASE_INFO, magicRandom,
caseInfo{soc, chargeStatus, lidStatus, inCaseStatus, errorCode}}, the
schema in g2-kit-unofficial's glasses_case.proto. The Even app reads it
with an empty caseInfo (ProtoBaseSettings.getGlassesCaseInfo).

Observed on G2 2.3.0.24 / Faceclaw/38: while docked the right temple
pushes a GlassesCaseInfo (flag 0x01, its own magic) on every change --
docking, lid, USB, each 1% -- and answers a read with full data; out of
the case a read comes back with an empty caseInfo. Docking pushes
{inCaseStatus=1} before the level, and proto3 drops zero fields, so a
missing soc means no level rather than 0%.

So read it once at session start (docked glasses may be fully charged
and never enter charging mode) and again on entering charging mode,
which clears the queue; pushes carry every later change. Every 0x81
frame is taken as state in onNotification (pushes skip the ack path by
flag). New readings reach TypeScript through
FaceclawBleCommunicatorListener.onCaseState (level, case-charging), on
Android and iOS. Captured frames are tests.
2026-11-01 20:38:32 +05:30
James Babcock 188bdff0d1 Port fixes from Glassly fork 2026-10-31 23:17:13 -07:00
James Babcock 3c3aa31542 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-31 17:05:00 -07:00
James Babcock 49ea8a7d74 Use Spotify API if an API key is provided 2026-10-31 17:03:09 -07:00
James Babcock 210b111e27 Unify Android and iOS config formats 2026-10-31 15:43:55 -07:00
James Babcock c5f80a0b00 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-31 14:04:41 -07:00
James Babcock a5eda29368 Port developer debug tests to iOS 2026-10-31 13:57:18 -07:00
James Babcock 032797f0f6 Add support for embedded icons and custom layouts in Android notifications
The practical upshot of this is that gmail, messages and similar apps
show the sender's profile picture on the glasses, the Dexcom app shows a
2026-10-31 12:39:20 -07:00
James Babcock 71bccd2c1c Ensure connection is re-set to 2M on reconnect 2026-10-27 13:05:13 -07:00
James Babcock 4f8c082b52 Merge branch 'main' into ring-health-fixes 2026-10-25 01:06:38 -07:00
James Babcock ce8b7656f4 Improve notification-icon display handling 2026-10-25 00:36:51 -07:00
James Babcock e36fb14a34 Add an Apple Music integration on iOS 2026-10-23 23:05:49 -07:00
James Babcock 40082b3f5b Show images attached to Android notifications 2026-10-23 22:26:36 -07:00
James Babcock de653f0fbb Hermes integration survives restarts, fix error hadling, increase turn length limit 2026-10-17 13:52:43 -07:00
James Babcock 5bf809d733 Bugfixes in iOS Transcribe and Whisper server integration 2026-10-17 13:43:52 -07:00
James Babcock e12ea8ee4c Merge commit 'ecb6c7' into fix-ios-crashes 2026-10-17 12:20:23 -07:00
FilipAlexanderandClaude Opus 5.5 ecb6c7f465 iOS: fix the audio-packet and window-close crashes, stop retrying the ring
- Rename IosAudioPacketListener.onAudioPacket to onNsDataAudioPacket. The
  selector collided with FaceclawAudioPacketListener's, Kotlin/Native mangled
  one of the two depending on export order, and the first glasses audio packet
  aborted the app with "unrecognized selector".
- Closing a window (e.g. Transcribe from its system menu) aborted the app: the
  menu pop re-renders the window, that frame is queued behind the surface
  removal, and Kotlin's compositor throws on the unknown surface. The bridge now
  tracks which surfaces Kotlin holds and drops stale calls inside the queue,
  submit re-checks the window is still open, and the IosGlassesSession surface
  wrappers log and drop compositor errors instead of aborting.
- Only connect to the R1 ring from the phone in "Direct" mode, as Android does.
  iOS always tried; the failing connect retried every few seconds and starved
  the glasses link, so the glasses image lagged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 16:42:41 +02:00
James Babcock 60595d7321 Fixes to EvenHub webview handling 2026-10-16 18:22:35 -07:00
James Babcock 2bb7b3ea1a Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-14 02:17:56 -07:00
James Babcock 60c87521b0 Fix a bunch of microphones app reliability issues
<details>
<summary>Prompts Used</summary>
User: Review the Microphones app, focusing on error handling and reliability.

Opus 5.5:
> I've finished reading the Microphones code and its native pieces. The 2026-10-14 commit `b50898e` fixed a lot, and the main problem left is the same in most places: the app reports "Listening" or "captions on" but never checks that audio is actually arriving. When something fails partway, the session usually stops working with no visible sign.
>
> ## High: silent loss of function or data
>
> **1. In stock mode, other mic users can take the mic away for good.**
> - `stopRawCapture()` tears down the single shared audio tap no matter who started it (`app/native/voice-control.ts:257`).
> - The EvenHub mic router calls it whenever a mic-using EvenHub app goes to the background or closes (`app/apps/evenhub/mic-router.ts:88`). That kills Microphones' audio.
> - Transcribe takes the mic without setting `voiceActivity` (`voice-control.ts:269`), so Microphones is never told to get it back.
> - If capture starts while the session isn't ready, the native controller only records the request, so no audio ever flows.
> - `checkExtendedCapture` (`mic-session.ts:456`) only covers 4-mic mode. Its comment says the stock path "is resumed by the voice-control bridge", but that only covers the session-dropped-while-active case.
> - In 4-mic mode the problem runs the other way: Transcribe takes over the shared Kotlin listener (`GlassesSessionCore.kt:470`) while the glasses keep sending 4-mic frames, so Transcribe gets audio in the wrong format.
> - **Fix:** track who holds the raw tap (like `captureHolders`), add a stock-mode branch to the 3 s check, and have Transcribe set the same "mic in use" signal as the assistant.
>
> **2. One bad utterance ends captions for the rest of the session.**
> - The caption worker has no per-utterance error handling. Any exception from `recognize()` or `embed()` stops the worker for good (`CaptionEngineCore.kt:334`).
> - `started` stays true, so TypeScript still thinks captions are running and recording carries on. The only sign is a "Captions failed:" status string.
> - **Fix:** catch errors per utterance in `onUtterance` (`:367`). Report a fatal worker exit through its own callback so the session can restart the engine or mark captions inactive.
>
> **3. A crash or low-memory kill during recording leaves files that are never shown or deleted.**
> - The WAV header is written with a data size of 0 (`FaceclawWavRecorder.kt:26`) and only corrected in `finish()`.
> - The file is only linked to its conversation in `stopRecording` (`mic-session.ts:1200`), and the conversation row is never closed.
> - Nothing cleans up `mic-recordings/`, so after a crash you get the same never-expired orphan files the last commit fixed for the captions-off case.
> - **Fix:** link the file to its conversation when recording starts. In the boot-time cleanup: close conversations left open (`ended_at IS NULL`), repair the WAV header from the file length, and delete files no conversation points to.
>
> **4. When the transcoder's success callback fails, it deletes both copies of the recording.**
> - `FaceclawAudioTranscoder.kt:66-83` deletes the WAV before calling `onDone`. If `onDone` throws, the catch block deletes the M4A too, and the database still points at the deleted WAV.
> - Separately, the JS listener is referenced only from Java (`mic-session.ts:1208`). Under `markingMode: 'none'` that's the situation `mic-control.ts:74` warns about.
> - I haven't confirmed the garbage-collection risk. `asr-model.ts` does the same thing and its downloads seem to work. But the delete order is wrong either way.
> - `rediarize.ts` and `mic-models.ts` hold their listeners the same way.
>
> ## Medium
>
> 5. **The last utterance is lost when you close the app or turn captions off.** When the caption engine stops, it flushes the utterance in progress and posts it to the main thread. By the time it arrives, `stop()` has already run `finishSessionRow()` (`mic-session.ts:328`), so the line is never saved and is missing from the session insights. If decoding takes longer than the 2 s wait, the line can land in the *next* session with wrong times. Also, `stop()` can block the UI for up to 2 s.
> 6. **The beam filter can mute 4-mic captions permanently.** The direction and signal strength only update on frames with a non-zero angle or strength (`:685`), and they never go stale. If you turn off "Direction detection", the last direction stays frozen. If it was outside the beam, `deliverExtendedAudio` drops all audio from then on (`:751-757`). Stock mode avoids this because it updates the strength on every frame. **Fix:** timestamp the fused direction and treat it as unknown after `TEMPLE_STALE_MS`.
> 7. **The 4-mic health check looks at a phone-side flag, not at whether audio is arriving.**
>    - The check only re-arms when `isAudioCaptureActive()` is false.
>    - A configure command can be skipped while the session isn't ready even though `send()` returns true (`mic-control.ts:155`, `GlassesSessionCore.kt:890`).
>    - A temple can reboot on its own, or `apply()` can refuse to arm when the developer setting is turned off mid-session.
>    - Each of these leaves forwarding "active" with no frames arriving.
>    - **Fix:** you already track `templeFrameAtMs`. Re-send the config when an enabled temple has gone quiet, and show that in the status instead of "Listening (4-mic array)".
> 8. **A failed start is permanent.** `start()` returns before installing any timers (`:280`). A failed mode switch mid-session (`:376`) leaves `forwardingProxy` null, so the health check exits at `:457` every time. **Fix:** stay in a "waiting for glasses" state and let the timer retry.
> 9. **The caption feed can flip between left and right on every frame.** The choice at `:744` doesn't take the current side into account, and `strongerChanged` is just a `!==` check. Near the 1.5× threshold, each flip splices about 25 ms from the other temple (which runs on its own clock) into captions and the recording, either repeating or skipping audio.
>
> ## Lower
> - **Unguarded speaker call (`:1034`):** the speaker rename for a pending introduction runs before the line is added. A database error drops the line and leaves the introduction pending, so the next 2 minutes of lines from unnamed speakers fail the same way. Add the line first, then do the extra steps each with its own error handling.
> - **Speaker list grows forever:** every unknown voice adds a permanent "Speaker N" with a voiceprint. Retention never prunes them. `assign()` reloads and decodes every profile on every utterance, on the main thread.
> - **Translation:** there's no timeout, so lines can wait indefinitely while a model download is stuck offline. The detected language is only saved after translation finishes (`:1132`). The "hysteresis" comment at `:1120` describes logic that doesn't exist: `conversationLang` is written but never read.
> - **Frame fields ignored:** the sequence number (but `TempleAudioPipeline` assumes consecutive frames), the truncated flag, and `sampleRateHz`. The UI offers 8–48 kHz while the code assumes 16 kHz.
> - **Duplicated model path:** `moonshineModelDir()` (`:1337`) copies `asr-model.ts`'s directory name. A model version bump there would quietly drop captions to "no ASR model".
> - **Silent recording gaps:** `WavRecorder.append` swallows each write error, so a full disk means log spam and silent gaps in the recording.
>
> The startup order with cleanup on failure, the 90 s lease that recovers on its own from a lost STOP, and keeping the last good speaker cache all hold up well.
>
> I worked from the code only; nothing was tested on hardware. #6 relies on the protocol comment saying the angle is 0 when direction detection is off.
>
> I'd fix #1, #2, #5 and #6 first, since those are the ones you'd hit in normal use. Want me to start on them?

<hr>

User: Thanks. Let's fix all of the high and medium issues you found now.

Opus 5.5:
> All nine high and medium issues are fixed. TS typecheck, lint, 813 TS tests, the Kotlin host tests (4 new), the iOS Kotlin compile and `build.sh` all pass, and I confirmed the new code is in the APK. Nothing has been tested on the glasses yet, and nothing is committed.
>
> **High**
> 1. **Mic taken by other apps:**
>    - The shared audio tap now tracks who's using it, so an EvenHub app backgrounding no longer cuts off Microphones.
>    - Transcribe now marks the mic as in use, like the assistant does, so Microphones and EvenHub apps step aside while it runs.
>    - A check every 3 s now covers both modes. It restarts a dead stock tap and switches back to stock mode if the developer setting is turned off mid-session.
> 2. **One bad utterance ending captions:** a decode error now skips that utterance instead of stopping the caption engine (`CaptionEngineCore.kt`). If the engine dies anyway, it's restarted on the same conversation up to 3 times. After that, the Captions toggle shows off and the view says why; turning it on retries.
> 3. **Recordings orphaned by a crash:** the recording is linked to its conversation as soon as it starts. At boot, the app closes conversations a crash left open, repairs their WAV headers (`FaceclawWavRecorder.repairHeader`), transcodes them, and deletes files no conversation points to. That cleanup matches by file name, so moving the app to other storage can't delete live recordings.
> 4. **Transcoder deleting both copies:** a failure while recording the new file no longer deletes both files. The app now deletes the WAV itself, only after the conversation points at the M4A. The transcoder, re-diarize and model-download callbacks are now held by the app until they fire.
>
> **Medium**
> 5. **Last utterance lost:** stopping captions no longer blocks the UI. The engine finishes the audio still queued, then sends a new `onStopped` callback, and the conversation stays open until that arrives (20 s fallback). A late line from a previous session won't show up in a newly reopened one.
> 6. **Beam filter muting captions:** the direction of arrival only filters audio while it's under 1 s old, and it resets when "Direction detection" changes.
> 7. **4-mic health check:** a temple that sends no audio for 5 s gets its config re-sent, to that temple only. Mic-control sends now report when they weren't actually queued. The status line says when no audio is arriving, but not after routine restarts like the voice dialog closing.
> 8. **Start failures being permanent:** the session now waits ("Waiting for the glasses microphone...") and the 3 s check retries. Captions start only once the mic is up, so waiting doesn't create empty conversations.
> 9. **Left/right flapping:** the caption feed only switches temples when the other one is clearly louder for 500 ms.
>
> Behaviour changes you'll notice:
> - The watch shows "Listening…" during Transcribe.
> - EvenHub mic apps get no audio while Transcribe runs.
> - In 4-mic mode, an EvenHub mic app still takes over the single native audio listener. Fixing that needs a change on the Kotlin side, so I left it.
>
> On hardware, I'd test:
> - The assistant, Transcribe and an EvenHub mic app taking the mic while Microphones is open, then giving it back.
> - Reconnecting in both stock and 4-mic modes.
> - Killing the app mid-recording, then checking the conversation is recovered at the next start.
>
> I saved these notes to memory as `microphones-reliability.md`.
</details>
2026-10-14 02:17:05 -07:00
James Babcock bf92a5a2ce Clear notification popups when they're dismissed on the phone, and avoid showing some notifications that are spurious 2026-10-14 02:05:23 -07:00
James Babcock 13da595de0 Fix a bug where album art and track info could get out of sync if you advance twice quickly 2026-10-13 16:17:07 -07:00
James Babcock 406cefe0c5 Add stats collection and first-run on-glasses onboarding app 2026-10-10 17:33:32 -07:00
James Babcock d7a02f25fb Autoconfigure ring for direct connection 2026-10-06 22:29:03 -07:00
James Babcock f38595422c Build out Ring Status debug tool; fix some aspects of ring direct communication 2026-10-06 19:53:43 -07:00
James Babcock e12cfaaf38 Merge branch 'main' of https://github.com/jimrandomh/faceclaw into ring-health-fixes 2026-10-06 13:32:49 -07:00
James Babcock f210489fd4 Merge branch 'idempotent-evenhub-connect' of https://github.com/jimrandomh/faceclaw into idempotent-evenhub-connect 2026-10-06 09:37:27 -07:00
AI agent (Serac) and James Babcock (jimrandomh) ca2e55e70e Make reconnection more resilient by making the mode/layout setup idempotent 2026-10-05 18:22:37 -07:00
James Babcock 38da8e6168 Fix several bugs that could lead to reconnect-looping
Fix misinterpretation of EvenHub event type 5.
Notification modals stacked, in a way that's O(n) in device resource memory; replace with a queue.
2026-10-05 01:36:40 -07:00
James Babcock 4234d3e9e9 Fix an issue where animations on the phone preview stopped prematurely 2026-10-03 15:54:57 -07:00
James Babcock 66ca06f584 Refactor iOS glasses connection handling to shared code, fixing incompatible-version-detection and other error cases 2026-10-03 15:08:37 -07:00