Commit Graph
634 Commits
Author SHA1 Message Date
FilipAlexanderandClaude Opus 5.5 0a9625806e iOS: IOS_BUNDLE_ID overrides the bundle ID for sideloading
Building under another signing team needs a bundle ID that team owns.
IOS_BUNDLE_ID (build_paths.sh) sets the iOS app ID, defaulting to
com.faceclaw.app, and the iOS config transfer scripts use it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 16:42:42 +02:00
James Babcock 2bb7b3ea1a Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-14 02:17:56 -07:00
James Babcock 60c87521b0 Fix a bunch of microphones app reliability issues
<details>
<summary>Prompts Used</summary>
User: Review the Microphones app, focusing on error handling and reliability.

Opus 5.5:
> I've finished reading the Microphones code and its native pieces. The 2026-10-14 commit `b50898e` fixed a lot, and the main problem left is the same in most places: the app reports "Listening" or "captions on" but never checks that audio is actually arriving. When something fails partway, the session usually stops working with no visible sign.
>
> ## High: silent loss of function or data
>
> **1. In stock mode, other mic users can take the mic away for good.**
> - `stopRawCapture()` tears down the single shared audio tap no matter who started it (`app/native/voice-control.ts:257`).
> - The EvenHub mic router calls it whenever a mic-using EvenHub app goes to the background or closes (`app/apps/evenhub/mic-router.ts:88`). That kills Microphones' audio.
> - Transcribe takes the mic without setting `voiceActivity` (`voice-control.ts:269`), so Microphones is never told to get it back.
> - If capture starts while the session isn't ready, the native controller only records the request, so no audio ever flows.
> - `checkExtendedCapture` (`mic-session.ts:456`) only covers 4-mic mode. Its comment says the stock path "is resumed by the voice-control bridge", but that only covers the session-dropped-while-active case.
> - In 4-mic mode the problem runs the other way: Transcribe takes over the shared Kotlin listener (`GlassesSessionCore.kt:470`) while the glasses keep sending 4-mic frames, so Transcribe gets audio in the wrong format.
> - **Fix:** track who holds the raw tap (like `captureHolders`), add a stock-mode branch to the 3 s check, and have Transcribe set the same "mic in use" signal as the assistant.
>
> **2. One bad utterance ends captions for the rest of the session.**
> - The caption worker has no per-utterance error handling. Any exception from `recognize()` or `embed()` stops the worker for good (`CaptionEngineCore.kt:334`).
> - `started` stays true, so TypeScript still thinks captions are running and recording carries on. The only sign is a "Captions failed:" status string.
> - **Fix:** catch errors per utterance in `onUtterance` (`:367`). Report a fatal worker exit through its own callback so the session can restart the engine or mark captions inactive.
>
> **3. A crash or low-memory kill during recording leaves files that are never shown or deleted.**
> - The WAV header is written with a data size of 0 (`FaceclawWavRecorder.kt:26`) and only corrected in `finish()`.
> - The file is only linked to its conversation in `stopRecording` (`mic-session.ts:1200`), and the conversation row is never closed.
> - Nothing cleans up `mic-recordings/`, so after a crash you get the same never-expired orphan files the last commit fixed for the captions-off case.
> - **Fix:** link the file to its conversation when recording starts. In the boot-time cleanup: close conversations left open (`ended_at IS NULL`), repair the WAV header from the file length, and delete files no conversation points to.
>
> **4. When the transcoder's success callback fails, it deletes both copies of the recording.**
> - `FaceclawAudioTranscoder.kt:66-83` deletes the WAV before calling `onDone`. If `onDone` throws, the catch block deletes the M4A too, and the database still points at the deleted WAV.
> - Separately, the JS listener is referenced only from Java (`mic-session.ts:1208`). Under `markingMode: 'none'` that's the situation `mic-control.ts:74` warns about.
> - I haven't confirmed the garbage-collection risk. `asr-model.ts` does the same thing and its downloads seem to work. But the delete order is wrong either way.
> - `rediarize.ts` and `mic-models.ts` hold their listeners the same way.
>
> ## Medium
>
> 5. **The last utterance is lost when you close the app or turn captions off.** When the caption engine stops, it flushes the utterance in progress and posts it to the main thread. By the time it arrives, `stop()` has already run `finishSessionRow()` (`mic-session.ts:328`), so the line is never saved and is missing from the session insights. If decoding takes longer than the 2 s wait, the line can land in the *next* session with wrong times. Also, `stop()` can block the UI for up to 2 s.
> 6. **The beam filter can mute 4-mic captions permanently.** The direction and signal strength only update on frames with a non-zero angle or strength (`:685`), and they never go stale. If you turn off "Direction detection", the last direction stays frozen. If it was outside the beam, `deliverExtendedAudio` drops all audio from then on (`:751-757`). Stock mode avoids this because it updates the strength on every frame. **Fix:** timestamp the fused direction and treat it as unknown after `TEMPLE_STALE_MS`.
> 7. **The 4-mic health check looks at a phone-side flag, not at whether audio is arriving.**
>    - The check only re-arms when `isAudioCaptureActive()` is false.
>    - A configure command can be skipped while the session isn't ready even though `send()` returns true (`mic-control.ts:155`, `GlassesSessionCore.kt:890`).
>    - A temple can reboot on its own, or `apply()` can refuse to arm when the developer setting is turned off mid-session.
>    - Each of these leaves forwarding "active" with no frames arriving.
>    - **Fix:** you already track `templeFrameAtMs`. Re-send the config when an enabled temple has gone quiet, and show that in the status instead of "Listening (4-mic array)".
> 8. **A failed start is permanent.** `start()` returns before installing any timers (`:280`). A failed mode switch mid-session (`:376`) leaves `forwardingProxy` null, so the health check exits at `:457` every time. **Fix:** stay in a "waiting for glasses" state and let the timer retry.
> 9. **The caption feed can flip between left and right on every frame.** The choice at `:744` doesn't take the current side into account, and `strongerChanged` is just a `!==` check. Near the 1.5× threshold, each flip splices about 25 ms from the other temple (which runs on its own clock) into captions and the recording, either repeating or skipping audio.
>
> ## Lower
> - **Unguarded speaker call (`:1034`):** the speaker rename for a pending introduction runs before the line is added. A database error drops the line and leaves the introduction pending, so the next 2 minutes of lines from unnamed speakers fail the same way. Add the line first, then do the extra steps each with its own error handling.
> - **Speaker list grows forever:** every unknown voice adds a permanent "Speaker N" with a voiceprint. Retention never prunes them. `assign()` reloads and decodes every profile on every utterance, on the main thread.
> - **Translation:** there's no timeout, so lines can wait indefinitely while a model download is stuck offline. The detected language is only saved after translation finishes (`:1132`). The "hysteresis" comment at `:1120` describes logic that doesn't exist: `conversationLang` is written but never read.
> - **Frame fields ignored:** the sequence number (but `TempleAudioPipeline` assumes consecutive frames), the truncated flag, and `sampleRateHz`. The UI offers 8–48 kHz while the code assumes 16 kHz.
> - **Duplicated model path:** `moonshineModelDir()` (`:1337`) copies `asr-model.ts`'s directory name. A model version bump there would quietly drop captions to "no ASR model".
> - **Silent recording gaps:** `WavRecorder.append` swallows each write error, so a full disk means log spam and silent gaps in the recording.
>
> The startup order with cleanup on failure, the 90 s lease that recovers on its own from a lost STOP, and keeping the last good speaker cache all hold up well.
>
> I worked from the code only; nothing was tested on hardware. #6 relies on the protocol comment saying the angle is 0 when direction detection is off.
>
> I'd fix #1, #2, #5 and #6 first, since those are the ones you'd hit in normal use. Want me to start on them?

<hr>

User: Thanks. Let's fix all of the high and medium issues you found now.

Opus 5.5:
> All nine high and medium issues are fixed. TS typecheck, lint, 813 TS tests, the Kotlin host tests (4 new), the iOS Kotlin compile and `build.sh` all pass, and I confirmed the new code is in the APK. Nothing has been tested on the glasses yet, and nothing is committed.
>
> **High**
> 1. **Mic taken by other apps:**
>    - The shared audio tap now tracks who's using it, so an EvenHub app backgrounding no longer cuts off Microphones.
>    - Transcribe now marks the mic as in use, like the assistant does, so Microphones and EvenHub apps step aside while it runs.
>    - A check every 3 s now covers both modes. It restarts a dead stock tap and switches back to stock mode if the developer setting is turned off mid-session.
> 2. **One bad utterance ending captions:** a decode error now skips that utterance instead of stopping the caption engine (`CaptionEngineCore.kt`). If the engine dies anyway, it's restarted on the same conversation up to 3 times. After that, the Captions toggle shows off and the view says why; turning it on retries.
> 3. **Recordings orphaned by a crash:** the recording is linked to its conversation as soon as it starts. At boot, the app closes conversations a crash left open, repairs their WAV headers (`FaceclawWavRecorder.repairHeader`), transcodes them, and deletes files no conversation points to. That cleanup matches by file name, so moving the app to other storage can't delete live recordings.
> 4. **Transcoder deleting both copies:** a failure while recording the new file no longer deletes both files. The app now deletes the WAV itself, only after the conversation points at the M4A. The transcoder, re-diarize and model-download callbacks are now held by the app until they fire.
>
> **Medium**
> 5. **Last utterance lost:** stopping captions no longer blocks the UI. The engine finishes the audio still queued, then sends a new `onStopped` callback, and the conversation stays open until that arrives (20 s fallback). A late line from a previous session won't show up in a newly reopened one.
> 6. **Beam filter muting captions:** the direction of arrival only filters audio while it's under 1 s old, and it resets when "Direction detection" changes.
> 7. **4-mic health check:** a temple that sends no audio for 5 s gets its config re-sent, to that temple only. Mic-control sends now report when they weren't actually queued. The status line says when no audio is arriving, but not after routine restarts like the voice dialog closing.
> 8. **Start failures being permanent:** the session now waits ("Waiting for the glasses microphone...") and the 3 s check retries. Captions start only once the mic is up, so waiting doesn't create empty conversations.
> 9. **Left/right flapping:** the caption feed only switches temples when the other one is clearly louder for 500 ms.
>
> Behaviour changes you'll notice:
> - The watch shows "Listening…" during Transcribe.
> - EvenHub mic apps get no audio while Transcribe runs.
> - In 4-mic mode, an EvenHub mic app still takes over the single native audio listener. Fixing that needs a change on the Kotlin side, so I left it.
>
> On hardware, I'd test:
> - The assistant, Transcribe and an EvenHub mic app taking the mic while Microphones is open, then giving it back.
> - Reconnecting in both stock and 4-mic modes.
> - Killing the app mid-recording, then checking the conversation is recovered at the next start.
>
> I saved these notes to memory as `microphones-reliability.md`.
</details>
2026-10-14 02:17:05 -07:00
James Babcock 21c6f6f99c Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-14 02:05:26 -07:00
James Babcock bf92a5a2ce Clear notification popups when they're dismissed on the phone, and avoid showing some notifications that are spurious 2026-10-14 02:05:23 -07:00
James Babcock 63e0d71e2f Clear notification popups when they're dismissed on the phone, and avoid showing some notifications that are spurious 2026-10-14 02:03:51 -07:00
James Babcock 42c2d5cd55 Add depth aeffects to menu modals 2026-10-14 01:44:52 -07:00
James Babcock a01d8bd296 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-14 01:35:38 -07:00
James Babcock 9a0c843bc7 Status bar timer widget is selectable 2026-10-14 01:35:15 -07:00
James Babcock 08094a3be9 Add daily page date-picker to Roam app 2026-10-14 01:32:21 -07:00
James Babcock 132b90c21d Status bar widgets are selectable 2026-10-14 01:31:10 -07:00
James Babcock 8ae4b423b6 Glanceboard Terminal widget doesn't re-sort terminals while open 2026-10-14 01:21:51 -07:00
James Babcock 2f880479be Add __pycache__ to gitignore 2026-10-14 01:16:37 -07:00
James Babcock 5436744549 Fix "array status" and several other pages in the "Microphone setup" app not rendering when not full-screen 2026-10-14 01:15:51 -07:00
James Babcock 54f4852564 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-14 01:02:12 -07:00
James Babcock f01214f4a0 Phone screen input shows gesture help for wake (and not random other gesture-help) 2026-10-14 01:01:41 -07:00
James Babcock b50898ea02 Fix a bunch of microphone-related bugs
> Most serious fixes
> - Recordings kept forever: a recording made with captions off, or with "Save captions"
>   off, was transcoded to .m4a and kept indefinitely. No conversation linked to it, so
>   the phone UI never showed it and the retention settings never deleted it. Recording
>   now only happens alongside a saved caption session, and turning captions off ends
>   that session's recording properly.
> - 4-mic mode went silent after a reconnect: a disconnect clears the phone's audio
>   forwarding, and nothing turned it back on, so the app kept showing "Listening (4-mic
>   array)" while hearing nothing. A check every 3 s now restores forwarding and re-sends
>   the config when it has been lost. The normal single-stream mode already recovered on
>   its own.
> - Turning off "4-mic capture" mid-session went silent: it stopped the glasses' stream
>   but never switched to the normal stream. Setup changes now take effect live,
>   including switching between the two modes.
> - Setup changes were undone: after the assistant closed, the app re-sent the config it
>   loaded at launch to the glasses, reverting any changes you'd made since.
> - Startup failures: if the caption engine failed to start, the session got stuck
>   half-running. If the glasses weren't connected, listeners piled up and an empty
>   conversation could be left open. Startup now brings up the microphone first and
>   cleans up properly on failure.
>
> Other fixes
> - Audio quality: the 4-mic beamformer put a small dropout at the start of every 25 ms
>   frame (an audible 40 Hz artifact in the audio feeding captions). Fixed, with a test
>   that fails on the old code.
> - Stale direction data: an arm that stopped sending kept steering the direction
>   estimate and could hold the caption feed on a silent arm.
> - Compass: closing Microphones switched off the Compass app's compass, because both
>   used the same owner name.
> - Speaker database failures:
>   - A failed speaker reload wiped the in-memory list, so the next voice became a
>     duplicate "Speaker N".
>   - A failed speaker insert crashed caption handling; in re-diarize it reassigned
>     transcript lines to a nonexistent speaker.
> - Translation:
>   - A crash path inside the model-download callback is now caught.
>   - Phones set to Hebrew, Indonesian or Yiddish never got translations, because of
>     old-style language codes.
> - Ask questions: once the speaker list grew past the size limit (every unknown voice
>   adds a "Speaker N"), it pushed all transcript lines out of the context sent to the
>   model. When trimming is needed, the oldest lines now go first.
> - Smaller fixes:
>   - The captions view stopped jumping back to live speech after 60 lines.
>   - "This is Dana calling…" was treated as introducing someone else.
>   - A model download that failed to start was stuck at "downloading" until restart.
>   - The developer mic monitor grew without limit while hidden.
2026-10-14 00:56:54 -07:00
James Babcock 22c31f4240 Add calibration indicator to glanceboard compass 2026-10-13 17:21:46 -07:00
James Babcock df95cc4064 Add configuration options to the glanceboard system card and music card 2026-10-13 16:41:14 -07:00
James Babcock 13da595de0 Fix a bug where album art and track info could get out of sync if you advance twice quickly 2026-10-13 16:17:07 -07:00
James Babcock eb91d46657 Add thread options to t3 client 2026-10-13 00:46:54 -07:00
James Babcock ea9249422c Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-13 00:21:12 -07:00
James Babcock 26e4396972 Multi-row status/app-switcher bar layouts 2026-10-13 00:21:01 -07:00
James Babcock 96e3bd95f7 Center the phone screen preview 2026-10-12 17:12:36 -07:00
James Babcock f6201d8e41 Fix a spurious unit test failure 2026-10-12 12:22:31 -07:00
James Babcock ccbf523e9a Add EvenHub icon to phone preview screen 2026-10-12 02:33:22 -07:00
James Babcock 14d1f7f509 0.8.3 2026-10-10 23:04:12 -07:00
James Babcock f6a4615953 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-10 22:56:28 -07:00
James Babcock d01dbf5432 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-10 22:56:22 -07:00
James Babcock 920cce6a26 Add cloud voice model support for iOS 2026-10-10 22:56:07 -07:00
James Babcock 5a9ca08921 Unify vertical position and display mode settings; add to onboarding 2026-10-10 22:48:26 -07:00
James Babcock 6b1e64c5ac Onboarding: Add layout and depth pages 2026-10-10 21:52:19 -07:00
James Babcock 1d257e8bb6 Fix spurious unit test failures 2026-10-10 20:11:38 -07:00
James Babcock 8f0d4b0f8c Match iPhone signup requirements to TestFlight 2026-10-10 18:43:37 -07:00
James Babcock e5f6dc1976 Prepare 0.8.3-rc1 TestFlight beta and public signup 2026-10-10 18:43:02 -07:00
James Babcock 5df873e5b1 Merge branch 'main' of https://github.com/jimrandomh/faceclaw 2026-10-10 17:34:46 -07:00
James Babcock 58990d9617 Fix iOS styling issues on pairing screen 2026-10-10 17:34:06 -07:00
James Babcock 406cefe0c5 Add stats collection and first-run on-glasses onboarding app 2026-10-10 17:33:32 -07:00
James Babcock d6180e4457 Use ANCS notification categories for icons 2026-10-10 17:04:22 -07:00
James Babcock 84baee9c76 EvenHub extensions: Fonts, window icons, touch-start events 2026-10-07 01:22:42 -07:00
James Babcock 4ee0af86cc EvenHub API extensions: Set package name, fix some bugs 2026-10-06 23:02:08 -07:00
James Babcock 1e64a32c55 Update CHANGELOG 2026-10-06 10:58:56 -07:00
James Babcock f210489fd4 Merge branch 'idempotent-evenhub-connect' of https://github.com/jimrandomh/faceclaw into idempotent-evenhub-connect 2026-10-06 09:37:27 -07:00
AI agent (Serac) and James Babcock (jimrandomh) ca2e55e70e Make reconnection more resilient by making the mode/layout setup idempotent 2026-10-05 18:22:37 -07:00
James Babcock 38da8e6168 Fix several bugs that could lead to reconnect-looping
Fix misinterpretation of EvenHub event type 5.
Notification modals stacked, in a way that's O(n) in device resource memory; replace with a queue.
2026-10-05 01:36:40 -07:00
James Babcock e5303e7b3e Fix gitleaks false positive 2026-10-04 17:57:26 -07:00
James Babcock 9cf180e5a1 Merge branch 'fancy-weather' of https://github.com/jimrandomh/faceclaw into merged-testing 2026-10-04 03:13:50 -07:00
James Babcock 6db913af48 Weather app on glanceboard system card and status bar, and has an image 2026-10-04 03:08:37 -07:00
James Babcock 49a49d68ce Add glanceboard widget for T3code 2026-10-04 03:07:38 -07:00
James Babcock 043e5e74ce Merge branch 'main' of https://github.com/jimrandomh/faceclaw into t3code 2026-10-04 02:45:36 -07:00