Files
hyprland-headless-stream-pl…/docs/plans/2026-10-03-keyboard-input.md
T
HermenixandClaude Opus 5 561e0c28f5 Stage 2 fixes: the two tests the review named, and the docs drift
From docs/reviews/2026-10-04-stage2-3-review.md (Stage 3 done; Stage 2
done-with-fixes). Its findings 1, 2, 3 and 5, which it called the only
things standing before merge.

(A) REPLAY, THROUGH THE SINK. What existed was
    `test_every_keyboard_verb_is_cached_for_the_same_reason`, which
    asserts the two verbs are in MUTATING_VERBS -- and the verification
    doc had already named that as the false pass: "asserting
    `deduplicated` and not the sink. A handler that types and then sets
    the flag stays green." Two integration tests now, modelled on
    `test_a_retried_click_with_the_same_request_id_clicks_once`:

      test_a_retried_type_with_the_same_request_id_types_once
        `keyboard.type` of "ab" twice with request_id="retry-me".
        Asserts `deduplicated`, chars == 2 on both, and that the sink
        still has exactly 4 key events -- two characters, a down and an
        up each -- so a replay that typed again is caught by the count
        and not by the flag.

      test_a_retried_unmappable_type_stays_refused_and_types_nothing
        `café` twice with one id. Both refusals are `invalid_argument`,
        the second carries `deduplicated` (a recorded failure is
        replayed as the failure it was), and the sink is empty after
        both -- there is no partial type on either call.

    Mutation-checked rather than assumed: removing `keyboard.key` and
    `keyboard.type` from MUTATING_VERBS fails both.

(B) KILL, THROUGH kill(). `test_killing_the_holder_lifts_its_keys` calls
    `_drop_lease_for` directly, so it proves that helper lifts keys and
    says nothing about whether `kill` still calls it. The integration
    kill test now holds `Shift_L` before the kill and asserts the SINK
    saw `("key", 42, 0)` and `("modifiers", 0, 0, 0, 0)` -- the lease
    forgetting a key is not the compositor being told it came up, and
    only the second matters to the application. Code 42 because `type`
    never emits it, so the release cannot be a leftover.

    Mutation-checked: stubbing out the `_drop_lease_for` call in
    `kill()` fails it. Before this it stayed green.

(C) DOCS. The plan's stage table and risk paragraph still said acquire
    is one warp and that `focuswindow` was unbuilt Stage 4 work. The
    contract (v3 §4) describes what shipped; the plan did not. Rewritten
    so they agree, and the plan now says outright that the contract wins
    where they differ:

      * risk 1 marked RESOLVED, describing the two warps, the pinned
        `follow_mouse_threshold`, and `focuswindow` as the
        nudge-failure fallback selecting on `address:` (not
        `stable_id`, which is not a Hyprland selector)
      * Stage 3 marked Done; Stage 4 struck as not existing -- the
        answer to Stage 0 was the second warp, not a focus dispatch
      * the top banner, the §5 VM assertion list and open question 2
        no longer say "one warp"
      * verification §B3 subtest 1 amended: it is a live assertion
        rather than an expected red, and the check additionally requires
        `focus == "warp"` so the fallback is not accepted as success

    Also amended one sentence the review did not list, because leaving
    it would contradict the edit above inside the same document:
    verification §"Warp does not guarantee wl_keyboard.enter" predicted
    `focuswindow` was "likely mandatory". It was not; the second warp
    was.

    Written down rather than smoothed over: the `focuswindow` branch
    itself has no test, and if both mechanisms fail acquire still
    returns ok with `focus="none"` -- a lease whose first key may go to
    the previously focused window.

(D) The tap docstring claimed the `up` is sent even if the `down` was
    refused after sending. The next lines return early on `not ok`, and
    that early return is right: a refusal means the key was never
    pressed, so an `up` would ask the compositor to release something it
    never had. Sentence replaced with why the early return is correct
    and what actually covers a lost reply (the broker's `release_all`,
    not the CLI, which cannot know whether the request arrived).

Not done, and not claimed: findings 4, 6 and 7 (the broker chord test
checking first/last mask rather than the key; the at-cap test not
looking at the sink; the module-defaults test being a substring match).
The review rates all three as not holes in the claim, and the driver
tests are the locks for the first two.

  nix develop -c python -m pytest tests -q
    907 passed in 67.71s
  nix develop -c python -m pytest tests/pointer tests/sessions/unit -q
    605 passed in 27.16s

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GhNPM6rGcuzxocM1nATnuM
2026-10-05 04:43:56 +00:00

341 lines
41 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Keyboard input for sessions
> **Superseded in eight places. Read this first.**
>
> `docs/plans/2026-10-04-keyboard-verification.md` is an adversarial review of
> this file against the tree and against Hyprland v0.56.2. Its verdict is "safe
> to build on, not safe to implement as written", and the eight rows of its
> **"Changes required before code"** table **override this document wherever
> they differ**. Each one is also folded into the text below, marked
> `[SUPERSEDED]`, so nobody following a single paragraph gets the contradicted
> version:
>
> | # | What changed |
> | --- | --- |
> | 1 | Exit 2 vs `driver_refused` — §2's exit-code paragraph was wrong about what `_drive` does |
> | 2 | The keymap `size` includes the trailing NUL; `format = 1`; `time` is `_now_ms()` |
> | 3 | `release_all` no-ops its key half with no keyboard, and ends with `modifiers(0,0,0,0)` |
> | 4 | `input:virtualkeyboard:release_pressed_on_close = true` — destroy-while-held is no longer an open question |
> | 5 | The layout reaches the keymap **only** as `--xkb-layout` on the driver unit; no `[keyboard]` TOML table |
> | 6 | The modifier-keysym set is measured from the keymap; no Shift special case |
> | 7 | Stage 0's expectations, and `keyboard down` warns unconditionally |
> | 8 | Acquire's stderr mentions keyboard verbs; refusals carry a scalar index, never the string |
>
> **Stage 0 has run** (`tests/README-keyboard-probe.md`, with its transcript),
> and Stages 1–3 are built. The three open behavioural questions are answered,
> and one answer was worse than this document assumed: **a single idle warp
> does not move keyboard focus.** That is resolved rather than outstanding —
> `pointer acquire` now sends **two** warps a real pixel apart, and
> `nix/hyprland-conf.nix` pins the `follow_mouse_threshold` the second one
> depends on. See §6 risk 1, which is rewritten, and the Stage 3/4 rows, which
> are not pending work.
>
> **Where this file and `docs/CONTRACT-sessions.md` v3 disagree, the contract
> wins.** It describes what shipped; several paragraphs here describe what was
> planned before Stage 0 measured the compositor.
Typing stops being a non-goal. `docs/PLAN-workspaces-and-pointer.md` §1 defers it, and §6.7 already specifies the shape: same seat, same serialisation, `zwp_virtual_keyboard_manager_v1`, and the existing pointer lease rather than a new lock. Stages 0–3 of that plan are what the tree implements today (broker lease in `src/hhstreamd/pointer.py`, driver in `src/hhstream_pointer/`, CLI in `src/hhstream_session/__init__.py`). This plan is §6.7, with one correction noted under lease semantics.
The Chrome path in `examples/agent-eyes-demo.py` (`Input.insertText`) is not this feature. It never touches the seat. This plan is for whatever window is focused on the compositor's one seat.
---
## 1. Protocol
**Decision: `zwp_virtual_keyboard_v1`, spoken by the existing `hhstream-pointer` process. Not `wp_virtual_keyboard_v1`, not uinput/`ydotool`, not `wtype`.**
| Option | Verdict |
| --- | --- |
| `zwp_virtual_keyboard_v1` | Use it. Unstable wayland-protocols interface (`virtual-keyboard-unstable-v1.xml`). Manager global `zwp_virtual_keyboard_manager_v1`. Requests are `keymap(format, fd, size)`, `key(time, keycode, state)`, `modifiers(depressed, latched, locked, group)`. Version 1 only. |
| `wp_virtual_keyboard_v1` | Do not bind it. Nothing in this repo claims that global. A stable `wp_` rename is not what §6.7 measured. |
| uinput / `ydotool` | Reject. Needs `/dev/uinput` and a seat that actually reads that device. The headless compositor is driven as a Wayland client today (`WaylandSink` in `src/hhstream_pointer/backend.py`); the driver runs as `hermes` with group `hhstream` so it can open the Wayland socket (`nix/module.nix`). A kernel device is a second privilege and a second focus story, and it cannot be unit-tested as a recording sink. |
| `wtype` | Reject as the mechanism. It is a short-lived client of the same `zwp_` protocol: new virtual keyboard per invocation, its own keymap, no `release_all` across calls, no lease. Useful only as a reference for how text becomes keycodes. |
**What the running compositor exposes.** Not re-checked in this session (no registry dump). The only evidence in-tree is §6.7's **[RAN]** claim, against the Hyprland it measured (plan header: tag `v0.56.2`): `zwp_virtual_keyboard_manager_v1` is a live global. `wp_virtual_keyboard_v1` is not mentioned. Treat that as unverified until Stage 0 dumps the globals on `/run/hypr-stream`. If the dump shows a different name, stop. Do not probe uinput as a fallback.
**Tradeoffs, as they apply here:**
- **Keycode, not keysym, on the wire to the compositor.** `key` takes a Linux evdev code (the same value `wl_keyboard.key` carries; clients add 8 to reach an XKB keycode). Keysym names stay on our socket, the way button names stay on ours and evdev codes cross into `zwlr_virtual_pointer_v1` (`src/hhstream_pointer/buttons.py`).
- **Keymap upload is mandatory.** Until `keymap` is sent (format `xkb_v1`, memfd of an XKB text keymap), clients cannot decode a key to a character. Upload **once**, after connect and before `listen`, matching `__main__.py`'s rule that a listening socket is a promise the device works. Do not upload again per string: a second `keymap` resets client state under held keys.
**[SUPERSEDED — change 2.]** Three values this bullet left unstated, each of which fails silently:
- the `size` argument **includes the trailing NUL**. Hyprland `mmap`s exactly `size` bytes and hands the pointer to `xkb_keymap_new_from_string`, which wants a C string, so `size = len(text)` reads off the end of the mapping. The symptom is a keymap that will not compile, which from our side is indistinguishable from a compositor with no keyboard at all;
- the fd must outlive a **round trip**, not a flush: the compositor has to have received and mapped it before we close our end;
- `format` is the integer **1** (`XKB_KEYMAP_FORMAT_TEXT_V1`). Hyprland ignores it and parses as `TEXT_V2`, which accepts a V1 string; do not send 2;
- `time` on every `key` is `backend._now_ms()`, the same monotonic source the pointer requests use. The protocol requires one clock per object.
Also, a *second* upload is worse than "resets client state": Hyprland replaces its XKB state without walking the keys it believes are down, so re-uploading under a held modifier strands it.
- **No unicode request.** A codepoint is typeable only when the uploaded layout has a keysym for it. No `Ctrl+Shift+u`, no per-string custom keymap, no compose sequences in v1.
- **Modifiers are a separate request.** The compositor does not derive the modifier mask from `key`. Shifted text needs `modifiers` as well as the letter key. Details in §4.
- **No per-output target.** Unlike `create_virtual_pointer_with_output` (`backend.py`), a virtual keyboard is seat-global. Keys hit whichever surface has keyboard focus. There is no output argument to add.
**Missing global is not fatal to the cursor.** `ManagerTooOld` aborts process start because a version-1 pointer aims at the wrong output (`backend.py`). A missing keyboard global must not do that. Pointer ops keep working. Keyboard ops return `{"ok": false, "error": ...}` and the broker surfaces that as exit **10 COMPOSITOR**. No silent no-op.
---
## 2. Surface
**Decision: new CLI group `session keyboard`, new broker verbs `keyboard.*`, new ops on the existing broker↔driver socket. No new process, unit, binary, or socket.**
`CONTRACT-sessions.md` §5 freezes the binary `hhstream-pointer` and the socket `/run/hhstream/pointer.sock`. Renaming the package to "input" is churn with no behavioural gain. `server.py` already frames newline-delimited JSON at `MAX_LINE_BYTES` (1 MiB) and is unchanged.
### CLI
`pointer acquire` remains the only acquire. Keyboard verbs are gestures on that lease. They reuse `_lease_of`, `--lease`, and `$HHSTREAM_LEASE` (`src/hhstream_session/lease.py`). There is no `keyboard acquire` and no second cache file.
```
hhstream session pointer acquire ID # unchanged; also authorises typing
hhstream session pointer release ID # lifts buttons and keys
hhstream session pointer state # JSON gains "keys"
hhstream session keyboard type ID TEXT # one argument, raw UTF-8, no backslash escapes
hhstream session keyboard key ID NAME # tap: down then up
hhstream session keyboard down ID NAME
hhstream session keyboard up ID NAME
```
`NAME` is an XKB keysym name (`a`, `A`, `Return`, `BackSpace`, `Left`, `Control_L`, `Shift_L`), not an evdev number. Help text on both groups must say keyboard commands refuse with exit 9 until `pointer acquire`. `pointer --help` stays about the cursor.
`type` does not interpret `\n`. A newline is a real newline in the argument. The shell quotes it. The wire value is that string.
### Broker control socket
Extend `src/hhstreamd/protocol.py` (do not renumber `ExitCode`). Same codes as pointer: **8 BUSY** only on a second acquire, **9 NO_LEASE** when this session does not hold the lease, **2 USAGE** for a bad keysym or an unmappable character, **10 COMPOSITOR** when the driver has no keyboard global or the connection dies.
**[SUPERSEDED — change 1.]** "Exit 2 for a bad keysym" and "`Rejected`, same as a bad button" are not the same code, and this paragraph asserted both. `SessionManager._drive` turns *every* driver `ok: false` into `ManagerError("driver_refused")`, which is exit **10** — that is why `test_an_unknown_button_is_refused_by_the_driver_not_invented` asserts `driver_refused` and not usage. Exit 2 happens only for an `invalid_argument` the broker raises *before* `_drive`. So:
- the **broker** checks shape before it calls the driver: `text` must be a string, `state` must be `down`/`up`, `key` must be a non-empty string, and `text` must be within the 4096-scalar cap. Those are `invalid_argument`, exit 2;
- the **driver** tags a rejection it caused the caller to earn with a stable token, `kind: "invalid_argument"` (`hhstream_pointer.ops.KIND_INVALID`) — a bad keysym, an unmappable character, a dead key, a `type` under a held non-modifier. `_drive` maps that token to `invalid_argument`, exit 2;
- everything else keeps `kind: "refused"` and stays `driver_refused`, exit **10**: no keyboard global, no aimed output, a dead connection.
The token exists because the alternative is the broker telling "no such keysym on your layout" from "this compositor has no keyboard" by matching on English.
| Verb | Request beyond `verb`, `request_id` | Response |
| --- | --- | --- |
| `keyboard.key` | `id`, `lease_id`, `key`, `state` (`down`/`up`) | `keys[]` |
| `keyboard.type` | `id`, `lease_id`, `text` | `keys[]` (explicit holds still down; a `type` itself leaves none) |
| `pointer.state` | — | existing fields plus `keys[]` |
| `pointer.release` | unchanged | report lifted keys in the same place buttons are already reported (audit if that is where `buttons_lifted` lives today — do not invent a richer release body than buttons have) |
`keyboard.key` and `keyboard.type` join the mutation set beside `pointer.click` (`protocol.py`): a replayed `request_id` returns the recorded answer and must not type twice. A new `request_id` types again.
Neither verb calls `touch`. Same rule as every `pointer.*` verb (`CONTRACT-sessions.md` §4, module docstring of `src/hhstream_session/__init__.py`, `src/hhstreamd/manager.py`).
Cap `text` at **4096** Unicode scalars in both broker and driver. Over the cap is `invalid_argument`, not a truncated type.
**Audit:** `keyboard_key` records the keysym name and state. `keyboard_type` records `chars` (an integer) and **not the text**. Typed strings will include passwords; the audit log is the wrong place for them.
**[SUPERSEDED — change 8.]** `events.jsonl` is not the only place a string leaks, and the plan secured only that one. Two more:
- **Refusal messages and driver logs carry the scalar *index*, never the character and never the string.** "unmappable at scalar 3" is the whole of what a reader needs, and the alternative ends up in the journal. `AuditLog.emit` will happily write a `text` field if a caller passes one, so the rule is about every message, not only the audit event.
- **`pointer acquire`'s stderr is wrong once this lands.** It says "expires in … with no further **pointer** verb". A `keyboard.*` verb goes through `_refresh` and *does* move `expires_at`, so the sentence has to name both — otherwise an operator releases during a `type` that would have kept the lease, and the TTL test looks flaky against the help text.
(What the shell and `ps` show of the CLI argument is outside this rule: the text is an argv word by the time anything here sees it.)
### Driver socket (`CONTRACT-sessions.md` §3.4)
The driver stays an executor with no lease (`ops.py` docstring, `driver.py` docstring). New ops:
```json
{"op":"key","key":"Return","state":"down"} -> {"ok":true}
{"op":"key","key":"a","state":"up"} -> {"ok":true}
{"op":"type","text":"Hello"} -> {"ok":true}
{"op":"pressed"} -> {"ok":true,"buttons":["left"],"keys":["Control_L"]}
{"op":"release_all"} -> {"ok":true,"lifted":["left"],"lifted_keys":["Control_L"]}
```
There is no keyboard `frame`. `zwp_virtual_keyboard_v1` has none. Do not reuse the pointer's end-of-gesture `frame`; a key flushed only on `frame` would never arrive. `backend.py` already flushes every pointer request; keyboard requests flush the same way.
`pressed` and `release_all` grow key fields. Do not add a parallel `release_all_keys` op. Expiry has one call site (`SessionManager._expire_pointer`).
**[SUPERSEDED — change 3.]** Two things follow from there being exactly one call site, and this plan stated neither:
- `release_all` must **no-op its key half when no virtual keyboard exists**. If the key half raised, the one `release_all` would abandon the buttons it had just been asked to lift — and a lease that expired with a mouse button down is the failure the whole TTL exists to prevent.
- `release_all` ends with `modifiers(0, 0, 0, 0)` **whenever a mask was set**, because a virtual key-up clears no modifier state on this compositor. Without it a lease that ended under Shift hands the next holder a keyboard where every letter is a capital. One `modifiers(0, 0, 0, 0)` also goes out immediately after the initial keymap upload, so a previous driver's mask is not inherited.
Unknown keysym, non-finite or over-long text, and `type` while a non-modifier key is held are `Rejected` (answered, connection kept), same as a bad button.
### File-by-file
| File | Change |
| --- | --- |
| `src/hhstream_pointer/_protocol/virtual_keyboard_unstable_v1/` | Vendor pywayland output, same method as `_protocol/regenerate.py` (seed the import map with core `wl_*`, do not use `python -m pywayland.scanner` alone). XML is wayland-protocols `unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml`, not the wlr pointer XML. |
| `src/hhstream_pointer/_protocol/regenerate.py` | Second XML argument. State the path in the module docstring the way the pointer generator does. |
| `src/hhstream_pointer/keymap.py` | **New.** xkbcommon only here. Builds one keymap from rules (`evdev`, model, layout, variant, options). Resolves a keysym name or a string to a list of actions `(evdev_code, mods_depressed)` or raises. Resolves the **whole** string before returning, so a bad character emits nothing. |
| `src/hhstream_pointer/ops.py` | Extend `Dispatcher`. A small `Keymap` protocol (same idea as `PointerSink`) does the resolving, so this file still imports neither pywayland nor xkbcommon. New ops `key` and `type`. `_pressed` stays buttons. New insertion-ordered `_keys`. `release_all` lifts keys after buttons: key-up each held key in press order, then `modifiers(0, 0, 0, group)`. |
| `src/hhstream_pointer/backend.py` | Still the only pywayland importer. Bind the manager at version 1, `create_virtual_keyboard(seat)` **once** (not per output). `keymap` upload from a memfd. Sink methods `key(code, state)` and `modifiers(dep, latch, lock, group)` only — no names, no UTF-8. |
| `src/hhstream_pointer/__main__.py` | Flags below. Build the keymap and upload it before `listen`. `_lift_everything` already calls `release_all`; once that op lifts keys, shutdown covers them. Update the comment that today only mentions buttons. |
| `src/hhstream_pointer/server.py` | No change. |
| `src/hhstreamd/driver.py` | `key()`, `type_text()` on `PointerDriver`. `release_all` / `pressed` read the new fields. Same socket, same "do not retry a request that was written" rule. A failed `release_all` is still `DriverUnavailable`: a dropped lift leaves a key down. |
| `src/hhstreamd/pointer.py` | Not a new class. On `_Held`, a `keys` dict beside `buttons`. `press_key` / `unpress_key` / `keys_pressed`, mirroring `press` / `unpress` / `pressed`. `expire()` still returns `LeaseInfo` only — no keys. Callers read `keys_pressed()` first, as they must read `pressed()` first (`CONTRACT-sessions.md` §3.3). |
| `src/hhstreamd/manager.py` | `keyboard_key` and `keyboard_type` go through `_refresh` (live session, holder, not expired, container output required). Driver first, then the lease mirror — the `pointer_button` order, so a refused key is not recorded as down. `_expire_pointer` reads keys before `expire()` and uses the one `release_all`. Same on `pointer_release`, kill, reap, and driver loss. |
| `src/hhstreamd/protocol.py` | Verb allowlist plus the mutation set. |
| `src/hhstreamd/policy.py` | **[SUPERSEDED — change 5.] No change.** A `[keyboard]` table here would be read by nothing: `settings_from_dict` is read by `hhstreamd`, and the driver is a *different unit* that never opens that file. An operator could change the layout and every injected key would stay on `us`. The layout travels as `--xkb-layout` on the driver's argv, and nowhere else. |
| `src/hhstream_session/__init__.py` | `_build_keyboard` beside `_build_pointer`. |
| `src/hhstream_session/format.py` | `render_pointer_state` shows held keys. |
| `src/hhstream_session/lease.py` | No new store. `pointer-leases.json` remains the cache. |
| `nix/module.nix` | No new unit. Extend `hhstream-pointer.service` argv — **and nothing else**: **[SUPERSEDED — change 5]** the `[keyboard]` TOML emission is dropped, see `policy.py` above. Options under `sessions.keyboard`: `layout` (`lib.types.strMatching "[A-Za-z0-9_-]+"`, default `"us"` — the type is what refuses a comma-separated list), `variant` (default `""`), `options` (default `""`), `model` (default `"pc105"`). The driver also refuses a comma itself, before `xkb_keymap_new_from_names`, because the NixOS type cannot reach a binary run by hand. §3.4's "command line the module will pass, exactly" and the unit comment quoting it are amended in the same change. |
| `nix/package.nix`, `flake.nix` devshell, `pyproject.toml` | `xkbcommon` for the driver and for unit tests. No new console script. |
| `nix/hyprland-conf.nix` | Keep `input:follow_mouse = 1`. Extend the comment: keyboard events have no output, so the warp is what moves keyboard focus — but see §6 risk 1 and Stage 0, because **one** warp does not — and the lease is what stops another session stealing it mid-string. **[SUPERSEDED — change 4]** Also set `input:virtualkeyboard:release_pressed_on_close = true`, with the note that it releases keys and still does not clear the modifier mask. |
| `docs/CONTRACT-sessions.md` | Amend §3.3 (key mirror, `expire()` still carries no keys), §3.4 (ops above), §4 (verbs, idle rule). |
| `README.dev.md` item 3, `checks/vm-pointer.nix` header | Stale relative to the conf file as read: they still say `hyprland-conf.nix` ships `follow_mouse = 0` and that the keyboard task will choose. The conf file sets `1`. Fix the prose when implementing; do not set the conf back to `0`. |
Flags the unit passes, defaults matching §5's style of frozen defaults:
```
hhstream-pointer --socket /run/hhstream/pointer.sock \
--compositor-runtime-dir /run/hypr-stream \
--xkb-layout us --xkb-model pc105 \
--xkb-variant '' --xkb-options ''
```
A layout xkbcommon rejects, before any Wayland connect, exits **2**. Compositor failures stay **10**.
---
## 3. Lease
**Decision: one lease. Keyboard verbs require the pointer lease. Do not add a keyboard lease, a `keyboard` boolean, or a second sweeper.**
§6.7 says the lease "gains a `keyboard` flag" and that no new mechanism is needed. The flag is the wrong reading of the code that shipped. `_Held.buttons` exists because `release_all` must know **which** buttons went down and in what order (`src/hhstreamd/pointer.py`, `ops.py`). A boolean cannot lift `Control_L` and not `a`. The key set is that pattern, not a new state machine.
Why sharing is required by the seat, not just by the plan: there is one `wl_seat` and one keyboard focus. `follow_mouse = 1` (`nix/hyprland-conf.nix`) means another session's warp retargets the keys. Two leases would let session B move the pointer while session A believes it is typing into its own window. `LeaseBusy` on acquire is already the mutex (`PointerLease.acquire` refuses even the current holder a second id, so a drag — or a half-typed chord — cannot have its id silently replaced).
Rules that keyboard inherits unchanged:
- Acquire / refresh / hard cap (`ttl` 30 s, `max` 300 s, refresh moves only the idle deadline).
- `expire()` does not lift anything. The broker reads the mirror, drops the lease, then `release_all` on the driver (`_expire_pointer`).
- Driver death force-releases (`driver_gone`). The driver is not asked to remember who held the lease.
- Kill and reap of the holder drop the lease with the existing reasons (`kill` / `crash`).
- CLI cache is re-checked by the broker; a stale id is exit 9 and then forgotten.
- Gesture verbs do not mark activity.
**TTL expiry with keys held down.** Same as buttons (`PLAN` §6.2 "Never leave a button down", implemented in `_expire_pointer` and in `_lift_everything`):
1. Sweeper (or the next verb's `_refresh`) sees the deadline.
2. Broker reads `keys_pressed()` and `pressed()` **before** `expire()`.
3. `expire()` drops the lease. It is not kept open to finish the word.
4. Driver `release_all`: button-ups, then key-ups in press order, then `modifiers(0, 0, 0, group)`, then the pointer `frame`s it already sends for buttons. No keyboard frame.
5. Audit `pointer_lease_timeout` gains `keys_lifted`. The session's pointer bookmark is stamped `released_by: timeout` as today.
6. The next `acquire` is free. Nothing is replayed. A held Shift does not leak into the next holder.
If `release_all` fails, do what the button path does: log it, still drop the lease, do not retry a request whose outcome is unknown (`driver.py`). The stuck-key case is a risk in §6, not a second expiry protocol.
`type` refreshes **once per broker verb**, inside `_refresh`, not once per character. Explicit `down` of a modifier stays in the lease mirror across later `type` calls. `type` itself must not leave letters down.
A warp while a key is held stays legal, same as a drag. Focus can follow that warp. That is another reason both devices share the lease: the holder is the only session allowed to move focus until release or TTL.
---
## 4. Keymap and text
**Decision: one seat-wide XKB layout, chosen in config, default `us` / `pc105` / empty variant / empty options. The virtual keyboard's keymap is the one we upload. Hyprland's `input:kb_layout` is not read and does not affect injected keys.**
Do not query `hyprctl getoption` to fill a default. The broker's rule is that unverified fields stay null (`PLAN` §7); a layout guessed from the compositor would drift on reload. Operators who are not on `us` set `sessions.keyboard.layout`.
**Decision: v1 is a single group.** A comma-separated layout (`us,ru`) is rejected at config load. Group in every `modifiers` request is **0**. No group-switch keysym.
**Where it lives.** `[keyboard]` in the broker TOML (`policy.py`), rendered by `nix/module.nix`, passed as the flags in §2. The driver compiles it at startup and uploads that byte string. The session record does not gain a layout field. Per-session layouts would mean re-uploading `keymap` on every acquire, which drops held state; not in v1.
**How text is expressed.**
- `keyboard type` takes a Python `str` of Unicode scalars. The keymap maps each scalar to one keysym, then to one `(evdev code, modifier mask)`.
- Newline → keysym `Return`. Tab → `Tab`. Other C0 controls are rejected with the rest of the string, not skipped.
- `A` is Shift+`a` via the **mask**, not via a typed Shift key, unless the user also has Shift explicitly held.
- Explicit modifier holds are a baseline. `type` ORs each character's mask with that baseline, sends the letter, and restores the mask to the baseline afterwards. So `down Control_L` then `type c` is Ctrl+C, and Control is still down when `type` returns.
- If any **non-modifier** key is already held, `type` is rejected and emits nothing.
- Validation is complete before the first sink call. Unmappable character → the whole request fails, no prefix typed. This includes `é` on `us`.
- Dead keys (`dead_acute` and the like) are a rejection even if the layout names them. v1 does not emit a two-event compose. `us(intl)` is therefore a poor choice until a later stage; default `us` does not need it.
- No auto-repeat timer in our process. **[SUPERSEDED — change 7, and Stage 0 has measured it.]** Hyprland does not repeat on the wire: a key held 1.5 s produced exactly one `wl_keyboard.key`, and it publishes `repeat_info rate=25 delay=600` so the *client* repeats. That is not good news — a focused text field will insert characters from that one press until the key-up or the TTL — so the `down` help text warns **unconditionally** rather than only if a probe saw repeats, and v1 has no rate to offer because we do not generate them.
**Modifier decision:** for a shifted character, send `modifiers(shift_mask, 0, 0, 0)` and the letter's `key` down/up, then restore the previous mask. Do **not** also synthesize `Shift_L` down/up for that character. For an explicit `keyboard down Shift_L`, send the Shift **key** and a matching `modifiers` update, and remember both in the held-key record so `release_all` undoes both. Sending both a synthetic Shift key and a mask for an ordinary capital is how toolkits apply Shift twice or not at all; the VM check in §5 is what proves the letter path against a real `wl_keyboard` client.
Named non-character keysyms used by `key`/`down`/`up` go through the same resolver, not a second table that can disagree with the layout. `buttons.py` can stay a hand-written evdev table because the protocol takes those codes directly; keysyms cannot.
**[SUPERSEDED — change 6.]** This section never defined "modifier", and a Shift-only reading of it is wrong: `@` on `de` is a **level-three** symbol, so a resolver that only knew about Shift would type the wrong character. The definition is *measured from the keymap*, not listed — press each keycode in a throwaway XKB state and read the serialised masks back:
- a non-zero **depressed** mask and no locked mask makes it a **modifier**: on `us`/`pc105` that is exactly `Shift_L/R`, `Control_L/R`, `Alt_L/R`, `Super_L/R`, `ISO_Level3_Shift`, `ISO_Level5_Shift`. (`Meta_L/R` and `Hyper_L/R` are canonical modifier keysyms that **no keycode on this layout carries** — which is the argument for measuring rather than listing.)
- a non-zero **locked** mask makes it a **lock**: `Caps_Lock`, `Num_Lock`. A lock is not a modifier, and `type` refuses while one is held rather than producing a case nobody asked for.
The resolver returns `(evdev code, mask)` for a keysym at *any* level, so one code path serves `us` and `de`. And `key`/`type` never call `_aimed()`: a keyboard has no output, so demanding a prior warp would refuse a caller who only ever wanted to type.
---
## 5. Tests
Match the split already in the tree: the dispatcher is tested against a recording sink and is not tested for authority (`tests/pointer/test_ops.py`); the lease is pure with `FakeClock` and no sleep (`tests/pointer/conftest.py`); the broker is tested with a real UNIX socket and `FakeDriver`, Wayland substituted (`tests/sessions/unit/test_pointer_verbs.py`). No integration test grows a fake compositor. `tests/sessions/integration/` does not gain Wayland.
### Unit
| Test | Asserts |
| --- | --- |
| `tests/pointer/test_keymap.py` | Real xkbcommon, layout `us`: `a` → evdev 30, mask 0; `A` → 30 plus Shift; `Return` and `\n` → `KEY_ENTER`; empty string → `[]`; `é` and a C0 control other than `\n`/`\t` raise; the raise happens with zero sink calls when driven through the dispatcher. Dead keysym raises. |
| `tests/pointer/test_ops.py` | Extend `RecordingSink` with `key(code, state)` and `modifiers(...)`. `key down` then `pressed` lists the name. `up` of a key that was not down reports it not lifted (same as `unpress`). `release_all` lifts keys in press order and ends with a zeroed `modifiers`, even if the last `warp` was to another output (keys are not per output). `type` emits down/up pairs and leaves `keys` empty. `type` with `Control_L` held ORs the mask and leaves Control down. `type` with `a` held emits nothing and returns `ok: false`. A keymap that raises on the third character emits nothing. |
| `tests/pointer/test_lease.py` | `press_key` / `keys_pressed` order. `expire()` still returns no key list; the test reads `keys_pressed()` first, then `expire()`, then sees an empty lease. Existing busy/stale/`lease_id` cases stay as they are. |
| `tests/pointer/test_vendored_protocol.py` | Interface name `zwp_virtual_keyboard_manager_v1` and requests `keymap`, `key`, `modifiers`. No display. |
| `tests/sessions/unit/test_keyboard_verbs.py` | New, harness copied from `test_pointer_verbs.py`. No lease → 9. Other session holds the lease → 9 (not 8). Second `pointer.acquire` while typing → 8. `type` refreshes TTL: advance the `ManualClock` by less than `ttl` after a type and the lease survives; advance past it and `release_all` runs with the held key. Driver rejection does not stick the key in the lease. `type` does not move `last_activity_boottime`. `pointer.release` and kill both lift keys. `pointer.state` includes `keys`. |
| ~~`tests/sessions/unit/test_settings.py`~~ | **[SUPERSEDED — change 5.] Dropped.** A TOML default of `us` that nothing uploads is precisely the false pass that file's own docstring warns about. Replaced by `tests/pointer/test_driver_flags.py`: an omitted `--xkb-layout` is `us`, `de` is `de`, and `us,ru` exits 2 **with an injected sink proving the compositor was never touched**. |
| CLI surface test (`tests/sessions/unit/test_cli_surface.py`) and broker verb list (`test_broker_protocol.py`) | `keyboard` in help; help says the pointer lease is required; `keyboard.type` / `keyboard.key` are known verbs and are in the mutation set. |
`nix develop -c python -m pytest tests/pointer tests/sessions/unit -q` is the gate. Not run for this plan.
### Integration / VM
**Decision: a new `checks/vm-keyboard.nix`, same shape as `checks/vm-pointer.nix`. Do not call the feature done on unit tests alone.** `docs/TODO/scroll-has-no-compositor-proof.md` is the precedent: scroll shipped on the driver socket with no compositor proof. Keyboard's only purpose is that the client receives symbols.
- Victim: a small Wayland client beside `checks/lib/pointer_target.py` (or an extension of it) that sets `xdg_toplevel` app id from `$HHSTREAM_APP_ID` and prints `wl_keyboard.enter`, each `key` (evdev code, state), each `modifiers` mask, and the keysym it decodes from the keymap the compositor forwarded. Read-back is `hhstream session logs`, as in `vm-pointer`.
- Assertions, real time, no injected clock (the unit clock already covers the arithmetic; `vm-pointer`'s expiry subtest states why):
1. After `pointer acquire` on session A, A's log shows `wl_keyboard.enter`. This is the focus measurement, and it is asserted against acquire itself: acquire is two warps (§6 risk 1), so this is a live assertion rather than one expected red. `checks/vm-keyboard.nix` also requires the reply's `focus == "warp"`, so the nudge-failure fallback is not accepted as success, and it asserts `wl_pointer.enter` separately — a check that saw only the cursor arrive would pass while the first keystroke went elsewhere. Do not weaken it.
2. `keyboard type A hello` decodes to the characters `hello` on A and to nothing on session B.
3. `keyboard down A Shift_L`, wait out a short `--ttl`, A's log shows the key-up and a cleared mask, and `pointer state` shows no holder.
4. A second acquire during the hold exits 8.
- Mutation, per `PLAN` §14: the character assertion must fail if `keymap` was never uploaded (keycodes without a decoded `hello` are a failure, not a pass).
- `globalTimeout` on the order of `vm-pointer` (1800). This check is slow; that is not a reason to fold it into a unit test.
---
## 6. Risks, open questions, stages
### Risks
- **Focus does not follow a single virtual-pointer warp. [MEASURED — Stage 0, and this is the headline risk of the feature.]** One `motion_absolute` onto an unfocused window, after the cursor had been still for 0.8 s, delivered `wl_pointer.enter` and **no** `wl_keyboard.enter`; the focus change arrived only after a *later* motion that changed the floored pixel. The mechanism is `InputManager::onMouseWarp` → `mouseMoveUnified(refocus=false)`: the delta is set to 0 when the previous movement was over 0.5 s ago, `mouseMoveUnified` returns early when the floored pixel did not change at all, and keyboard focus is that delta compared against `input:follow_mouse_threshold` (measured 0.0, unset). Transcript in `tests/README-keyboard-probe.md`.
**[RESOLVED in Stage 2 — `pointer acquire` is now two warps.]** This paragraph used to end "acquire is one warp, so the first key of a lease can land in whatever was already focused; the fallback is not built in Stages 1–2". Both halves are out of date, and `docs/CONTRACT-sessions.md` v3 §4 is the text that describes what shipped:
- `pointer_acquire` calls `_warp_taking_focus`, which warps to a neighbouring pixel and *then* to the target. The second motion carries a non-zero `m_mousePosDelta`, so it moves keyboard focus; the cursor finishes on the target, so the bookmark is unchanged. `FOCUS_NUDGE` is 0.004, about 2.88 px at the frozen 1280×720 mode, which is what makes the *floored* pixel differ.
- This depends on `input:follow_mouse_threshold` being 0, so `nix/hyprland-conf.nix` now sets it **explicitly** rather than inheriting the default. Raise it and acquire stops taking the keyboard.
- `dispatch focuswindow` **is** built, and it is not what this risk needed: it runs only when the *nudge* warp could not be sent, and it selects on `address:` from the read-back (falling back to `pid:`), never on `stable_id`, which is not a Hyprland selector. The target warp is still attempted afterwards and its failure still drops the lease. The acquire reply and the audit event carry `focus`: `warp`, `focuswindow`, or `none`.
- Still *not* the fix, and worth keeping written down: warping twice to the same bookmark (the same floored pixel returns early, before the delta is touched) and waiting longer after one warp (the measured settle window was 0.4 s and contained the pointer enter alone).
Measured end to end by `checks/vm-keyboard.nix` on both layouts: with session B holding keyboard focus, acquire on A delivers `wl_pointer.enter` **and** `wl_keyboard.enter` to A and `wl_keyboard.leave` to B. That subtest requires `focus == "warp"`, so it does not accept the fallback as success.
What is **not** covered by a test: the `focuswindow` branch itself. A driver that is down fails the target warp too and the acquire aborts, so the branch is the narrow one where the nudge fails and the target warp succeeds. If both mechanisms fail, acquire still returns `ok` with `focus="none"` — a lease whose first key may go to the previously focused window.
- **Destroy of the virtual keyboard leaves a key down inside Hyprland. [MEASURED — Stage 0.]** `destroy` with a key held, with a second virtual keyboard keeping the seat's capability up, delivered no key-up; `input:virtualkeyboard:release_pressed_on_close` is `0` and unset. Worse than "stuck": `InputManager::onKeyboardKey` drops a *press* whose code is already in the seat's `m_pressed`, so the next driver's press of that code is ignored and the key is **dead** until something sends a matching up. Our shutdown path sends key-ups first (`_lift_everything`); a `-9` skips it, which is why **[change 4]** `nix/hyprland-conf.nix` now sets `release_pressed_on_close = true`. That setting emits key-ups only and still never clears the modifier mask, so `release_all` keeps sending `modifiers(0, 0, 0, 0)` regardless. Still no retry loop the button path does not have.
- **Auto-repeat is the client's, which does not make it harmless. [MEASURED — Stage 0.]** `press_count=1` over a 1.5 s hold, with `repeat_info rate=25 delay=600` published at bind time. So the flood is real and it is foot/GTK/Qt producing it from our one press, for as long as the key is down — up to the TTL (default 30 s). **[SUPERSEDED — change 7]** the help text for `down` therefore warns unconditionally; the old "if it does, the help text says so" is the false pass the review named. v1 has no repeat-rate control, and `zwp_virtual_keyboard_v1` has no `repeated` state to ask the compositor about. Chords (`Control_L` held across a `type`) remain the justified use of `down`.
- **Mask versus Shift keycode.** §4's letter path can still be wrong for a toolkit that ignores `modifiers` and only shifts when it sees `KEY_LEFTSHIFT`. The VM victim must print both the mask and the decoded sym. Chromium is not the first victim (`vm-pointer` avoided it; it is expensive and the app-id rule in `README.dev.md` item 4 only binds clients that honour `$HHSTREAM_APP_ID`).
- **The lease is not a security boundary.** `PLAN` §6.6: any client may bind the pointer global. The keyboard global is the same. Stage 4 sandboxing is unchanged and out of this work.
- **Concurrent typing is impossible** on one compositor, for the same reason concurrent pointers are (`PLAN` §1). A second seat is Stage 5 and still CPU-gated. This work does not pretend otherwise.
- **`xkbcommon` in the devshell** is a new dependency. Tests that import `keymap.py` cannot skip when it is absent; an optional dependency would let the unmappable-character path rot.
### Open questions
Questions 1–4 are **closed by Stage 0** (`tests/README-keyboard-probe.md` and its transcript). Question 5 is answered by the operator.
1. ~~Stage 0 global name and version on the actual `/run/hypr-stream` socket.~~ **`zwp_virtual_keyboard_manager_v1` at version 1.** No `wp_virtual_keyboard_v1`. The gate holds.
2. ~~Does acquire's warp produce `wl_keyboard.enter`?~~ **Not from a single idle warp — so acquire stopped being a single warp.** See risk 1 above. Measured again end to end by `vm-keyboard` on both layouts, where the two-warp acquire does deliver `enter` to the aimed window and `leave` to the one that held focus.
3. ~~On destroy, does Hyprland release keys that were down?~~ **No.** See risk 2.
4. ~~Does Hyprland auto-repeat those keys?~~ **Not on the wire; the client does.** See the auto-repeat risk.
5. **Answered: `us`, and configurable.** The operator's decision is a NixOS option in the `hhstream` namespace, string, default `"us"`, documented in `README.dev.md` — reaching the keymap as `--xkb-layout` on the driver unit, per change 5.
6. A later stage, only if agents must type arbitrary UTF-8: a throwaway keymap that maps spare keycodes to `0x01000000 + codepoint` for the duration of one `type`, then restores the layout keymap. Not v1. It reintroduces the "upload resets held modifiers" problem, so it has to refuse to run while any key is held.
### Stages
| Stage | Work | Rough size | Gate |
| --- | --- | --- | --- |
| 0 | **Done.** `tests/keyboard_probe.py`, its README and its transcript. Two windows, not one; read-only `hyprctl` for the boxes; `(a1)` and `(a2)` measured separately; no `hyprctl dispatch`. | S | **Passed.** Named global is `zwp_virtual_keyboard_manager_v1` v1. |
| 1 | Driver: vendor, `keymap.py`, ops, backend upload, `__main__` flags, pointer unit tests. Pointer still runs if the global is missing. | M, about 3–5 days | `pytest tests/pointer` green. `type` of an unmappable character emits no events. |
| 2 | Broker mirror, verbs, CLI, module argv, session unit tests, contract doc amendments, stale `follow_mouse` comments. **No `policy.py` and no TOML table** (change 5). | M, about 2–4 days | `pytest tests/pointer tests/sessions/unit` green. Expiry test shows `release_all` lifting a held key. No new unit under `nix/module.nix`, and nothing there beyond argv. |
| 3 | **Done.** `checks/vm-keyboard.nix` and `checks/lib/keyboard_target.py`, exposed twice (`vm-keyboard`, `vm-keyboard-de`) because the layout reaches the keymap only through the driver unit's argv. Its enter subtest **is** written against acquire, because acquire is two warps now — see the risk paragraph above and the file's own header. | M | **Passed**, both layouts: characters on A, silence on B, key-up after a real TTL, second acquire is 8, red if the keymap was not uploaded. |
| ~~4~~ | **Does not exist as pending work.** It was "build `focuswindow` before the first key of a lease, if Stage 0 shows no enter". Stage 0 did show no enter after a *single* warp, and the answer was the second warp in `_warp_taking_focus`, not a focus dispatch — so the Stage 3 enter assertion is green without loosening it. `focuswindow` shipped in Stage 2 as the nudge-failure fallback *inside* acquire, which is a different mechanism for a different failure. Nothing here is outstanding. | — | — |
Rollback of Stages 1–2 is reverting the commit: old brokers never send `key`/`type`, and an old driver rejects unknown ops with `ok: false` (`ops.py` already answers unknown `op` instead of dropping the connection). No registry schema bump. Bookmarks stay as they are.