docs(plan): keyboard input for sessions — approach and open questions
This commit is contained in:
1 parent
4b5cb5197a
commit
14b3f691c4
1 file changed
+259
@@ -0,0 +1,259 @@
|
||||
# Keyboard input for sessions
|
||||
|
||||
Typing stops being a non-goal. `docs/PLAN-workspaces-and-pointer.md` §1 defers it, and §6.7 already specifies the shape: same seat, same serialisation, `zwp_virtual_keyboard_manager_v1`, and the existing pointer lease rather than a new lock. Stages 0–3 of that plan are what the tree implements today (broker lease in `src/hhstreamd/pointer.py`, driver in `src/hhstream_pointer/`, CLI in `src/hhstream_session/__init__.py`). This plan is §6.7, with one correction noted under lease semantics.
|
||||
|
||||
The Chrome path in `examples/agent-eyes-demo.py` (`Input.insertText`) is not this feature. It never touches the seat. This plan is for whatever window is focused on the compositor's one seat.
|
||||
|
||||
---
|
||||
|
||||
## 1. Protocol
|
||||
|
||||
**Decision: `zwp_virtual_keyboard_v1`, spoken by the existing `hhstream-pointer` process. Not `wp_virtual_keyboard_v1`, not uinput/`ydotool`, not `wtype`.**
|
||||
|
||||
| Option | Verdict |
|
||||
| --- | --- |
|
||||
| `zwp_virtual_keyboard_v1` | Use it. Unstable wayland-protocols interface (`virtual-keyboard-unstable-v1.xml`). Manager global `zwp_virtual_keyboard_manager_v1`. Requests are `keymap(format, fd, size)`, `key(time, keycode, state)`, `modifiers(depressed, latched, locked, group)`. Version 1 only. |
|
||||
| `wp_virtual_keyboard_v1` | Do not bind it. Nothing in this repo claims that global. A stable `wp_` rename is not what §6.7 measured. |
|
||||
| uinput / `ydotool` | Reject. Needs `/dev/uinput` and a seat that actually reads that device. The headless compositor is driven as a Wayland client today (`WaylandSink` in `src/hhstream_pointer/backend.py`); the driver runs as `hermes` with group `hhstream` so it can open the Wayland socket (`nix/module.nix`). A kernel device is a second privilege and a second focus story, and it cannot be unit-tested as a recording sink. |
|
||||
| `wtype` | Reject as the mechanism. It is a short-lived client of the same `zwp_` protocol: new virtual keyboard per invocation, its own keymap, no `release_all` across calls, no lease. Useful only as a reference for how text becomes keycodes. |
|
||||
|
||||
**What the running compositor exposes.** Not re-checked in this session (no registry dump). The only evidence in-tree is §6.7's **[RAN]** claim, against the Hyprland it measured (plan header: tag `v0.56.2`): `zwp_virtual_keyboard_manager_v1` is a live global. `wp_virtual_keyboard_v1` is not mentioned. Treat that as unverified until Stage 0 dumps the globals on `/run/hypr-stream`. If the dump shows a different name, stop. Do not probe uinput as a fallback.
|
||||
|
||||
**Tradeoffs, as they apply here:**
|
||||
|
||||
- **Keycode, not keysym, on the wire to the compositor.** `key` takes a Linux evdev code (the same value `wl_keyboard.key` carries; clients add 8 to reach an XKB keycode). Keysym names stay on our socket, the way button names stay on ours and evdev codes cross into `zwlr_virtual_pointer_v1` (`src/hhstream_pointer/buttons.py`).
|
||||
- **Keymap upload is mandatory.** Until `keymap` is sent (format `xkb_v1`, memfd of an XKB text keymap), clients cannot decode a key to a character. Upload **once**, after connect and before `listen`, matching `__main__.py`'s rule that a listening socket is a promise the device works. Do not upload again per string: a second `keymap` resets client state under held keys.
|
||||
- **No unicode request.** A codepoint is typeable only when the uploaded layout has a keysym for it. No `Ctrl+Shift+u`, no per-string custom keymap, no compose sequences in v1.
|
||||
- **Modifiers are a separate request.** The compositor does not derive the modifier mask from `key`. Shifted text needs `modifiers` as well as the letter key. Details in §4.
|
||||
- **No per-output target.** Unlike `create_virtual_pointer_with_output` (`backend.py`), a virtual keyboard is seat-global. Keys hit whichever surface has keyboard focus. There is no output argument to add.
|
||||
|
||||
**Missing global is not fatal to the cursor.** `ManagerTooOld` aborts process start because a version-1 pointer aims at the wrong output (`backend.py`). A missing keyboard global must not do that. Pointer ops keep working. Keyboard ops return `{"ok": false, "error": ...}` and the broker surfaces that as exit **10 COMPOSITOR**. No silent no-op.
|
||||
|
||||
---
|
||||
|
||||
## 2. Surface
|
||||
|
||||
**Decision: new CLI group `session keyboard`, new broker verbs `keyboard.*`, new ops on the existing broker↔driver socket. No new process, unit, binary, or socket.**
|
||||
|
||||
`CONTRACT-sessions.md` §5 freezes the binary `hhstream-pointer` and the socket `/run/hhstream/pointer.sock`. Renaming the package to "input" is churn with no behavioural gain. `server.py` already frames newline-delimited JSON at `MAX_LINE_BYTES` (1 MiB) and is unchanged.
|
||||
|
||||
### CLI
|
||||
|
||||
`pointer acquire` remains the only acquire. Keyboard verbs are gestures on that lease. They reuse `_lease_of`, `--lease`, and `$HHSTREAM_LEASE` (`src/hhstream_session/lease.py`). There is no `keyboard acquire` and no second cache file.
|
||||
|
||||
```
|
||||
hhstream session pointer acquire ID # unchanged; also authorises typing
|
||||
hhstream session pointer release ID # lifts buttons and keys
|
||||
hhstream session pointer state # JSON gains "keys"
|
||||
|
||||
hhstream session keyboard type ID TEXT # one argument, raw UTF-8, no backslash escapes
|
||||
hhstream session keyboard key ID NAME # tap: down then up
|
||||
hhstream session keyboard down ID NAME
|
||||
hhstream session keyboard up ID NAME
|
||||
```
|
||||
|
||||
`NAME` is an XKB keysym name (`a`, `A`, `Return`, `BackSpace`, `Left`, `Control_L`, `Shift_L`), not an evdev number. Help text on both groups must say keyboard commands refuse with exit 9 until `pointer acquire`. `pointer --help` stays about the cursor.
|
||||
|
||||
`type` does not interpret `\n`. A newline is a real newline in the argument. The shell quotes it. The wire value is that string.
|
||||
|
||||
### Broker control socket
|
||||
|
||||
Extend `src/hhstreamd/protocol.py` (do not renumber `ExitCode`). Same codes as pointer: **8 BUSY** only on a second acquire, **9 NO_LEASE** when this session does not hold the lease, **2 USAGE** for a bad keysym or an unmappable character, **10 COMPOSITOR** when the driver has no keyboard global or the connection dies.
|
||||
|
||||
| Verb | Request beyond `verb`, `request_id` | Response |
|
||||
| --- | --- | --- |
|
||||
| `keyboard.key` | `id`, `lease_id`, `key`, `state` (`down`/`up`) | `keys[]` |
|
||||
| `keyboard.type` | `id`, `lease_id`, `text` | `keys[]` (explicit holds still down; a `type` itself leaves none) |
|
||||
| `pointer.state` | — | existing fields plus `keys[]` |
|
||||
| `pointer.release` | unchanged | report lifted keys in the same place buttons are already reported (audit if that is where `buttons_lifted` lives today — do not invent a richer release body than buttons have) |
|
||||
|
||||
`keyboard.key` and `keyboard.type` join the mutation set beside `pointer.click` (`protocol.py`): a replayed `request_id` returns the recorded answer and must not type twice. A new `request_id` types again.
|
||||
|
||||
Neither verb calls `touch`. Same rule as every `pointer.*` verb (`CONTRACT-sessions.md` §4, module docstring of `src/hhstream_session/__init__.py`, `src/hhstreamd/manager.py`).
|
||||
|
||||
Cap `text` at **4096** Unicode scalars in both broker and driver. Over the cap is `invalid_argument`, not a truncated type.
|
||||
|
||||
**Audit:** `keyboard_key` records the keysym name and state. `keyboard_type` records `chars` (an integer) and **not the text**. Typed strings will include passwords; the audit log is the wrong place for them.
|
||||
|
||||
### Driver socket (`CONTRACT-sessions.md` §3.4)
|
||||
|
||||
The driver stays an executor with no lease (`ops.py` docstring, `driver.py` docstring). New ops:
|
||||
|
||||
```json
|
||||
{"op":"key","key":"Return","state":"down"} -> {"ok":true}
|
||||
{"op":"key","key":"a","state":"up"} -> {"ok":true}
|
||||
{"op":"type","text":"Hello"} -> {"ok":true}
|
||||
{"op":"pressed"} -> {"ok":true,"buttons":["left"],"keys":["Control_L"]}
|
||||
{"op":"release_all"} -> {"ok":true,"lifted":["left"],"lifted_keys":["Control_L"]}
|
||||
```
|
||||
|
||||
There is no keyboard `frame`. `zwp_virtual_keyboard_v1` has none. Do not reuse the pointer's end-of-gesture `frame`; a key flushed only on `frame` would never arrive. `backend.py` already flushes every pointer request; keyboard requests flush the same way.
|
||||
|
||||
`pressed` and `release_all` grow key fields. Do not add a parallel `release_all_keys` op. Expiry has one call site (`SessionManager._expire_pointer`).
|
||||
|
||||
Unknown keysym, non-finite or over-long text, and `type` while a non-modifier key is held are `Rejected` (answered, connection kept), same as a bad button.
|
||||
|
||||
### File-by-file
|
||||
|
||||
| File | Change |
|
||||
| --- | --- |
|
||||
| `src/hhstream_pointer/_protocol/virtual_keyboard_unstable_v1/` | Vendor pywayland output, same method as `_protocol/regenerate.py` (seed the import map with core `wl_*`, do not use `python -m pywayland.scanner` alone). XML is wayland-protocols `unstable/virtual-keyboard/virtual-keyboard-unstable-v1.xml`, not the wlr pointer XML. |
|
||||
| `src/hhstream_pointer/_protocol/regenerate.py` | Second XML argument. State the path in the module docstring the way the pointer generator does. |
|
||||
| `src/hhstream_pointer/keymap.py` | **New.** xkbcommon only here. Builds one keymap from rules (`evdev`, model, layout, variant, options). Resolves a keysym name or a string to a list of actions `(evdev_code, mods_depressed)` or raises. Resolves the **whole** string before returning, so a bad character emits nothing. |
|
||||
| `src/hhstream_pointer/ops.py` | Extend `Dispatcher`. A small `Keymap` protocol (same idea as `PointerSink`) does the resolving, so this file still imports neither pywayland nor xkbcommon. New ops `key` and `type`. `_pressed` stays buttons. New insertion-ordered `_keys`. `release_all` lifts keys after buttons: key-up each held key in press order, then `modifiers(0, 0, 0, group)`. |
|
||||
| `src/hhstream_pointer/backend.py` | Still the only pywayland importer. Bind the manager at version 1, `create_virtual_keyboard(seat)` **once** (not per output). `keymap` upload from a memfd. Sink methods `key(code, state)` and `modifiers(dep, latch, lock, group)` only — no names, no UTF-8. |
|
||||
| `src/hhstream_pointer/__main__.py` | Flags below. Build the keymap and upload it before `listen`. `_lift_everything` already calls `release_all`; once that op lifts keys, shutdown covers them. Update the comment that today only mentions buttons. |
|
||||
| `src/hhstream_pointer/server.py` | No change. |
|
||||
| `src/hhstreamd/driver.py` | `key()`, `type_text()` on `PointerDriver`. `release_all` / `pressed` read the new fields. Same socket, same "do not retry a request that was written" rule. A failed `release_all` is still `DriverUnavailable`: a dropped lift leaves a key down. |
|
||||
| `src/hhstreamd/pointer.py` | Not a new class. On `_Held`, a `keys` dict beside `buttons`. `press_key` / `unpress_key` / `keys_pressed`, mirroring `press` / `unpress` / `pressed`. `expire()` still returns `LeaseInfo` only — no keys. Callers read `keys_pressed()` first, as they must read `pressed()` first (`CONTRACT-sessions.md` §3.3). |
|
||||
| `src/hhstreamd/manager.py` | `keyboard_key` and `keyboard_type` go through `_refresh` (live session, holder, not expired, container output required). Driver first, then the lease mirror — the `pointer_button` order, so a refused key is not recorded as down. `_expire_pointer` reads keys before `expire()` and uses the one `release_all`. Same on `pointer_release`, kill, reap, and driver loss. |
|
||||
| `src/hhstreamd/protocol.py` | Verb allowlist plus the mutation set. |
|
||||
| `src/hhstreamd/policy.py` | `[keyboard]` next to `[pointer]`. |
|
||||
| `src/hhstream_session/__init__.py` | `_build_keyboard` beside `_build_pointer`. |
|
||||
| `src/hhstream_session/format.py` | `render_pointer_state` shows held keys. |
|
||||
| `src/hhstream_session/lease.py` | No new store. `pointer-leases.json` remains the cache. |
|
||||
| `nix/module.nix` | No new unit. Extend `hhstream-pointer.service` argv. Emit `[keyboard]` into the generated TOML the way `[pointer]` is emitted. Options under `sessions.keyboard`: `layout` (default `"us"`), `variant` (default `""`), `options` (default `""`), `model` (default `"pc105"`). |
|
||||
| `nix/package.nix`, `flake.nix` devshell, `pyproject.toml` | `xkbcommon` for the driver and for unit tests. No new console script. |
|
||||
| `nix/hyprland-conf.nix` | Keep `input:follow_mouse = 1`. Extend the comment: keyboard events have no output, so this warp is what moves keyboard focus, and the lease is what stops another session stealing it mid-string. |
|
||||
| `docs/CONTRACT-sessions.md` | Amend §3.3 (key mirror, `expire()` still carries no keys), §3.4 (ops above), §4 (verbs, idle rule). |
|
||||
| `README.dev.md` item 3, `checks/vm-pointer.nix` header | Stale relative to the conf file as read: they still say `hyprland-conf.nix` ships `follow_mouse = 0` and that the keyboard task will choose. The conf file sets `1`. Fix the prose when implementing; do not set the conf back to `0`. |
|
||||
|
||||
Flags the unit passes, defaults matching §5's style of frozen defaults:
|
||||
|
||||
```
|
||||
hhstream-pointer --socket /run/hhstream/pointer.sock \
|
||||
--compositor-runtime-dir /run/hypr-stream \
|
||||
--xkb-layout us --xkb-model pc105 \
|
||||
--xkb-variant '' --xkb-options ''
|
||||
```
|
||||
|
||||
A layout xkbcommon rejects, before any Wayland connect, exits **2**. Compositor failures stay **10**.
|
||||
|
||||
---
|
||||
|
||||
## 3. Lease
|
||||
|
||||
**Decision: one lease. Keyboard verbs require the pointer lease. Do not add a keyboard lease, a `keyboard` boolean, or a second sweeper.**
|
||||
|
||||
§6.7 says the lease "gains a `keyboard` flag" and that no new mechanism is needed. The flag is the wrong reading of the code that shipped. `_Held.buttons` exists because `release_all` must know **which** buttons went down and in what order (`src/hhstreamd/pointer.py`, `ops.py`). A boolean cannot lift `Control_L` and not `a`. The key set is that pattern, not a new state machine.
|
||||
|
||||
Why sharing is required by the seat, not just by the plan: there is one `wl_seat` and one keyboard focus. `follow_mouse = 1` (`nix/hyprland-conf.nix`) means another session's warp retargets the keys. Two leases would let session B move the pointer while session A believes it is typing into its own window. `LeaseBusy` on acquire is already the mutex (`PointerLease.acquire` refuses even the current holder a second id, so a drag — or a half-typed chord — cannot have its id silently replaced).
|
||||
|
||||
Rules that keyboard inherits unchanged:
|
||||
|
||||
- Acquire / refresh / hard cap (`ttl` 30 s, `max` 300 s, refresh moves only the idle deadline).
|
||||
- `expire()` does not lift anything. The broker reads the mirror, drops the lease, then `release_all` on the driver (`_expire_pointer`).
|
||||
- Driver death force-releases (`driver_gone`). The driver is not asked to remember who held the lease.
|
||||
- Kill and reap of the holder drop the lease with the existing reasons (`kill` / `crash`).
|
||||
- CLI cache is re-checked by the broker; a stale id is exit 9 and then forgotten.
|
||||
- Gesture verbs do not mark activity.
|
||||
|
||||
**TTL expiry with keys held down.** Same as buttons (`PLAN` §6.2 "Never leave a button down", implemented in `_expire_pointer` and in `_lift_everything`):
|
||||
|
||||
1. Sweeper (or the next verb's `_refresh`) sees the deadline.
|
||||
2. Broker reads `keys_pressed()` and `pressed()` **before** `expire()`.
|
||||
3. `expire()` drops the lease. It is not kept open to finish the word.
|
||||
4. Driver `release_all`: button-ups, then key-ups in press order, then `modifiers(0, 0, 0, group)`, then the pointer `frame`s it already sends for buttons. No keyboard frame.
|
||||
5. Audit `pointer_lease_timeout` gains `keys_lifted`. The session's pointer bookmark is stamped `released_by: timeout` as today.
|
||||
6. The next `acquire` is free. Nothing is replayed. A held Shift does not leak into the next holder.
|
||||
|
||||
If `release_all` fails, do what the button path does: log it, still drop the lease, do not retry a request whose outcome is unknown (`driver.py`). The stuck-key case is a risk in §6, not a second expiry protocol.
|
||||
|
||||
`type` refreshes **once per broker verb**, inside `_refresh`, not once per character. Explicit `down` of a modifier stays in the lease mirror across later `type` calls. `type` itself must not leave letters down.
|
||||
|
||||
A warp while a key is held stays legal, same as a drag. Focus can follow that warp. That is another reason both devices share the lease: the holder is the only session allowed to move focus until release or TTL.
|
||||
|
||||
---
|
||||
|
||||
## 4. Keymap and text
|
||||
|
||||
**Decision: one seat-wide XKB layout, chosen in config, default `us` / `pc105` / empty variant / empty options. The virtual keyboard's keymap is the one we upload. Hyprland's `input:kb_layout` is not read and does not affect injected keys.**
|
||||
|
||||
Do not query `hyprctl getoption` to fill a default. The broker's rule is that unverified fields stay null (`PLAN` §7); a layout guessed from the compositor would drift on reload. Operators who are not on `us` set `sessions.keyboard.layout`.
|
||||
|
||||
**Decision: v1 is a single group.** A comma-separated layout (`us,ru`) is rejected at config load. Group in every `modifiers` request is **0**. No group-switch keysym.
|
||||
|
||||
**Where it lives.** `[keyboard]` in the broker TOML (`policy.py`), rendered by `nix/module.nix`, passed as the flags in §2. The driver compiles it at startup and uploads that byte string. The session record does not gain a layout field. Per-session layouts would mean re-uploading `keymap` on every acquire, which drops held state; not in v1.
|
||||
|
||||
**How text is expressed.**
|
||||
|
||||
- `keyboard type` takes a Python `str` of Unicode scalars. The keymap maps each scalar to one keysym, then to one `(evdev code, modifier mask)`.
|
||||
- Newline → keysym `Return`. Tab → `Tab`. Other C0 controls are rejected with the rest of the string, not skipped.
|
||||
- `A` is Shift+`a` via the **mask**, not via a typed Shift key, unless the user also has Shift explicitly held.
|
||||
- Explicit modifier holds are a baseline. `type` ORs each character's mask with that baseline, sends the letter, and restores the mask to the baseline afterwards. So `down Control_L` then `type c` is Ctrl+C, and Control is still down when `type` returns.
|
||||
- If any **non-modifier** key is already held, `type` is rejected and emits nothing.
|
||||
- Validation is complete before the first sink call. Unmappable character → the whole request fails, no prefix typed. This includes `é` on `us`.
|
||||
- Dead keys (`dead_acute` and the like) are a rejection even if the layout names them. v1 does not emit a two-event compose. `us(intl)` is therefore a poor choice until a later stage; default `us` does not need it.
|
||||
- No auto-repeat timer in our process. Whether Hyprland repeats a virtual key that we leave down is unverified (Stage 0).
|
||||
|
||||
**Modifier decision:** for a shifted character, send `modifiers(shift_mask, 0, 0, 0)` and the letter's `key` down/up, then restore the previous mask. Do **not** also synthesize `Shift_L` down/up for that character. For an explicit `keyboard down Shift_L`, send the Shift **key** and a matching `modifiers` update, and remember both in the held-key record so `release_all` undoes both. Sending both a synthetic Shift key and a mask for an ordinary capital is how toolkits apply Shift twice or not at all; the VM check in §5 is what proves the letter path against a real `wl_keyboard` client.
|
||||
|
||||
Named non-character keysyms used by `key`/`down`/`up` go through the same resolver, not a second table that can disagree with the layout. `buttons.py` can stay a hand-written evdev table because the protocol takes those codes directly; keysyms cannot.
|
||||
|
||||
---
|
||||
|
||||
## 5. Tests
|
||||
|
||||
Match the split already in the tree: the dispatcher is tested against a recording sink and is not tested for authority (`tests/pointer/test_ops.py`); the lease is pure with `FakeClock` and no sleep (`tests/pointer/conftest.py`); the broker is tested with a real UNIX socket and `FakeDriver`, Wayland substituted (`tests/sessions/unit/test_pointer_verbs.py`). No integration test grows a fake compositor. `tests/sessions/integration/` does not gain Wayland.
|
||||
|
||||
### Unit
|
||||
|
||||
| Test | Asserts |
|
||||
| --- | --- |
|
||||
| `tests/pointer/test_keymap.py` | Real xkbcommon, layout `us`: `a` → evdev 30, mask 0; `A` → 30 plus Shift; `Return` and `\n` → `KEY_ENTER`; empty string → `[]`; `é` and a C0 control other than `\n`/`\t` raise; the raise happens with zero sink calls when driven through the dispatcher. Dead keysym raises. |
|
||||
| `tests/pointer/test_ops.py` | Extend `RecordingSink` with `key(code, state)` and `modifiers(...)`. `key down` then `pressed` lists the name. `up` of a key that was not down reports it not lifted (same as `unpress`). `release_all` lifts keys in press order and ends with a zeroed `modifiers`, even if the last `warp` was to another output (keys are not per output). `type` emits down/up pairs and leaves `keys` empty. `type` with `Control_L` held ORs the mask and leaves Control down. `type` with `a` held emits nothing and returns `ok: false`. A keymap that raises on the third character emits nothing. |
|
||||
| `tests/pointer/test_lease.py` | `press_key` / `keys_pressed` order. `expire()` still returns no key list; the test reads `keys_pressed()` first, then `expire()`, then sees an empty lease. Existing busy/stale/`lease_id` cases stay as they are. |
|
||||
| `tests/pointer/test_vendored_protocol.py` | Interface name `zwp_virtual_keyboard_manager_v1` and requests `keymap`, `key`, `modifiers`. No display. |
|
||||
| `tests/sessions/unit/test_keyboard_verbs.py` | New, harness copied from `test_pointer_verbs.py`. No lease → 9. Other session holds the lease → 9 (not 8). Second `pointer.acquire` while typing → 8. `type` refreshes TTL: advance the `ManualClock` by less than `ttl` after a type and the lease survives; advance past it and `release_all` runs with the held key. Driver rejection does not stick the key in the lease. `type` does not move `last_activity_boottime`. `pointer.release` and kill both lift keys. `pointer.state` includes `keys`. |
|
||||
| `tests/sessions/unit/test_settings.py` | `[keyboard]` defaults to `us`; a non-table is `PolicyError`; `us,ru` is rejected. |
|
||||
| CLI surface test (`tests/sessions/unit/test_cli_surface.py`) and broker verb list (`test_broker_protocol.py`) | `keyboard` in help; help says the pointer lease is required; `keyboard.type` / `keyboard.key` are known verbs and are in the mutation set. |
|
||||
|
||||
`nix develop -c python -m pytest tests/pointer tests/sessions/unit -q` is the gate. Not run for this plan.
|
||||
|
||||
### Integration / VM
|
||||
|
||||
**Decision: a new `checks/vm-keyboard.nix`, same shape as `checks/vm-pointer.nix`. Do not call the feature done on unit tests alone.** `docs/TODO/scroll-has-no-compositor-proof.md` is the precedent: scroll shipped on the driver socket with no compositor proof. Keyboard's only purpose is that the client receives symbols.
|
||||
|
||||
- Victim: a small Wayland client beside `checks/lib/pointer_target.py` (or an extension of it) that sets `xdg_toplevel` app id from `$HHSTREAM_APP_ID` and prints `wl_keyboard.enter`, each `key` (evdev code, state), each `modifiers` mask, and the keysym it decodes from the keymap the compositor forwarded. Read-back is `hhstream session logs`, as in `vm-pointer`.
|
||||
- Assertions, real time, no injected clock (the unit clock already covers the arithmetic; `vm-pointer`'s expiry subtest states why):
|
||||
1. After `pointer acquire` on session A, A's log shows `wl_keyboard.enter`. This is the focus measurement. If it fails, the check is red and Stage 4 is in play — do not weaken the assertion.
|
||||
2. `keyboard type A hello` decodes to the characters `hello` on A and to nothing on session B.
|
||||
3. `keyboard down A Shift_L`, wait out a short `--ttl`, A's log shows the key-up and a cleared mask, and `pointer state` shows no holder.
|
||||
4. A second acquire during the hold exits 8.
|
||||
- Mutation, per `PLAN` §14: the character assertion must fail if `keymap` was never uploaded (keycodes without a decoded `hello` are a failure, not a pass).
|
||||
- `globalTimeout` on the order of `vm-pointer` (1800). This check is slow; that is not a reason to fold it into a unit test.
|
||||
|
||||
---
|
||||
|
||||
## 6. Risks, open questions, stages
|
||||
|
||||
### Risks
|
||||
|
||||
- **Focus may not follow a virtual pointer.** The conf comment says `follow_mouse = 1` was set so pointer motion moves keyboard focus, after `vm-pointer` measured that `follow_mouse = 0` delivers clicks to nobody when the aimed window is unfocused. That measurement was pointer delivery, not `wl_keyboard.enter`. Unverified for a `zwlr_virtual_pointer_v1` warp. If Stage 0 shows no `enter`, the fallback is one existing IPC call — `dispatch focuswindow` on the session's `window.stable_id` — immediately before the first key of a lease. Do not build that fallback in Stages 1–2.
|
||||
- **Hard death of the virtual keyboard can leave a key down inside Hyprland** if destroy does not release keys. Unverified on this build. Our shutdown path sends key-ups first (`_lift_everything`). A killed `-9` driver skips it. Do not add a retry loop the button path does not have.
|
||||
- **Auto-repeat.** If Hyprland repeats a virtual key we leave down, `keyboard down a` floods the window until TTL (default 30 s). Stage 0 records whether it repeats. If it does, the help text says so; v1 still has no repeat-rate control. Chords (`Control_L` held across a `type`) are the justified use of `down`.
|
||||
- **Mask versus Shift keycode.** §4's letter path can still be wrong for a toolkit that ignores `modifiers` and only shifts when it sees `KEY_LEFTSHIFT`. The VM victim must print both the mask and the decoded sym. Chromium is not the first victim (`vm-pointer` avoided it; it is expensive and the app-id rule in `README.dev.md` item 4 only binds clients that honour `$HHSTREAM_APP_ID`).
|
||||
- **The lease is not a security boundary.** `PLAN` §6.6: any client may bind the pointer global. The keyboard global is the same. Stage 4 sandboxing is unchanged and out of this work.
|
||||
- **Concurrent typing is impossible** on one compositor, for the same reason concurrent pointers are (`PLAN` §1). A second seat is Stage 5 and still CPU-gated. This work does not pretend otherwise.
|
||||
- **`xkbcommon` in the devshell** is a new dependency. Tests that import `keymap.py` cannot skip when it is absent; an optional dependency would let the unmappable-character path rot.
|
||||
|
||||
### Open questions
|
||||
|
||||
1. Stage 0 global name and version on the actual `/run/hypr-stream` socket. §6.7's [RAN] is the claim; this session did not repeat it.
|
||||
2. Does acquire's warp produce `wl_keyboard.enter` on the aimed window under the shipped `follow_mouse = 1`?
|
||||
3. On virtual-keyboard resource destroy, does Hyprland release keys that were down?
|
||||
4. Does Hyprland auto-repeat those keys?
|
||||
5. Operator: is default layout `us` acceptable for this machine, or must the NixOS option be set to something else before the VM check is written?
|
||||
6. A later stage, only if agents must type arbitrary UTF-8: a throwaway keymap that maps spare keycodes to `0x01000000 + codepoint` for the duration of one `type`, then restores the layout keymap. Not v1. It reintroduces the "upload resets held modifiers" problem, so it has to refuse to run while any key is held.
|
||||
|
||||
### Stages
|
||||
|
||||
| Stage | Work | Rough size | Gate |
|
||||
| --- | --- | --- | --- |
|
||||
| 0 | Read-only. Dump globals. One tiny client, or a few minutes in the existing VM, to record `enter`, repeat, and destroy-while-held. Write the results down next to the check. No broker changes. | S, about half a day | Named global is `zwp_virtual_keyboard_manager_v1`. Anything else stops the project. |
|
||||
| 1 | Driver: vendor, `keymap.py`, ops, backend upload, `__main__` flags, pointer unit tests. Pointer still runs if the global is missing. | M, about 3–5 days | `pytest tests/pointer` green. `type` of an unmappable character emits no events. |
|
||||
| 2 | Broker mirror, verbs, CLI, policy, module argv, session unit tests, contract doc amendments, stale `follow_mouse` comments. | M, about 2–4 days | `pytest tests/pointer tests/sessions/unit` green. Expiry test shows `release_all` lifting a held key. No new unit file under `nix/module.nix` beyond argv and TOML. |
|
||||
| 3 | `vm-keyboard.nix` and the keyboard target client. | M, about 2–3 days | Characters on A, silence on B, key-up after a real TTL, second acquire is 8. Red if the keymap was not uploaded. |
|
||||
| 4 | Only if Stage 0 or Stage 3 shows no `wl_keyboard.enter` after warp. `focuswindow` via the broker's existing compositor IPC, then re-run Stage 3. | S, about a day, contingent | The Stage 3 enter assertion goes green without loosening it. |
|
||||
|
||||
Rollback of Stages 1–2 is reverting the commit: old brokers never send `key`/`type`, and an old driver rejects unknown ops with `ok: false` (`ops.py` already answers unknown `op` instead of dropping the connection). No registry schema bump. Bookmarks stay as they are.
|
||||
Reference in new issue
Block a user