capture: an in-process Wayland backend, and the measurements that justify it
The frame server could only capture by running a program once per frame
(grim). That works, but ~95% of a grim frame is waiting -- fork, connect,
handshake, round trip -- not encoding, and grim makes the compositor import
a *new* shared-memory buffer for every single frame.
This adds `--capture wayland[:OUTPUT]`: one long-lived zwlr-screencopy-v1
client per server, with one shm buffer reused for the whole run, capturing
inside the process. Measured here at 1280x720 (spike capture-speed, 200
frames; live re-check on Virtual-1, 60 frames):
grim (subprocess) 117.3 ms/frame 8.1 fps compositor 116.4 ms
grim-rs (Rust crate) 124.6 ms/frame 7.5 fps compositor 123.1 ms
this backend 78.0 ms/frame 13.9 fps compositor 67.5 ms
live, PNG both sides: this 128.5 ms/frame against grim's 389.3 ms (3.0x)
Three findings, each of which shaped the code:
* The win is buffer reuse, not language. grim and grim-rs both hand the
compositor a fresh buffer per frame, which is why their compositor CPU is
nearly double ours. The per-frame allocate-copy-swizzle that grim-rs pays
was measured in isolation at 2.4 ms of *client* time; the rest of its
deficit lands inside the compositor.
* ext-image-copy-capture-v1 is no faster (78.0 vs 77.7 ms) and its
create_frame path appears in a Hyprland segfault stack, so this keeps the
deprecated protocol grim uses. The switch is one constant when that crash
is fixed.
* PNG only, deliberately: PNG is the one encoding this package produces
without a dependency, so --format jpeg is refused rather than publishing a
Hello whose MIME type the frames do not have. A PNG frame is ~2.1 MB
against ~120 KB for JPEG q80, which is why the command backend stays the
default and why a libjpeg-turbo encoder (1.7 ms/frame, measured) is the
obvious follow-up.
The timeout is the load-bearing part of the client: pywayland only reads
events from read()/dispatch(block=True), neither of which takes a timeout,
and the server captures in a worker thread where SIGALRM is unavailable. It
therefore polls the display fd itself, so a compositor that stops answering
produces a CaptureError instead of a capture thread that never returns.
The protocol bindings are vendored under src/hhstream/_protocol/ exactly the
way hhstream_pointer already vendors its two, with regenerate.py recording
where each XML came from. No new dependency: pywayland was already required
by the input driver.
Contract: the new server flag and the two new module options are documented
in docs/CONTRACT.md, and docs/ARCHITECTURE.md's prediction that a native
screencopy client would be "a swap rather than a rewrite" is now the record
of what the swap cost.
Unit suite: 977 passed, 2 skipped, 0 failed (nix build -L .#checks.unit).
The live numbers were taken on this VM against the real compositor.
This commit is contained in:
1 parent
f3ac9ada27
commit
a7d9357b43
37 files changed
+7556
-10
No files matched your search
@@ -0,0 +1,121 @@
|
||||
"""`hhstream.png`: the hand-rolled PNG encoder, checked against zlib itself.
|
||||
|
||||
There is no image library in the unit environment (the whole point of this
|
||||
encoder is that it needs none), so the decoder below is written from the PNG
|
||||
spec too. That is not a weakness of the test: it re-derives the scanlines from
|
||||
the file the encoder produced and compares them with the pixels that went in,
|
||||
which is exactly the property that matters -- a wrong stride, a wrong colour
|
||||
type or a byte order slip all show up as a mismatch rather than as a file that
|
||||
merely starts with the right eight bytes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
import pytest
|
||||
|
||||
from hhstream.png import encode_rgb
|
||||
|
||||
SIGNATURE = b"\x89PNG\r\n\x1a\n"
|
||||
|
||||
|
||||
def decode(png: bytes) -> tuple[int, int, bytes]:
|
||||
"""Return (width, height, rgb scanlines) for a PNG this encoder can produce."""
|
||||
assert png.startswith(SIGNATURE), "missing PNG signature"
|
||||
offset = len(SIGNATURE)
|
||||
width = height = None
|
||||
idat = b""
|
||||
while offset < len(png):
|
||||
(length,) = struct.unpack(">I", png[offset:offset + 4])
|
||||
kind = png[offset + 4:offset + 8]
|
||||
payload = png[offset + 8:offset + 8 + length]
|
||||
crc = png[offset + 8 + length:offset + 12 + length]
|
||||
assert struct.unpack(">I", crc)[0] == zlib.crc32(kind + payload) & 0xFFFFFFFF, (
|
||||
f"bad CRC on {kind!r} chunk -- the file is corrupt"
|
||||
)
|
||||
if kind == b"IHDR":
|
||||
width, height, depth, color, comp, filt, interlace = struct.unpack(">IIBBBBB", payload)
|
||||
assert (depth, color, comp, filt, interlace) == (8, 2, 0, 0, 0)
|
||||
elif kind == b"IDAT":
|
||||
idat += payload
|
||||
offset += 12 + length
|
||||
|
||||
assert width is not None and height is not None, "no IHDR chunk in the file"
|
||||
raw = zlib.decompress(idat)
|
||||
stride = width * 3
|
||||
assert len(raw) == height * (1 + stride), "the scanline payload is the wrong length"
|
||||
out = bytearray()
|
||||
for row in range(height):
|
||||
start = row * (1 + stride)
|
||||
assert raw[start] == 0, "every scanline should be written unfiltered"
|
||||
out += raw[start + 1:start + 1 + stride]
|
||||
return width, height, bytes(out)
|
||||
|
||||
|
||||
def bgrx(width: int, height: int, stride: int | None = None) -> bytes:
|
||||
"""A frame in the compositor's memory order: B, G, R, X per pixel."""
|
||||
stride = stride if stride is not None else width * 4
|
||||
out = bytearray()
|
||||
for y in range(height):
|
||||
row = bytearray()
|
||||
for x in range(width):
|
||||
row += bytes(((x * 3) % 256, (y * 5) % 256, (x + y) % 256, 0x00))
|
||||
row += b"\xee" * (stride - width * 4) # row padding, if any
|
||||
out += row
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def test_pixels_round_trip_in_the_right_channel_order():
|
||||
frame = bgrx(4, 2)
|
||||
width, height, rgb = decode(encode_rgb(4, 2, 16, frame))
|
||||
assert (width, height) == (4, 2)
|
||||
# Pixel (x, y) went in as B,G,R,X; it must come out as R,G,B.
|
||||
for y in range(2):
|
||||
for x in range(4):
|
||||
b, g, r, _ = frame[(y * 4 + x) * 4:(y * 4 + x) * 4 + 4]
|
||||
assert rgb[(y * 4 + x) * 3:(y * 4 + x) * 3 + 3] == bytes((r, g, b)), (
|
||||
f"channel order is wrong at ({x}, {y})"
|
||||
)
|
||||
|
||||
|
||||
def test_a_padded_stride_is_honoured_rather_than_assumed():
|
||||
"""`stride` is the compositor's row pitch, not `width * 4`.
|
||||
|
||||
Assuming it would shear the image, which is the bug grim 1.5 shipped -- so
|
||||
the padding bytes below are deliberately not black: if they leak into the
|
||||
output the comparison fails.
|
||||
"""
|
||||
width, height, stride = 3, 2, 20 # 12 meaningful bytes, 8 bytes of padding
|
||||
frame = bgrx(width, height, stride)
|
||||
_, _, rgb = decode(encode_rgb(width, height, stride, frame))
|
||||
assert rgb == b"".join(
|
||||
bytes(reversed(frame[(y * stride + x * 4):(y * stride + x * 4) + 3]))
|
||||
for y in range(height)
|
||||
for x in range(width)
|
||||
)
|
||||
|
||||
|
||||
def test_a_memoryview_of_a_mapping_is_accepted():
|
||||
"""The caller passes `memoryview(shm_map)`, not `bytes`."""
|
||||
frame = bgrx(2, 2)
|
||||
assert encode_rgb(2, 2, 8, memoryview(frame)) == encode_rgb(2, 2, 8, frame)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"kwargs, message",
|
||||
[
|
||||
(dict(width=0, height=4, stride=0), "positive size"),
|
||||
(dict(width=4, height=0, stride=16), "positive size"),
|
||||
(dict(width=4, height=4, stride=8), "too small"),
|
||||
],
|
||||
)
|
||||
def test_bad_geometry_is_rejected_before_anything_is_encoded(kwargs, message):
|
||||
with pytest.raises(ValueError, match=message):
|
||||
encode_rgb(pixels=b"\x00" * 64, **kwargs)
|
||||
|
||||
|
||||
def test_a_short_buffer_is_rejected_rather_than_encoding_garbage():
|
||||
with pytest.raises(ValueError, match="needs"):
|
||||
encode_rgb(4, 4, 16, b"\x00" * 63)
|
||||
@@ -0,0 +1,130 @@
|
||||
"""The in-process capture backend, in the parts that need no compositor.
|
||||
|
||||
Everything that needs a real Wayland connection is left to the vm checks
|
||||
(`checks/vm-hyprland.nix` runs the server against the actual headless
|
||||
compositor), so what is worth testing here is the behaviour around the edges: the
|
||||
timeout that stops a frame from hanging forever, the recovery path that makes a
|
||||
dead compositor reconnectable, and the argv the module and the CLI agree on.
|
||||
|
||||
The `_pump` tests are the point of the file. `pywayland` only reads events from
|
||||
`read()` and `dispatch(block=True)`, neither of which takes a timeout, so a
|
||||
compositor that stops answering used to mean a capture thread that never returns
|
||||
-- and a frame server that has silently stopped serving while its socket still
|
||||
accepts connections. The stub display below is a real pipe fd that never becomes
|
||||
readable: the test passes only if the deadline is what ends the wait.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
from hhstream.capture import CaptureError
|
||||
from hhstream.server import _build_capture_backend, _join_capture_tokens, build_parser
|
||||
from hhstream.wayland_capture import WaylandCapture
|
||||
|
||||
|
||||
class _StubDisplay:
|
||||
"""Just enough `Display` for `_pump`: a fd, and a dispatch that finds nothing."""
|
||||
|
||||
def __init__(self, fd: int) -> None:
|
||||
self._fd = fd
|
||||
self.reads = 0
|
||||
|
||||
def get_fd(self) -> int:
|
||||
return self._fd
|
||||
|
||||
def flush(self) -> None:
|
||||
return None
|
||||
|
||||
def dispatch(self, *, block: bool = False, queue=None) -> int:
|
||||
return 0
|
||||
|
||||
def read(self, *, queue=None) -> None:
|
||||
# Reached only if select() said the fd was readable; on this pipe it never
|
||||
# is, so arriving here means the wait was not bounded by the deadline.
|
||||
self.reads += 1
|
||||
raise AssertionError("read() called although nothing was readable")
|
||||
|
||||
|
||||
def test_a_frame_that_never_arrives_raises_instead_of_hanging():
|
||||
capture = WaylandCapture(timeout=0.05)
|
||||
read_fd, write_fd = os.pipe()
|
||||
stub = _StubDisplay(read_fd)
|
||||
capture._display = stub
|
||||
try:
|
||||
with pytest.raises(CaptureError, match="timed out"):
|
||||
capture._pump(lambda: False, "the compositor never sent the frame")
|
||||
assert stub.reads == 0, "it waited on a readable fd instead of on the clock"
|
||||
finally:
|
||||
os.close(read_fd)
|
||||
os.close(write_fd)
|
||||
|
||||
|
||||
def test_the_pump_returns_immediately_when_it_is_already_done():
|
||||
"""A queued event must not cost a select() timeout to notice."""
|
||||
capture = WaylandCapture(timeout=0.05)
|
||||
read_fd, write_fd = os.pipe()
|
||||
capture._display = _StubDisplay(read_fd)
|
||||
try:
|
||||
capture._pump(lambda: True, "nothing to wait for")
|
||||
finally:
|
||||
os.close(read_fd)
|
||||
os.close(write_fd)
|
||||
|
||||
|
||||
def test_size_is_unknown_until_a_frame_has_actually_been_captured():
|
||||
"""The Hello must never carry a size that was guessed."""
|
||||
capture = WaylandCapture()
|
||||
assert capture.size() == (0, 0)
|
||||
|
||||
|
||||
def test_close_is_safe_twice_and_before_anything_was_opened():
|
||||
"""`close()` is the recovery path, so it runs on a half-torn-down backend."""
|
||||
capture = WaylandCapture()
|
||||
capture.close()
|
||||
capture.close()
|
||||
assert capture.size() == (0, 0)
|
||||
|
||||
|
||||
def test_a_timeout_of_zero_or_less_is_refused_at_construction():
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
WaylandCapture(timeout=0)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"spec, expected_output",
|
||||
[("wayland", None), ("wayland:Virtual-1", "Virtual-1"), ("wayland:", None)],
|
||||
)
|
||||
def test_the_backend_spec_selects_an_output_without_connecting(spec, expected_output):
|
||||
"""Constructing the backend must not open a connection -- grab() does."""
|
||||
backend = _build_capture_backend(spec, "png")
|
||||
assert isinstance(backend, WaylandCapture)
|
||||
assert backend.output_name == expected_output
|
||||
assert backend._display is None, "the connection belongs to grab(), not to __init__"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("spec", ["grim", "wayland:too:many:colons:here", "x11"])
|
||||
def test_an_unknown_backend_spec_is_refused_rather_than_guessed(spec, caplog):
|
||||
"""`wayland:a:b` is not an output name, and guessing would capture the wrong
|
||||
output silently."""
|
||||
assert _build_capture_backend(spec, "png") is None
|
||||
|
||||
|
||||
def test_jpeg_is_refused_rather_than_advertised_in_the_hello(caplog):
|
||||
"""The Hello carries the MIME type; publishing jpeg over PNG frames would
|
||||
mis-decode every frame in every client, so it is refused up front."""
|
||||
assert _build_capture_backend("wayland", "jpeg") is None
|
||||
assert "PNG only" in caplog.text
|
||||
|
||||
|
||||
def test_the_backend_flag_survives_the_token_folding_that_capture_tokens_need():
|
||||
folded = _join_capture_tokens(["--capture", "wayland:Virtual-1", "--fps", "10"])
|
||||
assert folded == ["--capture=wayland:Virtual-1", "--fps", "10"]
|
||||
assert build_parser().parse_args(folded).capture == "wayland:Virtual-1"
|
||||
|
||||
|
||||
def test_no_backend_flag_still_means_the_command_backend():
|
||||
args = build_parser().parse_args(["--socket", "/tmp/x.sock"])
|
||||
assert args.capture is None
|
||||
Reference in new issue
Block a user