As part of the GNOME project, we have Orca. It is a screen reader that reads what an application is displaying out loud, or sends it to a Braille display. It allows a blind user to interact with GNOME. There are currently several issues with Orca on Wayland, since it (and AT-SPI, the accessibility API that it uses) were originally designed for X11, without regard to the kind of security constraints that Wayland has.

Traditionally, Orca has used what amounts to a key filter implemented by AT-SPI. The toolkit (gtk, qt, etc) would make a dbus method call to AT-SPI when a key is pressed, passing on the keycode, keysym, and text associated with the key (Orca uses the latter to announce the key that was just pressed). The AT-SPI registry daemon would make a method call to Orca, Orca would return a boolean to indicate whether it consumed the key or not, and AT-SPI would return the result to the toolkit. This is no longer supported by gtk 4, so I am trying to solve things another way. There is a good amount of discussion at https://gitlab.gnome.org/GNOME/gtk/-/issues/1739

Orca uses this key filter for a few things. It wants to announce the key that the user just pressed. If the user presses the control key, then Orca will stop reading whatever it is reading. It also implements commands in this way (the user can press a key to have it read the current line, to give one example). For X11, I can have AT-SPI/Orca watch for XI_KeyPress events and call XIGrabKeycode when Orca wants to grab a key, but I'm struggling to figure out what to do on Wayland. I think that I would need to propose a new protocol or two, but I foresee security being a challenge/concern, so I'm wondering if anyone has any advice in particular. Xwayland-keyboard-grab looks like it would do part of what I need, if Orca ran as an xwayland client, but it wouldn't help in terms of passing key press notifications on to Orca for the purposes of key echoing and interrupting speech, unless a grab was active for the particular key. If I enable my X11 code for Wayland, then I don't see XI_KeyPress events for keys going to other applications, which I think is expected given Wayland's security model.

Thanks,
-Mike

_______________________________________________
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/wayland-devel

Reply via email to