During the Plasma sprint in March, we've been fleshing out plans to create a
mechanisms for apps to be represented by the shell.
The goals it to achieve a richer integration between apps and the workspace.
The idea is that windows can announce a richer set of metadata that can be
used to enhance the window's representation in the workspace/shell. Think of
- task switchers with more meaningful thumbnails of better quality
- server-side decoration actions to "remote-control" the app
- user-visible features similar to the mpris-controls in the task bar
- currently open document
More thoughts on this are explained on this wiki page
https://community.kde.org/KWin/Window_Metadata (which is not quite up-to-date
with the implementation I'm proposing here).
These features are achieved by using D-Bus for client-server communication.
I've written some basic components that provide the basic mechanisms for that.
There's a library for the clients, one for the server, qtquick components for
both sides, autotests and interactive testing apps. The functionality is
fairly slim, I've only implemented to allow apps to announce titles and
thumbnails. The title property is rather simple, the thumbnail property is a
bit more involved -- the server (compositor, task manager, task switcher,
etc.) requests thumbnails to be passed via a connection-private file
descriptor and communicates the semantics over D-Bus.
Here's a run-down of what's currently there. I think it provides a good base
for the above goals. It's not complete by any means, api docs are also missing
still (I expect some semantics to change).
Attached, find a README.md file that explains the implementation and its
components.
The code can be found in the kde:scratch/sebas/windowmetadata repository.
Feedback is most welcome.
--
sebas
Sebastian Kügler • http://vizZzion.org • http://www.kde.org
# WindowMetadata
WindowMetadata is a task-based IPC framework for communication between applications and the
windowing system
WindowMetadata allows applications to provide metadata as hints to the windowing system, such
as task switchers. WindowMetadata is an abstract representation of an appliation or window to
make identifying and monitoring it visually easier.
Clients can send a window or application thumbnail that is more useful for identifying a
certain window than a scaled screenshot of the window.
## Overview
WindowMetadata provides the following components:
- WindowMetadata::Client : used in applications to provide metadata and thumbnails
- WindowMetadata::Server : used by a window manager or task manager to show apps' metadata
- 2 QtQuick import plugins providing a declarative API for both client, and server
## Using WindowMetadata in your application
### Client Library
The WindowMetadataClient library is the basic thing that is used in an application. It
offers a QObject-based fully abstracted API to register a client (typically a window or
an app).
After registering, a WindowMetadata::Client::MetadataClient, the client...
- sends the basic set of metadata to the server (use the property setters)
- reacts to thumbnail requests by painting or loading QImage MetadataClient::setThumbnail()
- indicates when the thumbnail is outdated: MetadataClient::setDirty()
Thumbnails can only be sent by the client after request by the server. The filedescriptor
used to pass the thumbnail to the server is entirely controlled by the server after it has
been passed off. The server asks for a thumbnail at a given size, the rendered thumbnail
should be rendered by the client given the size constraints.
This mechanism is used to
- make sure the preview is painted in a size that makes sense for the server
- allows to only update the thumbnail when it's really needed and useful, and thus saves
the client a lot of cycles as rendering effort is minimized
The is NOT meant to do live updates, but rather display a preview of the app that's easy to
recognize visually, for example when an opened document changes, or the application's state
is interesting to the user currently involved in a different task.
Internally a D-Bus Adaptor is created. The client's lifecycle is managed in terms of
creating the client, registering it, and deleting the object.
This library has minimal dependencies, public QtGui and privately also QtDBus.
Caveat: Clients should try to avoid painting text into the thumbnail, as this may lead to
DPI problems when a client's DPI (in which this text is rendered) is passed to to the
server, which renders it on a display with a different DPI. Don't do this. Rather paint
a nice picture and provide the text in the other metadata as strings.
### Client QtQuick import
The Client QtQuick import offers a declarative API to the client library. The declarative
API revolves around the MetadataClient QtObject. This object allows to specify the
thumbnail as QQuickItem, which will rendered offscreen and only when the server asks for
an update, and then sent to the server.
Use it
import org.kde.kwin.windowmetadata.client 1.0
Example and test code can be found in tests/client/ClientTest.qml
This plugin has a small footprint in terms of library dependencies, it only needs the Client
library and additionally QtQuick and QtQml.
### Server Library
The server library provides a model of connected clients and metadata for each client. This
is typically used by the compositor or task manager.
### Server QtQuick import
The server QtQuick import provides a QList<QObject> based declarative model of the clients.
Each client in the model provides a thumbnail path, which is understood by the imageprovider
in this plugin. This image provider provides the bridge between each client's thumbnail and
correctly takes care of updating an Image item.
Use it
import org.kde.kwin.windowmetadata.server 1.0
Example and test code can be found in tests/server/ItemList.qml
### With CMake
WindowMetadata installs a CMake Config file which allows to use WindowMetadata as imported targets. There is
one library for the Client and one for the Server.
To find the package use for example:
find_package(KF5WindowMetadata CONFIG)
set_package_properties(KF5WindowMetadata PROPERTIES TYPE OPTIONAL )
add_feature_info("KF5WindowMetadata" KF5WindowMetadata_FOUND "Required for the awesome application window metadata integration")
Now to link against the Client library use:
add_executable(exampleApp example.cpp)
target_link_libraries(exampleApp KF5::WindowMetadataClient)
To link against the Server library use:
add_executable(exampleServer exampleServer.cpp)
target_link_libraries(exampleServer KF5::WindowMetadataServer)
### With QMake
WindowMetadata installs .pri files for the Client and Server library allowing easy usage in QMake based
applications.
Just use:
QT += WindowMetadataClient
Respectively:
QT += WindowMetadataServer
Please make sure that your project is configured with C++11 support:
CONFIG += c++11
# Testing
This library contains both autotests and interactive graphics test applications. In order
to verify that your changes don't break anything, please run make test. If you add or
change anything, please make sure it has a test case under autotests/. (Extend an
existing one or add a new one.)
The following test apps can be used to test and demo this library:
- tests/client/wmclient : simple, QWidget based app allowing to register and manipulate a
client)
- tests/client/ClientTest.qml : simple, QtQuick "app" allowing to register and manipulate
a client (run with qmlscene)
- tests/server/ItemList.qml : simple, QtQuick "app" showing clients and thumbnails (run
with qmlscene)
- tests/server/wmserver : very basic, QWidget based app allowing showing the number of
registered clients)
Autotests are running on a separate dbus interface as to not get confused by "real" clients.
# Protocol implementation
The thumbnails in the protocol are transported by filedescriptors. These are "made up"
by the items on the server side and passed to the client in a thumbnail update request.
The fd is not passed along with the thumbnail update signals back from the client to the
server, the server keeps track of where the client is supposed to store the new image data.
This avoids announcing protocol internal filedescriptors on the D-Bus session bus.
Here's how the ownership of the images changes:
- server requests thumbnail, passes filedescriptor, at this point, it
MUST NOT touch it anymore, as the client will write to it now
- the client writes to it, closes it and sends an update to the server,
at this point, the client MUST NOT touch it anymore, as the server may
now read it
- The server will clean up the images, when
- the controller gets deleted
- the item isn't interesting anymore (size changed)
- the client goes away (and with it the item deleted)
- the server gets destroyed or exits
- The server is the sole responsible for deleting the thumbnail images
- The client can indicate that the current thumbnail is outdated
- The server may, or may not request a new thumbnail image (using a new fd)
# Work-in-progress
You can find the current list of work items in the TODO.txt file in the root of this repository.
_______________________________________________
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
