Hi list,
My name is Brett Stottlemyer, here with my 2nd sandbox request. I work for
Ford Motor Company, and Ford has graciously agreed to let us contribute some of
the cool stuff we've developed back to Qt. We aren't quite done with
everything yet (still waiting on the Corporate CLA), but I will be at Qt
Contributors Summit in a week and a half to discuss this and a 2nd project
(previously posted QQSM), and answer (hopefully a bunch) of good questions.
I'd like to officially request a sandbox for: Replicant
What is it?
Replicant is a Qt library for Inter Process Communication (IPC). It is meant
as an alternate to QtDBus, but with a very different usage pattern.
What need does this module solve?
There were a number of reasons that made QtDBus unattractive for my use case.
1) Desire to work on Windows as one of the target platforms. This required
building the module/dependencies from source, by every developer working on the
project.
2) Complexity in initialization vs. notification. There did not seem to be a
good way to get the desired properties of an object and enable change
notification from that instant forward. Getting properties and getting
notifications are separate requests, so there is the possibility of a race
condition that would cause the client to be out of sync with the server.
3) Using QtDBus as an IPC mechanism between multiple physical computers does
not seem to be a recommended use-case.
4) QtDBus makes it easy to incorporate an existing D-Bus implementation (often
not in Qt) and incorporate it into Qt. However, QtDBus requires a fair amount
of refactoring if all programs are Qt programs, to marshal to/from D-Bus types
and connect the event handler.
Detailed description
At an overly simple level, the Replicant module is like Qt's QueuedConnection
mechanism extended into an IPC mechanism. In a QueuedConnection, when a Signal
is emitted, the arguments are copied into an event, which is placed in the
event queue of the receiving thread. When that event is reached in the
receiving thread, the Slot connected to the Signal is passed the argument
copies to act on. This is what prevents the emitting thread from having to
block waiting for queued Slots to run. In Replicant, you have the "same"
QObject in multiple processes, and everything needed to handle Property changes
and Signal/Slots is copied between processes. Replicant handles almost all of
the details behind the scenes.
While the intention is for Replicant to be easy for a user to work with, the
description provided here is intended more for the developers of Qt itself, or
anyone who wants to know how Replicant works internally.
Replicant is implemented using QIODevice to marshal information between
processes. It is implemented through copies of objects, not sharing via shared
memory. In Replicant speak, there is a Prime (the "real" object) and
Replicants (one or more copies). The Replicants are "proxies" to the real
object, in the sense that property changes are forwarded to the Prime, and when
the Prime changes, those changes are propagated to every Replicant.
Replicants are "latent copies". That is, all changes to the Prime will be
received by every Replicant, in time-order, although there is a
non-deterministic delay for that to happen.
Replicant relies on Qt's existing eventloop (time) serialization to make its
guarantees. The first issue is initialization, the second is synchronization.
When a Replicant is requested, the process that owns the Prime gets the request
via the QIODevice. When the event is processed, the current values of every
property are gathered, and sent in a reply to the Replicant process. This is
not an atomic operation, so it assumes all property changes will be handled in
the eventloop as well. In this way, any future changes to object properties
will be forwarded over the QIODevice as well, but they will be handled in the
Replicant process in the same order they are handled in the Prime process.
The actual mechanism for handling Signals/Slots and Property (Q_Property)
changes is tied into the internals of qt_metacall. In effect, the parameters
to qt_metacall themselves are marshalled over the QIODevice, and passed into
qt_metacall for each object within the Replicant process. Note that in order
to achieve this, you need 1) a custom qt_metacall implementation (not generated
by MOC) and 2) strict versioning to ensure that there isn't a mismatch between
the Prime and Replicant. Both of these are handled by providing a Replicant
Compiler, repc. The repc program reads a template file that describes the
properties, signals and slots of a type, and generates a Prime and Replicant
version of a header file for that type. This is built as a custom Qt compiler,
so your templates can be added to your .pro file and automatically built for
you.
As mentioned, Replicant handles Properties, Signals and Slots. But there are
certain assumptions made on their usage. Replicant assumes that Slots cause
side effects, and Signals are notifications. Since Slots cause side effects,
they must run on the Prime, so changes get propagated to every Replicant. The
header file for the Prime generates a pure virtual function for every slot, and
a derived class must be created to implement the Slot. The header file created
for the Replicant automatically forwards Slot invocations to the Prime. In
Replicant terms, a notification is a side effect. Replicants can connect
handlers to Signals, but they must call a Slot if they need a Signal emitted.
Any Signal emitted by the Prime will be emitted by every Replicant. Obviously
Property changes are side effects as well. Slots must be used to change
values, not Property setters.
These may seem like serious limitations, but in practice it is expected that
*if* fully interactive objects were needed in each process, each process would
have their own full object, rather than trying to use Replicant for this
purpose. While these are not expected limitations for a QObject, they are
common in IPC implementations. But another way, it is necessary to prevent two
processes from making conflicting changes at the "same time". There are only
three ways to do this. 1) Atomic operations, 2) a locking mechanism shared by
all processes, or 3) choosing a "master" process responsible for controlling
changes. Replicant and QtDBus use the master approach (the server in QtDBus,
the Prime in Replicant).
>From an end user perspective, Replicant is not complicated. One process
>registers a Prime, and other process Acquire Replicant instances. Code using
>those objects is written just like any other Qt code, and other than a few
>lines of setup code, neither the Replicant nor Prime process needs to know or
>care that data is coming from another process. They define a template for
>their objects, then code to the defined API using the header files generated
>automatically. All of the IPC is hidden from view.
This one is pretty complex. Hard to do it justice, even with this wordy
description. I hope folks interested in this one will be at QtCS.
Please approve the sandbox!
Sincerely,
Brett Stottlemyer
Ford Motor Company
_______________________________________________
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
