[plugins] Re: comparison of plugin mechanisms

2009-02-13 Thread Zbigniew Chamski
Hello everybody,

2009/2/13 Basile STARYNKEVITCH  wrote

>Hello All,
>
>I am not sure of the relevance of the "new pass integration plugins" 
> examplified by MELT.
>[...]
>In my view, MELT fits quite precisely in the "production plugins" 
> definition, while indeed I expect it to be useful mostly for 
> "experimental/research" plugins.
>
>In my view also, the "new pass integration plugin" category should not 
> really exist, because it probably can fit inside one (or both) of the above 
> categories.
>[...]
>Also, I don't understand why "production plugins" or 
> "experimental/research plugins" could not be coded in another language than 
> C. For sure, they could probably be coded in a suitable subdialect of C++ at 
> least.

The classification of plugins into the three categories was intended
to illustrate the differences in API constraints; MELT is an example
of multi-language programming that calls for non-C bindings in the
plugin API.  This is in fact orthogonal to the level of abstraction,
which is the key differentiator between the "production" and the
"experimentation" approaches.  Efficiency is a concern in all cases,
but is addressed at different levels: the idea behind the experimental
part of the API is to help searching for a good/better way of dealing
with a problem, while the production API should provide the means of
implementing the chosen method as efficiently as possible.

>   I did not understood yet how exactly can Grigori's production plugin API be 
> used to add e.g. one plugin pass inside the pass manager. Eg how to add one 
> pass fooplugin_pass provided by a plugin just after the 
> pass_inline_parameters, or another plugin pass barplugin_pass just after 
> pass_ipa_struct_reorg?
>I assume such things could be possible...

To support "dynamic" passes (supplied by plugin), Grigori's API needs
two more primitives to register and unregister new passes (plus a
function to get the struct opt_pass by name for the production API).
The rest of the machinery remains unchanged.  The solution will be
then to:

* register the dynamic passes upon loading the plugin,
* generate an event in execute_pass_list (say, "select_next_pass"),
* modify pass chaining (or run passes directly) in the corresponding
callback, based on the current position in the pass list,
* unregister dynamic passes when unloading the plugin.

>Regarding MELT, I believe that if a plugin facility has, in addition of 
> Gregori's feature, the ability to insert new dynamic passes and the ability 
> to add some extra GTY root & marking routine, I could fit MELT into that. If 
> that happens, MELT would become somehow a "metaplugin" machinery, in the 
> sense of a plugin which will itself load some dynamic code.

Insertion of dynamic passes can be implemented very rapidly (cf.
above).  For GTY, maybe your patch came just too early - the
motivation for it is much stronger now.  Finally, the "meta-plugins"
seem a natural consequence of the plugin concept itself and I expect
to see more of them as soon as the plugin mechanism gains momentum in
the community.

All the best,


   Zbigniew


New Interactive Compilation Interface (ICI) v2.0 has been released

2009-06-05 Thread Zbigniew Chamski
Dear all,

We just finished and released the new Interactive Compilation
Interface v2.0 for GCC 4.4.0, based on the official GCC 4.4.0 release code.
Full source code, documentation and a variety of plugin examples
are available on the cTuning.org website at http://ctuning.org/ici and
http://ctuning.org/wiki/index.php/CTools:ICI:Documentation.

The ICI plugin infrastructure used in the ICI 2.0 release is
complementary to the developments on the GCC plugins branch, and provides
a high-level API aimed at supporting fast experimentation/prototyping of
new analyses and optimizations.  Specifically, the ICI API introduces:

* an abstraction of compiler state through the "feature" mechanism (with
possibility of limited adjustments to, e.g., the values of internal
compiler parameters);

* a simple interface for pass management: substitution of pass
manager, overriding of pass gates, tracking of current/next pass;

* dynamic registration of new plugin event kinds.

The integration of ICI API with the GCC plugin support developed on
the plugins-branch has already been demonstrated last February (cf.
http://gcc.gnu.org/ml/gcc-patches/2009-02/msg01242.html).  Future
releases of ICI will be based on - and will extend - the mainstream
GCC plugin infrastructure.

We hope that using ICI will make GCC more attractive and more accessible for
researchers.  We currently use it to solve long term research objectives
such as automatic tuning of compiler optimization heuristic or automatic
program optimization for multiple architectures.

There are several ICI extension projects that we hope to finish this summer,
such as:

* automatic selection and tuning of fine-grain program transformations,
* generic function cloning for program run-time adaptation,
* program instrumentation,
* GCC4CLI split-compilation for VMs.

You are warmly encouraged to use ICI and other collaborative tools of
the cTuning project.  Join the effort and follow the developments on the
cTuning community mailing list:
http://groups.google.com/group/ctuning-discussions

Yours,

  Zbigniew Chamski and the cTuning.org community