Re: Defining a common plugin machinery
Hi all,
As Grigori says, and well, the project is frozen (for around one year)
since there was no expectations on it, specially since future of
plugins by that time was really unclear.
In fact the idea by the time I stopped working in it was to extend
gengtype (Garbage Collector's structure parser) to store meta-
information of the structures so that having a root node to a
structure with all the important data would allow to externally
traverse / modify GCC data without having to include all the header
files from GCC.
The reason for that wasn't to avoid the usage of GCC header files but
instead to allow data serialization and perform plugins in other
languages.
Cheers,
Cupertino
On Oct 10, 2008, at 4:14 PM, Grigori Fursin wrote:
I currently don't have any preference for a specific way to deal
with data marshaling
since currently it's enough for the MILEPOST project just to return
void* pointer,
but just wanted to mention that last year Cupertino Miranda tried to
introduce
an intermediate data layer to ICI to separate program analysis from
transformations
and potentially simplify dealing with external optimization plugins.
I think the
project has been frozen or Cupertino can comment on that if he
follows this thread ;),
but I thought to give a link to the tech. report Cupertino presented
last year,
if anyone is interested:
http://gcc-ici.sourceforge.net/papers/mfpp2007.pdf
By the way, if I am correct, GCC MELT (developed by Basile)
also attempts to address some of these issues with data marshaling
to modularize GCC ...
Cheers,
Grigori
-Original Message-
From: Brendon Costa [mailto:[EMAIL PROTECTED]
Sent: Friday, October 10, 2008 2:33 AM
To: Dave Korn
Cc: 'Taras Glek'; 'Grigori Fursin'; 'Hugh Leather'; 'Basile
STARYNKEVITCH'; gcc@gcc.gnu.org;
'Sean Callanan'; 'Cupertino Miranda'; [EMAIL PROTECTED]; [EMAIL PROTECTED]
; 'Taras Glek';
'Diego Novillo'; 'Mike O'Boyle'
Subject: Re: Defining a common plugin machinery
Sounds like you're almost in need of a generic data marshalling
interface
here.
Why do we need the complication of data marshaling?
I don't see why we need to define that all plugin hooks have the same
function interface as currently proposed. I.e. a single void*. This
makes a lot of work marshaling data both as parameters and from
return
values. This is already done for us by the language (Though i may
have
mis-understood the train of thought here).
I will propose the start of a new idea. This needs to be fleshed
out a
lot but it would be good to get some feedback.
I will use the following terminology borrowed from QT:
signal: Is a uniquely identified "hook" to which zero or more slots
are
added. (I.e. Caller)
slot: Is a function implementation say in a plugin. This is added
to a
linked list for the specified signal. (I.e. Callee)
The main concepts in this plugin hook definition are:
* Signals can define any type of function pointer so can return
values
and accept any parameters without special data marshaling
* Each signal is uniquely identified as a member variable in a struct
called Hooks
* A signal is implemented as a linked list where each node has a
reference to a slot that has been connected to the signal
* A slot is a function pointer and a unique string identifier
This differs a bit from the QT definition but i find it helpful to
describe the entities.
Important things to note:
Multiple plugins are "chained" one after the other. I.e. It is the
responsibility of the plugin author to call any plugins that follow
it
in the list. This gives the plugin authors a bit more control over
how
their plugins inter-operate with other plugins, however it would be
STRONGLY recommended that they follow a standard procedure and just
call
the next plugin after they have done their work.
Basically, the idea is to provide the following structure and then
most
of the work will involve manipulation of the linked lists. I.e.
Querying
existing items in the LL, inserting new items before/after existing
items, removing items from the LL.
This is not a proposed end product. It is just to propose an idea.
There
are a few disadvantages with the way it is implemented right now:
* Too much boilerplate code for each signal definition
* The idea of chaining calls means the responsibility of calling the
next plugin ends up with the plugin developer which could be bad if a
plugin developer does not take due care, however it also provides
them
with more flexibility (not sure if that is necessary).
Now, i have NO experience with the current pass manager in GCC, but
would the passes be able to be managed using this same framework
assuming that each pass is given a unique identifier?
Thanks,
Brendon.
#include
#include
/* GCC : Code */
struct Hooks
{
/* Define th
Re: Automatic Parallelization & Graphite - future plans
Hello everyone, In attach I included the patch Albert Cohen was referring to. Middle-end selection is performed by marking the regions of the source code, that should be compiled for an specific ISA, using pragmas such as: #pragma target Or even to reset the above by just doing: #pragma target Which enables code again to be compiled to all the ISAs. A new parameter to cc1 (-ftarget=) was created to specify which regions should be compiled. For the propose of compiling for Cell, we created a script that would be making the rule of driver, using the 2 compiled versions of cc1 (PPU/SPU) and linkind everything together with IBM tools and PPU gcc driver. Apart from the pragmas there are a few hacks in the patch that enable the output to be usable. Please realize it was done as prof of concept and so it is still unstable and outdated for current trunk. ;-) Regards, Cupertino Miranda single_source_gcc.patch Description: Binary data On Mar 18, 2009, at 11:56 PM, Albert Cohen wrote: Steven Bosscher wrote: On Wed, Mar 18, 2009 at 8:17 PM, Albert Cohen wrote: Antoniu Pop wrote: (...) The multiple backends compilation is not directly related, so you should use a separate branch. It makes sense to go in that direction. Indeed. Work has been going on for years in this direction, but it has never gone very far. There has been some work in the area, using different approaches. I've been involved in one attempt, for the Cell, with Cupertino Miranda in CC. Cupertino: could the URL where to find documentation on your experiments, and the (old) patch to GCC and the (old) Cell SDK for that purpose? What approach was taken in these experiments? Cupertino will send you the documentation and reference to the old patch sent on the gcc-patches list. In brief, there is no hotswapping, just attributes to let the MIDDLE- END decide, right before expanding to RTL, which part of the trees to keep and which to drop. Selection is done at the function level, and at the variable declaration level. You still need multiple runs of cc1, but they could be hidden behind a single run of the driver. It may not be a good idea, though, since different optimization flags may be relevant for the different backends (and even for the different functions, but this is a distinct issue). The point, for the Cell, was to perform whole-program analysis across ISA boundaries. E.g., looking at IPCP or specialization and inlining. Another example is to be able to assess the profitability of a transformation on a function that compiles to target X, but internally depends (calls) a function on target Y. You would have to assess the side-effects, cost, and pass static and dynamic profile data across the boundaries again. This was only a target direction, we did not do anything there. We struggled a lot with the data types and API that differ widely and non-consistently between the Cell PPE and SPE... as if IBM did not think until very late that single-source-multiple-backend compilation was relevant! Albert
Fixed function compilation order
Hello everyone, I am currently trying to enable GCC to perform compilation without having to respect any compilation order, i.e. execute some pass to any function at any time (it is not only pass reordering). As what I have seen from (trunk version of) GCC, it doesn't seem an easy task to be achieved and it might traduce in many complications later in development. In that sense I would like to have some opinions and suggestions from you all: - Imagining I am able to change the function context by updating cfun, etc. Will passes (all_passes) execute without missing/wrong data, i.e. is all shared data between passes accessed/ updated thought pointers such as cfun, etc. ? - What should I initialise/finalise to be able to perform the function context change ? I would like to have your opinion and tips on how to approach it and which problems I might expect to have later. Thanks in advance, Cupertino Miranda
Fwd: Fixed function compilation order
Yesterday by mistake I started some private discussion with Daniel. I will forward his answer too. Begin forwarded message: From: Cupertino Miranda <[EMAIL PROTECTED]> Date: August 9, 2007 2:24:04 AM CEDT To: Daniel Berlin <[EMAIL PROTECTED]> Subject: Re: Fixed function compilation order On Aug 9, 2007, at 2:03 AM, Daniel Berlin wrote: On 8/8/07, Cupertino Miranda <[EMAIL PROTECTED]> wrote: Hello everyone, I am currently trying to enable GCC to perform compilation without having to respect any compilation order, i.e. execute some pass to any function at any time (it is not only pass reordering). As what I have seen from (trunk version of) GCC, it doesn't seem an easy task to be achieved and it might traduce in many complications later in development. In that sense I would like to have some opinions and suggestions from you all: For tree level optimization, this is really not difficult. For RTL, this is probably going to be really hard. - Imagining I am able to change the function context by updating cfun, etc. Will passes (all_passes) execute without missing/wrong data, i.e. is all shared data between passes accessed/ updated thought pointers such as cfun, etc. ? What do you mean "missing/wrong" data. Just as an example, imagine you just executed the pass for the creation of the CFG and after that, you decided to change context to another function to perform the same pass. Will I have to keep track of this CFG or it is also available and used (by other passes) through the "high level" structures pointed by cfun, etc. ? The CFG is just an example to possible very small "detail" data- structures that might exist in GCC and not such an obvious thing as a CFG. ;-) - What should I initialise/finalise to be able to perform the function context change ? See any IPA pass I would like to have your opinion and tips on how to approach it and which problems I might expect to have later. Thanks in advance, Cupertino Miranda
Fwd: Fixed function compilation order
Here is the answer. Begin forwarded message: From: "Daniel Berlin" <[EMAIL PROTECTED]> Date: August 9, 2007 2:31:08 AM CEDT To: "Cupertino Miranda" <[EMAIL PROTECTED]> Subject: Re: Fixed function compilation order On 8/8/07, Cupertino Miranda <[EMAIL PROTECTED]> wrote: On Aug 9, 2007, at 2:03 AM, Daniel Berlin wrote: On 8/8/07, Cupertino Miranda <[EMAIL PROTECTED]> wrote: Hello everyone, I am currently trying to enable GCC to perform compilation without having to respect any compilation order, i.e. execute some pass to any function at any time (it is not only pass reordering). As what I have seen from (trunk version of) GCC, it doesn't seem an easy task to be achieved and it might traduce in many complications later in development. In that sense I would like to have some opinions and suggestions from you all: For tree level optimization, this is really not difficult. For RTL, this is probably going to be really hard. - Imagining I am able to change the function context by updating cfun, etc. Will passes (all_passes) execute without missing/wrong data, i.e. is all shared data between passes accessed/ updated thought pointers such as cfun, etc. ? What do you mean "missing/wrong" data. Just as an example, imagine you just executed the pass for the creation of the CFG and after that, you decided to change context to another function to perform the same pass. Let me be clear. We create CFG/SSA for *all* functions at once right now, run a bunch of SSA passes, and then run a bunch of passes per-function. If you try to do something before this happens, yes, you will have issues. The simple answer is "don't do that". :) So basically, you can throw your pass in the list of passes and expect changing functions to work like it does in an IPA pass. The one datastructure you cannot have stored on multiple functions at once right now are those related to tree level aliasing. We are fixing this, but it is not done yet.
Re: Announcement: GCC BPF is now being tested on BPF CI
I remind that just as bad as the decl_tags it also misses a solution to the attribute ((preserve_access_index)). Something like #pragma clang push/pop is not viable in GCC. Jose proposed the patch in: https://lore.kernel.org/bpf/20240503111836.25275-1-jose.march...@oracle.com/ Maybe you could accept his patch in the meanwhile, and work on the intended improvements later. It would be passing more tests then roughly half. Thanks Thank you for getting this up and running! Hi everyone. GCC BPF support in BPF CI has been landed. The BPF CI dashboard is here: https://github.com/kernel-patches/bpf/actions/workflows/test.yml A summary of what happens on CI (relevant to GCC BPF): * Linux Kernel is built on a target source revision * Latest snapshots of GCC 15 and binutils are downloaded * GCC BPF compiler is built and cached * selftests/bpf test runners are built with BPF_GCC variable set * BPF_GCC triggers a build of test_progs-bpf_gcc runner * The runner contains BPF binaries produced by GCC BPF * In a separate job, test_progs-bpf_gcc is executed within qemu against the target kernel GCC BPF is only tested on x86_64. On x86_64 we test the following toolchains for building the kernel and test runners: gcc-13 (ubuntu 24 default), clang-17, clang-18. An example of successful test run (you have to login to github to see the logs): https://github.com/kernel-patches/bpf/actions/runs/12816136141/job/35736973856 Currently 2513 of 4340 tests pass for GCC BPF, so a bit more than a half. Effective BPF selftests denylist for GCC BPF is located here: https://github.com/kernel-patches/vmtest/blob/master/ci/vmtest/configs/DENYLIST.test_progs-bpf_gcc When a patch is submitted to BPF, normally a corresponding PR for kernel-patches/bpf github repo is automatically created to trigger a BPF CI run for this change. PRs opened manually will do that too, and this can be used to test patches before submission. Since the CI automatically pulls latest GCC snapshot, a change in GCC can potentially cause CI failures unrelated to Linux changes being tested. This is not the only dependency like that, of course. In such situations, a change is usually made in CI code to mitigate the failure in order to unblock the pipeline for patches. If that happens with GCC, someone (most likely me) will have to reach out to GCC team. I guess gcc@gcc.gnu.org would be the default point of contact, but if there are specific people who should be notified please let me know.
