Re: GSOC Proposal

[Richard Biener]

On Sun, 7 Apr 2019, nick wrote:

> 
> 
> On 2019-04-07 5:31 a.m., Richard Biener wrote:
> > On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
> >>
> >>
> >> On 2019-04-05 6:25 a.m., Richard Biener wrote:
> >>> On Wed, 3 Apr 2019, nick wrote:
> >>>
> 
> 
>  On 2019-04-03 7:30 a.m., Richard Biener wrote:
> > On Mon, 1 Apr 2019, nick wrote:
> >
> >>
> >>
> >> On 2019-04-01 9:47 a.m., Richard Biener wrote:
> >>> On Mon, 1 Apr 2019, nick wrote:
> >>>
>  Well I'm talking about the shared roots of this garbage
> >> collector core state 
>  data structure or just struct ggc_root_tab.
> 
>  But also this seems that this to be no longer shared globally if
> >> I'm not mistaken 
>  or this:
>  static vec extra_root_vec;
> 
>  Not sure after reading the code which is a bigger deal through
> >> so I wrote
>  my proposal not just asking which is a better issue for not
> >> being thread
>  safe. Sorry about that.
> 
>  As for the second question injection seems to not be the issue
> >> or outside
>  callers but just internal so phase 3 or step 3 would now be:
>  Find internal callers or users of x where x is one of the above
> >> rather
>  than injecting outside callers. Which answers my second question
> >> about
>  external callers being a issue still.
> 
>  Let me know which  of the two is a better issue:
>  1. struct ggc_root_tabs being shared
>  2.static vec extra_root_vec; as a shared
> >> heap or
>  vector of root nodes for each type of allocation
> 
>  and I will gladly rewrite my proposal sections for that
>  as needs to be reedited.
> >>>
> >>> I don't think working on the garbage collector as a separate
> >>> GSoC project is useful at this point.  Doing locking around
> >>> allocation seems like a good short-term solution and if that
> >>> turns out to be a performance issue for the threaded part
> >>> using per-thread freelists is likely an easy to deploy
> >>> solution.
> >>>
> >>> Richard.
> >>>
> >> I agree but we were discussing this:
> >> Or maybe a project to be more
> >> explicit about regions of the code that assume that the garbage-
> >> collector can't run within them?[3] (since the GC is state that
> >> would
> >> be shared by the threads).
> >
> > The process of collecting garbage is not the only issue (and that
> > very issue is easiest mitigated by collecting only at specific
> > points - which is what we do - and have those be serializing
> >> points).
> > The main issue is the underlying memory allocator (GCC uses memory
> > that is garbage collected plus regular heap memory).
> >
> >> In addition I moved my paper back to our discussion about garbage
> >> collector
> >> state with outside callers.Seems we really need to do something
> >> about
> >> my wording as the idea of my project in a nutshell was to figure
> >> out how to mark shared state by callers and inject it into the
> >> garbage collector letting it known that the state was not shared
> >> between
> >> threads or shared. Seems that was on the GSoc page and in our
> >> discussions the issue
> >> is marking outside code for shared state. If that's correct then
> >> my
> >> wording of outside callers is incorrect it should have been shared
> >> state between threads on outside callers to the garbage collector.
> >> If the state is that in your wording above then great as I
> >> understand
> >> where we are going and will gladly change my wording.
> >
> > I'm still not sure what you are shooting at, the above sentences do
> > not make any sense to me.
> >
> >> Also freelists don't work here as the state is shared at the
> >> caller's 
> >> end which would need two major issues:
> >> 1. Locking on nodes of the 
> >> freelists when two threads allocate at the same thing which can be
> >> a 
> >> problem if the shared state is shared a lot
> >> 2. Locking allocation with 
> >> large numbers of callers can starve threads
> >
> > First of all allocating memory from the GC pool is not the main
> > work of GIMPLE passes so simply serializing at allocation time
> >> might
> > work out.  Second free lists of course do work.  What you'd do is
> > have a fast path in allocation using a thread-local "free list"
> > which you can allocate from without taking any lock.  Maybe I
> >> should
> > explain "free list" since that term doesn't make too much sense in
> > a garbage collector world.  What I'd do is when a client thread
> > asks for memory of size N allocate M objects of that size but put
> > M - 1 on the client thread local "free list" to be allocated
> >> lock-free
> > from for the next M - 1 ca

New Documents For(gcc@gcc.gnu.org)Your Shipment (Shipping SF electronic invoice notification)

[SF Express]

   NB2r31v84�7AN30�1F�D0[A2b37�1A
 NB2 gcc@gcc.gnu.org
 
a1F�22`A8�09bE9�7AN30�1F�D0cD0O9Bv84�01�27g0DRA1002`A8u33�F7v84u35[50SD1y68]F2b10R9FSD1QFA�01SD1y68�E6`C5Y82N0B�1A
 1,44811594346; 2,44811504355; 3,44811504364; 4,81029652812; 5,811436742647;
 `A8SEFNE5pB9QFBNE5N0B�FEcA5N0B�7Du35[50SD1y68
 W28k64N0B�7DSD1y68

Re: Putting an all-zero variable into BSS

[Andrew Haley]

On 4/7/19 5:03 PM, Thomas Koenig wrote:
> Hi Richard,
> 
>> I don't know without looking, but I'd start at assemble_variable in varasm.c.
> 
> Thanks.  I've done that, and this is what a patch could look like.
> However, I will not have time to formally submit this until next
> weekend.
> 
> In the meantime, comments are still welcome :-)

Did you look at
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83100

This was the change that caused this behaviour.

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: non-volatile automatic variables in setjmp tests

[Richard Biener]

On Fri, Apr 5, 2019 at 6:25 PM Michael Matz  wrote:
>
> Hello,
>
> On Fri, 5 Apr 2019, Jozef Lawrynowicz wrote:
>
> > Some setjmp/longjmp tests[1] depend on the value of an auto set before 
> > setjmp
> > to to be retained after returning from the longjmp. As I understand, this
> > behaviour is actually undefined, according to the gccint manual.
> >
> > Section 3 "Interfacing to GCC Output" of gccint says:
> >   If you use longjmp, beware of automatic variables. ISO C says that 
> > automatic
> >   variables that are not declared volatile have undefined values after a
> >   longjmp. And this is all GCC promises to do, because it is very difficult 
> > to
> >   restore register variables correctly, and one of GCC’s features is that it
> >   can put variables in registers without your asking it to.
>
> That is very old text, from 1997, and doesn't reflect what GCC actually
> does, which is ...
>
> > However, ISO C says [2]:
> >   ... values of objects of automatic storage duration are unspecified if 
> > they
> >   meet all the following conditions:
> >   * They are local to the function containing the corresponding setjmp()
> > invocation.
> >   * They do not have volatile-qualified type.
> >   * They are changed between the setjmp() invocation and longjmp() call.
>
> ... supporting this (and in any case if there's a direct conflict between
> the GCC docu and a relevant standard, then the former is likely the wrong
> one).  The are two modi: (a) the target has a setjmp/longjmp combination
> which restores all callee-saved registers (to the values at just before
> the setjmp call) and (b) the target doesn't save all these.
>
> For (a) GCC treats the setjmp call as a normal function call from a
> register-clobber perspective, so any auto variable live over the
> setjmp that is stored in a callee-saved register is restored to the
> pre-setjmp value.
>
> For (b) GCC makes sure to not allocate variables live over setjmp to
> registers.  This is the conservative assumption.
>
> Most targets in GCC follow the (b) model, even though the specific
> setjmp/longjmp mechanism would allow for (a).
>
> > gcc.dg/torture/stackalign/setjmp-1.c and
> > gcc.c-torture/execute/built-in-setjmp.c actually fail at execution for
> > msp430-elf @ -O1, because the auto being tested after the longjmp has not 
> > been
> > restored correctly.
>
> So the testcases should indeed work without volatile and you need to
> investigate why the restoring doesn't happen correctly.

There's one known "hole" in that the abnormal edges created to support all
of the above on GIMPLE are thrown away at RTL expansion time and
expected to be re-created "properly" but that doesn't actually happen.

The proper fix is to not throw them away but carry them over (with the
additional complication that you have to deal with that
find-many-sub-basic-block
case in some way).

Not sure if in this case we run into an RTL optimization that breaks things
(PRE / scheduling / invariant motion are candidates).

> > These tests feature a contrived way of making the auto appear used up to the
> > longjmp, so I wonder if they have been written deliberately without a
> > volatile, or if it was just an oversight.
> >
> > For msp430-elf @ -O1, this code to make the auto appear used is
> > ineffective as an optimization replaces the variable with a constant,
> > which is what triggers the test failure.
>
> Well, then it should be the correct constant, and hence be unaffected by
> setjmp/longjmp, so if that still doesn't work the constant is wrong,
> right?
>
> > Are these tests that rely on the value of a non-volatile auto after a
> > longjmp invalid? I can patch them to make the auto variable a global
> > instead.
>
> No, they should work as is.
>
>
> Ciao,
> Michael.

Re: is re-running bootstrap after a change safe?

[Richard Biener]

On Sat, Apr 6, 2019 at 1:09 AM Martin Sebor  wrote:
>
> On 4/5/19 4:02 PM, Jeff Law wrote:
> > On 4/5/19 3:37 PM, Martin Sebor wrote:
> >> On 4/5/19 3:29 PM, Jeff Law wrote:
> >>> On 4/5/19 2:50 PM, Eric Botcazou wrote:
> > Say if the first bootstrap succeeds and I then change a single
> > GCC .c file and rerun make bootstrap, am I guaranteed to see
> > the same fallout of the change as I would if I did a pristine
> > build in a clean directory?
> 
>  No, this would imply deleting the stage2 and stage3 compilers and
>  that isn't
>  what happens.  Instead the compiler of each stage is updated in
>  isolation.
> 
> >>> RIght.  Thus I always blow away stage2-* stage3-*, and stage1 target
> >>> directories along with the "compare" stamp file.
> >>
> >> Thanks (all of you).  It's amazing that I have been getting away
> >> with it for all these years.
> > I got away without removing the "compare" stamp file for a long time,
> > then broke the trunk with a comparison failure :(
> >
> >>
> >> Why is this not done automatically?  I mean, what is the use case
> >> for make bootstrap without doing these steps first?
> > During development folks often want to rebuild without going through a
> > full bootstrap.  Obviously for testing the final version of a patch the
> > "quick" approach of just rebuilding without blowing away the stage
> > directories isn't sufficient.
>
> I see.  So after a bootstrap and a subsequent change to a .c file,
> at each stage the next bootstrap recompiles just the changed file
> and relinks gcc.  It doesn't actually recompile all source files
> in stage 2 or 3 with the changed compiler from the last stage.
> That's why it's so much faster!  Make check then correctly reflects
> the change but the compiler doesn't get as fully exercised because
> the test suite has low coverage.
>
> > This is actually one of the things I'd really like to just automate.
> > You point to a git commit in a public repo, the tester picks it up and
> > does a bootstrap & regression test from scratch on whatever targets you
> > ask for.
>
> That would be great to validate the final patch.
>
> So to be clear: the safe and also most efficient to "rebootstrap"
> GCC is to remove what exactly?

Just remove everything?  Toplevel configure and stage1 build is
fast anyway.

That's what I'm doing since forever.

Richard.

>  (I don't see any stage2 or stage3
> directories in my build tree.)  Is there a make target for this?
>
> Thanks
> Martin

Re: GSOC

[Richard Biener]

On Sun, Apr 7, 2019 at 11:10 AM ashwina kumar  wrote:
>
> Hi ,
>
> While working I just figured out that -Wconversion is buggy. Please see the
> below code- -
>
> $ cat b.c
> #include 
>
> void main (void)
> {
> //contains build errors
> uint16_t x = 1;
> uint16_t y = 2;
> y += x;  /* contains error */
>
> }
>
> $ gcc b.c -Wconversion
> b.c: In function ‘main’:
> b.c:22:7: warning: conversion to ‘uint16_t {aka short unsigned int}’
> from ‘int’ may alter its value [-Wconversion]
>   y += x;  /* contains error */

The warning is correct unless you factor in that x == 1 and y == 2.

> Please help me to know as an GSOC student can I work on this for this year
> to make -Wconversion more robust.
>
> Thanks & Regards,
> Ashwina
>
> --
> Ashwina Kumar
> BIT Mesra

Re: Putting an all-zero variable into BSS

[Richard Biener]

On Mon, Apr 8, 2019 at 10:38 AM Andrew Haley  wrote:
>
> On 4/7/19 5:03 PM, Thomas Koenig wrote:
> > Hi Richard,
> >
> >> I don't know without looking, but I'd start at assemble_variable in 
> >> varasm.c.
> >
> > Thanks.  I've done that, and this is what a patch could look like.
> > However, I will not have time to formally submit this until next
> > weekend.
> >
> > In the meantime, comments are still welcome :-)
>
> Did you look at
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83100
>
> This was the change that caused this behaviour.

Actually that just changed the behavior for DECL_COMMONs
which may of course match the fortran case here in case you
bisected this.  OTOH DECL_COMMONs are tentative and
do not have an initializer (and not go to .rodata either).

Richard.

>
> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. 
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Putting an all-zero variable into BSS

[Richard Biener]

On Mon, Apr 8, 2019 at 11:33 AM Richard Biener
 wrote:
>
> On Mon, Apr 8, 2019 at 10:38 AM Andrew Haley  wrote:
> >
> > On 4/7/19 5:03 PM, Thomas Koenig wrote:
> > > Hi Richard,
> > >
> > >> I don't know without looking, but I'd start at assemble_variable in 
> > >> varasm.c.
> > >
> > > Thanks.  I've done that, and this is what a patch could look like.
> > > However, I will not have time to formally submit this until next
> > > weekend.
> > >
> > > In the meantime, comments are still welcome :-)
> >
> > Did you look at
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83100
> >
> > This was the change that caused this behaviour.
>
> Actually that just changed the behavior for DECL_COMMONs
> which may of course match the fortran case here in case you
> bisected this.  OTOH DECL_COMMONs are tentative and
> do not have an initializer (and not go to .rodata either).

That is, the C testcase

const char x[1024*1024] = {};

reproduces the "issue".  The comment in bss_initializer_p though
explicitely says

  /* Do not put non-common constants into the .bss section, they belong in
 a readonly section, except when NAMED is true.  */
  return ((!TREE_READONLY (decl) || DECL_COMMON (decl) || named)

(where named refers to explicit .bss section marked decls).  Note
the docs for -fzero-initialized-in-bss doesn't mention that this doesn't
apply for readonly variables.

Richard.

> Richard.
>
> >
> > --
> > Andrew Haley
> > Java Platform Lead Engineer
> > Red Hat UK Ltd. 
> > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: GSOC

[Martin Jambor]

Hello,

On Sun, Apr 07 2019, ashwina kumar wrote:
> Hi ,
>
> While working I just figured out that -Wconversion is buggy. Please see the
> below code- -
>
> $ cat b.c
> #include 
>
> void main (void)
> {
> //contains build errors
> uint16_t x = 1;
> uint16_t y = 2;
> y += x;  /* contains error */
>
> }
>
> $ gcc b.c -Wconversion
> b.c: In function ‘main’:
> b.c:22:7: warning: conversion to ‘uint16_t {aka short unsigned int}’
> from ‘int’ may alter its value [-Wconversion]
>   y += x;  /* contains error */
>
> Please help me to know as an GSOC student can I work on this for this year
> to make -Wconversion more robust.

Unfortunately I am not quite sure what you think is a problem.  The
option -Werror warns "for implicit conversions that may alter a value."
In this case, the C language mandates that the addition is performed in
full integer type, which is then stored to a shorter type, which is a
conversion which may alter the value.  Thus the warning.

In this particular example the values are known constants and so one
could argue that the result of addition is known at compile not to
exceed the uint16_t range.  However, the infrastructure that would have
to be added to the C/C++ front-ends to perform such value tracking would
not be justified with this simple use-case - especially given that the
warning should be perfectly silence-able with an explicit conversion.

Therefore, we are very unlikely to accept such GSoC proposal, sorry.

Martin

Re: non-volatile automatic variables in setjmp tests

[Michael Matz]

Hi,

On Mon, 8 Apr 2019, Richard Biener wrote:

> Not sure if in this case we run into an RTL optimization that breaks things
> (PRE / scheduling / invariant motion are candidates).

That's true, what Josef sees might point to a genuine bug in the 
middle-end observed only on msp430; but we do want to make this situation 
work generally, as required by ISO C, not like how it's spelled in our 
manual.


Ciao,
Michael.

Re: non-volatile automatic variables in setjmp tests

[Richard Biener]

On Mon, Apr 8, 2019 at 2:31 PM Michael Matz  wrote:
>
> Hi,
>
> On Mon, 8 Apr 2019, Richard Biener wrote:
>
> > Not sure if in this case we run into an RTL optimization that breaks things
> > (PRE / scheduling / invariant motion are candidates).
>
> That's true, what Josef sees might point to a genuine bug in the
> middle-end observed only on msp430; but we do want to make this situation
> work generally, as required by ISO C, not like how it's spelled in our
> manual.

Yes, and there's at least one existing bug, PR57067 for which it was
observed the scheduler genrates wrong-code.

Richard.

>
> Ciao,
> Michael.

Re: is re-running bootstrap after a change safe?

[Jeff Law]

On 4/8/19 3:19 AM, Richard Biener wrote:
> On Sat, Apr 6, 2019 at 1:09 AM Martin Sebor  wrote:
>>
>> On 4/5/19 4:02 PM, Jeff Law wrote:
>>> On 4/5/19 3:37 PM, Martin Sebor wrote:
 On 4/5/19 3:29 PM, Jeff Law wrote:
> On 4/5/19 2:50 PM, Eric Botcazou wrote:
>>> Say if the first bootstrap succeeds and I then change a single
>>> GCC .c file and rerun make bootstrap, am I guaranteed to see
>>> the same fallout of the change as I would if I did a pristine
>>> build in a clean directory?
>>
>> No, this would imply deleting the stage2 and stage3 compilers and
>> that isn't
>> what happens.  Instead the compiler of each stage is updated in
>> isolation.
>>
> RIght.  Thus I always blow away stage2-* stage3-*, and stage1 target
> directories along with the "compare" stamp file.

 Thanks (all of you).  It's amazing that I have been getting away
 with it for all these years.
>>> I got away without removing the "compare" stamp file for a long time,
>>> then broke the trunk with a comparison failure :(
>>>

 Why is this not done automatically?  I mean, what is the use case
 for make bootstrap without doing these steps first?
>>> During development folks often want to rebuild without going through a
>>> full bootstrap.  Obviously for testing the final version of a patch the
>>> "quick" approach of just rebuilding without blowing away the stage
>>> directories isn't sufficient.
>>
>> I see.  So after a bootstrap and a subsequent change to a .c file,
>> at each stage the next bootstrap recompiles just the changed file
>> and relinks gcc.  It doesn't actually recompile all source files
>> in stage 2 or 3 with the changed compiler from the last stage.
>> That's why it's so much faster!  Make check then correctly reflects
>> the change but the compiler doesn't get as fully exercised because
>> the test suite has low coverage.
>>
>>> This is actually one of the things I'd really like to just automate.
>>> You point to a git commit in a public repo, the tester picks it up and
>>> does a bootstrap & regression test from scratch on whatever targets you
>>> ask for.
>>
>> That would be great to validate the final patch.
>>
>> So to be clear: the safe and also most efficient to "rebootstrap"
>> GCC is to remove what exactly?
> 
> Just remove everything?  Toplevel configure and stage1 build is
> fast anyway.
> 
> That's what I'm doing since forever.
I suspect it's very fast if ccache is set up :-)  What's left for the
stage1 build would be all the configury gunk.

jeff

Re: non-volatile automatic variables in setjmp tests

[Jozef Lawrynowicz]

On Mon, 8 Apr 2019 14:45:17 +0200
Richard Biener  wrote:

> On Mon, Apr 8, 2019 at 2:31 PM Michael Matz  wrote:
> >
> > Hi,
> >
> > On Mon, 8 Apr 2019, Richard Biener wrote:
> >  
> > > Not sure if in this case we run into an RTL optimization that breaks 
> > > things
> > > (PRE / scheduling / invariant motion are candidates).  
> >
> > That's true, what Josef sees might point to a genuine bug in the
> > middle-end observed only on msp430; but we do want to make this situation
> > work generally, as required by ISO C, not like how it's spelled in our
> > manual.  
> 
> Yes, and there's at least one existing bug, PR57067 for which it was
> observed the scheduler genrates wrong-code.
> 
> Richard.
> 
> >
> > Ciao,
> > Michael.  

Thank you both for the information.

I'm still digging to find the cause of the failure, but I think it's a bug in
reload rather than an earlier pass having issues with setjmp/longjmp. The RTL
up to reload looks good to me. Also when I managed to coerce a build of GCC to
complete with LRA enabled for msp430, the test passed.

It was r255136 that exposed the failure by replacing usage of a variable with
a constant. Taking that optimization into account I'm now trying to find
which earlier commit exposed/caused the failure, as the test works with
r247017 when GCC8 development started on trunk. But that might just be because
the RTL when we get to reload is different, and doesn't expose the bug.

I'll file a bugzilla once I have more concrete details.

Jozef

Re: GSOC Proposal

[nick]

On 2019-04-08 3:29 a.m., Richard Biener wrote:
> On Sun, 7 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-07 5:31 a.m., Richard Biener wrote:
>>> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:


 On 2019-04-05 6:25 a.m., Richard Biener wrote:
> On Wed, 3 Apr 2019, nick wrote:
>
>>
>>
>> On 2019-04-03 7:30 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>


 On 2019-04-01 9:47 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
>
>> Well I'm talking about the shared roots of this garbage
 collector core state 
>> data structure or just struct ggc_root_tab.
>>
>> But also this seems that this to be no longer shared globally if
 I'm not mistaken 
>> or this:
>> static vec extra_root_vec;
>>
>> Not sure after reading the code which is a bigger deal through
 so I wrote
>> my proposal not just asking which is a better issue for not
 being thread
>> safe. Sorry about that.
>>
>> As for the second question injection seems to not be the issue
 or outside
>> callers but just internal so phase 3 or step 3 would now be:
>> Find internal callers or users of x where x is one of the above
 rather
>> than injecting outside callers. Which answers my second question
 about
>> external callers being a issue still.
>>
>> Let me know which  of the two is a better issue:
>> 1. struct ggc_root_tabs being shared
>> 2.static vec extra_root_vec; as a shared
 heap or
>> vector of root nodes for each type of allocation
>>
>> and I will gladly rewrite my proposal sections for that
>> as needs to be reedited.
>
> I don't think working on the garbage collector as a separate
> GSoC project is useful at this point.  Doing locking around
> allocation seems like a good short-term solution and if that
> turns out to be a performance issue for the threaded part
> using per-thread freelists is likely an easy to deploy
> solution.
>
> Richard.
>
 I agree but we were discussing this:
 Or maybe a project to be more
 explicit about regions of the code that assume that the garbage-
 collector can't run within them?[3] (since the GC is state that
 would
 be shared by the threads).
>>>
>>> The process of collecting garbage is not the only issue (and that
>>> very issue is easiest mitigated by collecting only at specific
>>> points - which is what we do - and have those be serializing
 points).
>>> The main issue is the underlying memory allocator (GCC uses memory
>>> that is garbage collected plus regular heap memory).
>>>
 In addition I moved my paper back to our discussion about garbage
 collector
 state with outside callers.Seems we really need to do something
 about
 my wording as the idea of my project in a nutshell was to figure
 out how to mark shared state by callers and inject it into the
 garbage collector letting it known that the state was not shared
 between
 threads or shared. Seems that was on the GSoc page and in our
 discussions the issue
 is marking outside code for shared state. If that's correct then
 my
 wording of outside callers is incorrect it should have been shared
 state between threads on outside callers to the garbage collector.
 If the state is that in your wording above then great as I
 understand
 where we are going and will gladly change my wording.
>>>
>>> I'm still not sure what you are shooting at, the above sentences do
>>> not make any sense to me.
>>>
 Also freelists don't work here as the state is shared at the
 caller's 
 end which would need two major issues:
 1. Locking on nodes of the 
 freelists when two threads allocate at the same thing which can be
 a 
 problem if the shared state is shared a lot
 2. Locking allocation with 
 large numbers of callers can starve threads
>>>
>>> First of all allocating memory from the GC pool is not the main
>>> work of GIMPLE passes so simply serializing at allocation time
 might
>>> work out.  Second free lists of course do work.  What you'd do is
>>> have a fast path in allocation using a thread-local "free list"
>>> which you can allocate from without taking any lock.  Maybe I
 should
>>> explain "free list" since that term doesn't make too much sense in
>>> a garbage collector world.  What I'd do is when a client thread
>>> asks for memory of size N allocate M objects of that size but put
>>> M - 1 on the client thread local "free list" to be allo

Re: GSOC Proposal

[Richard Biener]

On Mon, 8 Apr 2019, nick wrote:

> 
> 
> On 2019-04-08 3:29 a.m., Richard Biener wrote:
> > On Sun, 7 Apr 2019, nick wrote:
> > 
> >>
> >>
> >> On 2019-04-07 5:31 a.m., Richard Biener wrote:
> >>> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
> 
> 
>  On 2019-04-05 6:25 a.m., Richard Biener wrote:
> > On Wed, 3 Apr 2019, nick wrote:
> >
> >>
> >>
> >> On 2019-04-03 7:30 a.m., Richard Biener wrote:
> >>> On Mon, 1 Apr 2019, nick wrote:
> >>>
> 
> 
>  On 2019-04-01 9:47 a.m., Richard Biener wrote:
> > On Mon, 1 Apr 2019, nick wrote:
> >
> >> Well I'm talking about the shared roots of this garbage
>  collector core state 
> >> data structure or just struct ggc_root_tab.
> >>
> >> But also this seems that this to be no longer shared globally if
>  I'm not mistaken 
> >> or this:
> >> static vec extra_root_vec;
> >>
> >> Not sure after reading the code which is a bigger deal through
>  so I wrote
> >> my proposal not just asking which is a better issue for not
>  being thread
> >> safe. Sorry about that.
> >>
> >> As for the second question injection seems to not be the issue
>  or outside
> >> callers but just internal so phase 3 or step 3 would now be:
> >> Find internal callers or users of x where x is one of the above
>  rather
> >> than injecting outside callers. Which answers my second question
>  about
> >> external callers being a issue still.
> >>
> >> Let me know which  of the two is a better issue:
> >> 1. struct ggc_root_tabs being shared
> >> 2.static vec extra_root_vec; as a shared
>  heap or
> >> vector of root nodes for each type of allocation
> >>
> >> and I will gladly rewrite my proposal sections for that
> >> as needs to be reedited.
> >
> > I don't think working on the garbage collector as a separate
> > GSoC project is useful at this point.  Doing locking around
> > allocation seems like a good short-term solution and if that
> > turns out to be a performance issue for the threaded part
> > using per-thread freelists is likely an easy to deploy
> > solution.
> >
> > Richard.
> >
>  I agree but we were discussing this:
>  Or maybe a project to be more
>  explicit about regions of the code that assume that the garbage-
>  collector can't run within them?[3] (since the GC is state that
>  would
>  be shared by the threads).
> >>>
> >>> The process of collecting garbage is not the only issue (and that
> >>> very issue is easiest mitigated by collecting only at specific
> >>> points - which is what we do - and have those be serializing
>  points).
> >>> The main issue is the underlying memory allocator (GCC uses memory
> >>> that is garbage collected plus regular heap memory).
> >>>
>  In addition I moved my paper back to our discussion about garbage
>  collector
>  state with outside callers.Seems we really need to do something
>  about
>  my wording as the idea of my project in a nutshell was to figure
>  out how to mark shared state by callers and inject it into the
>  garbage collector letting it known that the state was not shared
>  between
>  threads or shared. Seems that was on the GSoc page and in our
>  discussions the issue
>  is marking outside code for shared state. If that's correct then
>  my
>  wording of outside callers is incorrect it should have been shared
>  state between threads on outside callers to the garbage collector.
>  If the state is that in your wording above then great as I
>  understand
>  where we are going and will gladly change my wording.
> >>>
> >>> I'm still not sure what you are shooting at, the above sentences do
> >>> not make any sense to me.
> >>>
>  Also freelists don't work here as the state is shared at the
>  caller's 
>  end which would need two major issues:
>  1. Locking on nodes of the 
>  freelists when two threads allocate at the same thing which can be
>  a 
>  problem if the shared state is shared a lot
>  2. Locking allocation with 
>  large numbers of callers can starve threads
> >>>
> >>> First of all allocating memory from the GC pool is not the main
> >>> work of GIMPLE passes so simply serializing at allocation time
>  might
> >>> work out.  Second free lists of course do work.  What you'd do is
> >>> have a fast path in allocation using a thread-local "free list"
> >>> which you can allocate from without taking any lock.  Maybe I
>  should
> >>> explai

Re: GSoC OMPD

[Jakub Jelinek]

On Thu, Apr 04, 2019 at 02:24:09PM +0200, Martin Jambor wrote:
> > I know my first email is vague. I wanted to throw it out there since
> > the April 9th deadline is coming up.
> 
> I was hoping Jakub Jelinek, who would be the mentor, would chime in
> earlier.  But unfortunately he has probably not been online in the past
> few days.  (And I admit I struggle a little bit to answer all GSoC email
> in a timely manner this week too).

Sorry, I've been on vacation last week.

> >>Also what would the goals and timeline look like?
> 
> For this, especially the timeline, we probably need Jakub's input.  The
> goal would be to implement a substantial portion of the spec, but I
> cannot quickly provide any guidance about how time consuming different
> parts are likely to be.

It doesn't have to be a full OMPD spec implementation in the end, a major
portion thereof might be enough, it needs to be usable, on the other side
doesn't necessarily mean also writing GDB support to use that; it would be
nice to be in touch with some GDB folks (CCed Kevin) and discuss the
implementation with them too.
Likely no changes on the compiler side should be needed, as few changes in
libgomp.so.1 as possible (the goal is not to slow down the runtime if
possible, if not possible at least minimize the slow down and if possible
make it conditional on the process being actually OMPD debugged) and most of
it would be writing a new shared library with the OMPD APIs that a debugger
or similar tool can dlopen and use.  Ideally the OMPD shared library would
use https://infinitynotes.org/wiki/Infinity to communicate structure offsets
and how to get various properties from the libgomp.so.1 shared library, but
if it turns out to be too hard for a start, doing it some other way
initially might be acceptable too.  The implementation should also add
 and  headers, similar to the sample ones in
https://github.com/OpenMP/sources/tree/master/include
and actually implement the APIs in there.

Jakub

Re: GSOC Proposal

[nick]

On 2019-04-08 9:42 a.m., Richard Biener wrote:
> On Mon, 8 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-08 3:29 a.m., Richard Biener wrote:
>>> On Sun, 7 Apr 2019, nick wrote:
>>>


 On 2019-04-07 5:31 a.m., Richard Biener wrote:
> On April 5, 2019 6:11:15 PM GMT+02:00, nick  wrote:
>>
>>
>> On 2019-04-05 6:25 a.m., Richard Biener wrote:
>>> On Wed, 3 Apr 2019, nick wrote:
>>>


 On 2019-04-03 7:30 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
>
>>
>>
>> On 2019-04-01 9:47 a.m., Richard Biener wrote:
>>> On Mon, 1 Apr 2019, nick wrote:
>>>
 Well I'm talking about the shared roots of this garbage
>> collector core state 
 data structure or just struct ggc_root_tab.

 But also this seems that this to be no longer shared globally if
>> I'm not mistaken 
 or this:
 static vec extra_root_vec;

 Not sure after reading the code which is a bigger deal through
>> so I wrote
 my proposal not just asking which is a better issue for not
>> being thread
 safe. Sorry about that.

 As for the second question injection seems to not be the issue
>> or outside
 callers but just internal so phase 3 or step 3 would now be:
 Find internal callers or users of x where x is one of the above
>> rather
 than injecting outside callers. Which answers my second question
>> about
 external callers being a issue still.

 Let me know which  of the two is a better issue:
 1. struct ggc_root_tabs being shared
 2.static vec extra_root_vec; as a shared
>> heap or
 vector of root nodes for each type of allocation

 and I will gladly rewrite my proposal sections for that
 as needs to be reedited.
>>>
>>> I don't think working on the garbage collector as a separate
>>> GSoC project is useful at this point.  Doing locking around
>>> allocation seems like a good short-term solution and if that
>>> turns out to be a performance issue for the threaded part
>>> using per-thread freelists is likely an easy to deploy
>>> solution.
>>>
>>> Richard.
>>>
>> I agree but we were discussing this:
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that
>> would
>> be shared by the threads).
>
> The process of collecting garbage is not the only issue (and that
> very issue is easiest mitigated by collecting only at specific
> points - which is what we do - and have those be serializing
>> points).
> The main issue is the underlying memory allocator (GCC uses memory
> that is garbage collected plus regular heap memory).
>
>> In addition I moved my paper back to our discussion about garbage
>> collector
>> state with outside callers.Seems we really need to do something
>> about
>> my wording as the idea of my project in a nutshell was to figure
>> out how to mark shared state by callers and inject it into the
>> garbage collector letting it known that the state was not shared
>> between
>> threads or shared. Seems that was on the GSoc page and in our
>> discussions the issue
>> is marking outside code for shared state. If that's correct then
>> my
>> wording of outside callers is incorrect it should have been shared
>> state between threads on outside callers to the garbage collector.
>> If the state is that in your wording above then great as I
>> understand
>> where we are going and will gladly change my wording.
>
> I'm still not sure what you are shooting at, the above sentences do
> not make any sense to me.
>
>> Also freelists don't work here as the state is shared at the
>> caller's 
>> end which would need two major issues:
>> 1. Locking on nodes of the 
>> freelists when two threads allocate at the same thing which can be
>> a 
>> problem if the shared state is shared a lot
>> 2. Locking allocation with 
>> large numbers of callers can starve threads
>
> First of all allocating memory from the GC pool is not the main
> work of GIMPLE passes so simply serializing at allocation time
>> might
> work out.  Second free lists of course do work.  What you'd do is
> have a fast path in allocation using a thread-local "free list"
> which you can allocate from without ta