Hi,
See comments inline.
On 12/13, Bin.Cheng wrote:
> On Wed, Dec 12, 2018 at 11:46 PM Giuliano Augusto Faulin Belinassi
> wrote:
> >
> > Hi, I have some news. :-)
> >
> > I replicated the Martin Liška experiment [1] on a 64-cores machine for
> > gcc [2] and Linux kernel [3] (Linux kernel was fully parallelized),
> > and I am excited to dive into this problem. As a result, I want to
> > propose GSoC project on this issue, starting with something like:
> > 1- Systematically create a benchmark for easily information
> > gathering. Martin Liška already made the first version of it, but I
> > need to improve it.
> > 2- Find and document the global states (Try to reduce the gcc's
> > global states as well).
> > 3- Define the parallelization strategy.
> > 4- First parallelization attempt.
> Hi Giuliano,
>
> Thanks very much for working on this. It could be very useful, for
> example, one bottleneck we have is slow compilation of big single
> source file after intensively using distribution compilation. Of
> course, a good parallelization strategy is needed.
>
Interesting. How many lines the generated file has? Does it uses C++
templates?
The generated gimple-match.c file, for example, has 98786 lines and
takes about 30s to compile.
> Thanks,
> bin
> >
> > I also proposed this issue as a research project to my advisor and he
> > supported me on this idea. So I can work for at least one year on
> > this, and other things related to it.
> >
> > Would anyone be willing to mentor me on this?
> >
> > [1] https://gcc.gnu.org/bugzilla/attachment.cgi?id=43440
> > [2] https://www.ime.usp.br/~belinass/64cores-experiment.svg
> > [3] https://www.ime.usp.br/~belinass/64cores-kernel-experiment.svg
> > On Mon, Nov 19, 2018 at 8:53 AM Richard Biener
> > wrote:
> > >
> > > On Fri, Nov 16, 2018 at 8:00 PM Giuliano Augusto Faulin Belinassi
> > > wrote:
> > > >
> > > > Hi! Sorry for the late reply again :P
> > > >
> > > > On Thu, Nov 15, 2018 at 8:29 AM Richard Biener
> > > > wrote:
> > > > >
> > > > > On Wed, Nov 14, 2018 at 10:47 PM Giuliano Augusto Faulin Belinassi
> > > > > wrote:
> > > > > >
> > > > > > As a brief introduction, I am a graduate student that got interested
> > > > > >
> > > > > > in the "Parallelize the compilation using threads"(GSoC 2018 [1]). I
> > > > > > am a newcommer in GCC, but already have sent some patches, some of
> > > > > > them have already been accepted [2].
> > > > > >
> > > > > > I brought this subject up in IRC, but maybe here is a proper place
> > > > > > to
> > > > > > discuss this topic.
> > > > > >
> > > > > > From my point of view, parallelizing GCC itself will only speed up
> > > > > > the
> > > > > > compilation of projects which have a big file that creates a
> > > > > > bottleneck in the whole project compilation (note: by big, I mean
> > > > > > the
> > > > > > amount of code to generate).
> > > > >
> > > > > That's true. During GCC bootstrap there are some of those (see
> > > > > PR84402).
> > > > >
> > > >
> > > > > One way to improve parallelism is to use link-time optimization where
> > > > > even single source files can be split up into multiple link-time
> > > > > units. But
> > > > > then there's the serial whole-program analysis part.
> > > >
> > > > Did you mean this: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84402 ?
> > > > That is a lot of data :-)
> > > >
> > > > It seems that 'phase opt and generate' is the most time-consuming
> > > > part. Is that the 'GIMPLE optimization pipeline' you were talking
> > > > about in this thread:
> > > > https://gcc.gnu.org/ml/gcc/2018-03/msg00202.html
> > >
> > > It's everything that comes after the frontend parsing bits, thus this
> > > includes in particular RTL optimization and early GIMPLE optimizations.
> > >
> > > > > > Additionally, I know that GCC must not
> > > > > > change the project layout, but from the software engineering
> > > > > > perspective,
> > > > > > this may be a bad smell that indicates that the file should be
> > > > > > broken
> > > > > > into smaller files. Finally, the Makefiles will take care of the
> > > > > > parallelization task.
> > > > >
> > > > > What do you mean by GCC must not change the project layout? GCC
> > > > > happily re-orders functions and link-time optimization will reorder
> > > > > TUs (well, linking may as well).
> > > > >
> > > >
> > > > That was a response to a comment made on IRC:
> > > >
> > > > On Thu, Nov 15, 2018 at 9:44 AM Jonathan Wakely
> > > > wrote:
> > > > >I think this is in response to a comment I made on IRC. Giuliano said
> > > > >that if a project has a very large file that dominates the total build
> > > > >time, the file should be split up into smaller pieces. I said "GCC
> > > > >can't restructure people's code. it can only try to compile it
> > > > >faster". We weren't referring to code transformations in the compiler
> > > > >like re-ordering functions, but physically refactoring the source
> > > > >code.
> > > >
> > > > Y
