On 9/4/2012 11:09 PM, Nicholas Nethercote wrote:
On Tue, Sep 4, 2012 at 9:39 PM, Mike Hommey wrote:
On the other hand, the ccache miss rate is pretty high, at least it is
for me, so in the end, ccache might not be a win at all.
That's what I've found, for my compiling patterns.
I finally go
On Tue, Sep 4, 2012 at 9:39 PM, Mike Hommey wrote:
>
> On the other hand, the ccache miss rate is pretty high, at least it is
> for me, so in the end, ccache might not be a win at all.
That's what I've found, for my compiling patterns.
Nick
___
dev-pla
On 9/3/2012 6:45 PM, qheaden wrote:
On Sunday, September 2, 2012 5:15:40 PM UTC-4, Gregory Szorc wrote:
A decision has been made: we will be using Python files executing in a
sandboxed environment (using the technique that Hanno posted).
Supporting this decision are Ted (build system owner) an
On Tue, Sep 04, 2012 at 01:59:29PM +0300, Aryeh Gregor wrote:
> On Tue, Sep 4, 2012 at 3:19 AM, Gregory Szorc wrote:
> > I re-ran some builds on my desktop with my instrumented build system that
> > records system metrics and an SSD clobber build was only ~50s faster than
> > the same build on a 7
On Tue, Sep 4, 2012 at 3:19 AM, Gregory Szorc wrote:
> I re-ran some builds on my desktop with my instrumented build system that
> records system metrics and an SSD clobber build was only ~50s faster than
> the same build on a 7200RPM drive. This comes out to ~5%, which isn't much.
> Both results
On Sunday, September 2, 2012 5:15:40 PM UTC-4, Gregory Szorc wrote:
> A decision has been made: we will be using Python files executing in a
>
> sandboxed environment (using the technique that Hanno posted).
>
> Supporting this decision are Ted (build system owner) and Benjamin
>
> Smedberg a
On 8/31/2012 5:28 AM, Aryeh Gregor wrote:
On Thu, Aug 30, 2012 at 1:25 AM, Gregory Szorc wrote:
If you have 9+GB of memory dedicated to building mozilla-central and you are
doing warm builds all the time, great, you probably don't need an SSD. But,
those conditions are specialized. You have to
tion about the new world order will be communicated once
things are closer to landing. If you wish to influence it, please follow
bug 784841.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=784841
On 8/21/2012 4:36 PM, Gregory Szorc wrote:
tl;dr We're proposing moving away from Makefil
On 8/31/12 8:28 AM, Aryeh Gregor wrote:
In practice, I've noticed minimal difference using SSDs on my desktop,
which has 16G of RAM. I have lots of other stuff open, but all that
only uses a couple of gigs of RAM, so there should certainly be 9G for
page cache. (free -m reports about 7G used fo
On 12-08-31 8:28 AM, Aryeh Gregor wrote:
If you use a desktop and work on compiled code even part-time, at any
reasonable pay rate, I can't see any reason why you wouldn't get 16G
of RAM. Although it seems most people don't -- maybe people mostly
use laptops? 2x8G of desktop RAM is about $75 ri
On Thu, Aug 30, 2012 at 1:25 AM, Gregory Szorc wrote:
> If you have 9+GB of memory dedicated to building mozilla-central and you are
> doing warm builds all the time, great, you probably don't need an SSD. But,
> those conditions are specialized. You have to get things into the page
> cache. That'
On 8/29/12 2:51 PM, Karl Tomlinson wrote:
On 8/22/12 6:10 AM, Robert Kaiser wrote:
[...]
esp. given that basic file I/O is often costly (from watching my CPU
usage, a lot of the build time is spent in I/O wait when using spinning
disks - SSDs improve that hugely).
On Wed, 22 Aug 2012 12:29:2
> On 8/22/12 6:10 AM, Robert Kaiser wrote:
>> [...]
>> esp. given that basic file I/O is often costly (from watching my CPU
>> usage, a lot of the build time is spent in I/O wait when using spinning
>> disks - SSDs improve that hugely).
On Wed, 22 Aug 2012 12:29:24 -0700, Gregory Szorc wrote:
>
Gregory Szorc schrieb:
* "if"
is "ifneq (,$(foo))" or even "ifneq (,$(strip $(foo)))" in case some
extra whitespace snuck in there.
One thing that was hoped we could achieve with pymake was to possibly
switch to it everywhere at some time and then be able to replace those
ugly conditionals wi
On 29/08/12 19:45, Gregory Szorc wrote:
On 8/23/2012 9:11 AM, Hanno Schlichting wrote:
Instead of using Python's ast module, you can also do a simple trick
with the exec statement and limit the global scope and only allow
certain whitelisted names. An example implementation is at
https://gist.gi
On 8/23/2012 9:11 AM, Hanno Schlichting wrote:
Instead of using Python's ast module, you can also do a simple trick
with the exec statement and limit the global scope and only allow
certain whitelisted names. An example implementation is at
https://gist.github.com/3437909. Download it as restri
On 8/24/12 8:05 AM, John Hopkins wrote:
Suggestion: identify the ugliest sections of Makefile usage and use
those as a benchmark for evaluating different solutions. ie. how could
it be implemented in SCons, pymake, etc. or even, how could it be
reimplemented in Make in a clean fashion.
I've s
On 8/22/12 11:48 PM, Dirkjan Ochtman wrote:
On Wed, Aug 22, 2012 at 1:36 AM, Gregory Szorc wrote:
Up until now, the focus has been on making Makefile.in's themselves generic
and data-driven [1]. We would use pymake's API to parse, load, and extract
data from Makefile.in's to construct the build
On 8/24/12 12:42 PM, Brian Smith wrote:
I do not think it is reasonable to require support for alternate build systems
for all of Gecko/Firefox.
But, let's say were were to divide the build into three phases:
1. Generate any generated C/C++ source files.
2. Build all the C/C++ code into librari
Gregory Szorc wrote:
> 4. Native support for list and maps. Make files only support strings.
> The hacks this results in are barely tolerable.
>
> 5. Ability to handle conditionals. We need to be able to
> conditionally define things based on the presence or value of certain
> "variables."
> e.g.
On Friday, August 24, 2012 10:32:46 AM UTC-4, Ted Mielczarek wrote:
> On Fri, Aug 24, 2012 at 9:17 AM, qheaden
> wrote:
>
> > Is there any special reason why an existing build system such as SCcons
> > couldn't be used as a new build system for Mozilla? I know the Mozilla
> > source has a lot
On Fri, Aug 24, 2012 at 9:17 AM, qheaden wrote:
> Is there any special reason why an existing build system such as SCcons
> couldn't be used as a new build system for Mozilla? I know the Mozilla source
> has a lot of special build instructions, but SCons does allow you to create
> your own spec
On Tuesday, August 21, 2012 7:36:26 PM UTC-4, Gregory Szorc wrote:
> tl;dr We're proposing moving away from Makefile's as the sole source of
>
> the build system definition. This will lead to faster build times.
>
> Bikeshedding^wFeedback on the file format is
Gregory Szorc wrote:
We want feedback on the file format to use for the new build manifest
files.
Is there a good reason to shoehorn everything into a single file format?
--
Warning: May contain traces of nuts.
___
dev-platform mailing list
dev-plat
On Thu 23 Aug 2012 09:11:46 AM PDT, Hanno Schlichting wrote:
By default there's no import statement and no way to open files, or even just any of the
exception names. You can define functions, but not classes. One could add a special
import statement or some other function like "include" to all
On 23.08.2012, at 17:07 , Ehsan Akhgari wrote:
> I think Gregory's suggestion of using the python's ast module to limit the
> sorts of syntax constructs that we accept in the build manifest files is a
> great one. That would let us make python powerful enough for our needs, but
> not more powe
On 12-08-23 12:46 AM, Nicholas Nethercote wrote:
So...
Data-only formats such as JSON are too weak -- no comments, no conditionals.
Full languages (e.g. Python, Lua) are too powerful, which means it's
too tempting to use the full language features.
Should you invent something new? If it's jus
On Thu, Aug 23, 2012 at 9:39 AM, David Rajchenbach-Teller
wrote:
> Generally, this design is rather good at separating tasks that need a
> full language (implemented as rules) from tasks that only need a trivial
> description (implemented as description files).
This discussion is focused solely o
While I do not advocate the use of ocamlbuild, we could draw some
inspiration from the design.
Essentially, an ocamlbuild-based build system is based around a library
of rules (implemented in OCaml, we could do the same in Python, of
course), and any number of trivial description files. OCamlBuild
On Thu, Aug 23, 2012 at 08:43:31AM -0400, Ben Hearsum wrote:
> Yeah, I think I agree. My experience in RelEng has biased me strongly
> towards not allowing even temporary hacks like that, because it's rare
> that we ever remove them, and end up with a pile of very difficult to
> maintain hacks (Tin
On 08/23/12 12:05 AM, Robert O'Callahan wrote:
> On Thu, Aug 23, 2012 at 8:40 AM, Ben Hearsum wrote:
>
>> On 08/22/12 04:38 PM, Gregory Szorc wrote:
>>> Let's think of what can be done to secure/limit Python. Disabling import
>>> has already been mentioned. That's a start.
>>
>> I think it's wort
On Wed, Aug 22, 2012 at 1:36 AM, Gregory Szorc wrote:
> Up until now, the focus has been on making Makefile.in's themselves generic
> and data-driven [1]. We would use pymake's API to parse, load, and extract
> data from Makefile.in's to construct the build definition. In the long run,
> we'd real
On 08/22/2012 11:19 PM, Mike Hommey wrote:
On Wed, Aug 22, 2012 at 01:38:22PM -0700, Gregory Szorc wrote:
On 8/22/12 1:00 PM, Ben Hearsum wrote:
On 08/22/12 12:43 PM, Jeff Hammel wrote:
If we do go with python, it would be nice to keep the configuration
files as much configuration as possible.
On Wed, Aug 22, 2012 at 01:38:22PM -0700, Gregory Szorc wrote:
> On 8/22/12 1:00 PM, Ben Hearsum wrote:
> >On 08/22/12 12:43 PM, Jeff Hammel wrote:
> >>If we do go with python, it would be nice to keep the configuration
> >>files as much configuration as possible. The reason I question having
> >>
So...
Data-only formats such as JSON are too weak -- no comments, no conditionals.
Full languages (e.g. Python, Lua) are too powerful, which means it's
too tempting to use the full language features.
Should you invent something new? If it's just lists, maps, comments
and conditionals, that shou
On Thu, Aug 23, 2012 at 8:40 AM, Ben Hearsum wrote:
> On 08/22/12 04:38 PM, Gregory Szorc wrote:
> > Let's think of what can be done to secure/limit Python. Disabling import
> > has already been mentioned. That's a start.
>
> I think it's worth noting that even if you *do* limit what you can do
>
On Wed, Aug 22, 2012 at 3:38 PM, Gregory Szorc wrote:
> Let's think of what can be done to secure/limit Python. Disabling import
> has already been mentioned. That's a start.
>
> What about the ast module [1]? I /think/ it could be used to ensure parsed
> code conforms to whatever we allow. e.g.
On 08/22/12 04:38 PM, Gregory Szorc wrote:
> Let's think of what can be done to secure/limit Python. Disabling import
> has already been mentioned. That's a start.
I think it's worth noting that even if you *do* limit what you can do
through some technical means, you still have the option to chang
On 8/22/12 1:00 PM, Ben Hearsum wrote:
On 08/22/12 12:43 PM, Jeff Hammel wrote:
If we do go with python, it would be nice to keep the configuration
files as much configuration as possible. The reason I question having
any "full" language as a configuration language is that in practice I
see a l
On 08/22/2012 12:29 PM, Gregory Szorc wrote:
Switching to something else also has another advantage: the
opportunity for a clean slate. I'm hoping we'll use the opportunity to
scrape away 10+ years of cruft.
I support the sentiment, though this reasoning always makes me cautious,
as it oft
On 08/22/12 12:43 PM, Jeff Hammel wrote:
> If we do go with python, it would be nice to keep the configuration
> files as much configuration as possible. The reason I question having
> any "full" language as a configuration language is that in practice I
> see a lot of logic creep in to "configura
Jeff Hammel wrote:
While a bit of an unfair example, our buildbot-configs fall into this
category.
IMO not unfair at all.
(p.s. to stay on topic, +1 to all else you said)
~Justin Wood (Callek)
___
dev-platform mailing list
dev-platform@lists.mozilla
On 8/22/12 2:02 AM, Neil wrote:
Gregory Szorc wrote:
Up until now, the focus has been on making Makefile.in's themselves
generic and data-driven. We would use pymake's API to parse, load, and
extract data from Makefile.in's to construct the build definition. In
the long run, we'd realize that u
On 12-08-22 12:43 PM, Jeff Hammel wrote:
TL;DR - python if we want/need a full language, JSON if we can get away
with POD
I think JSON is the wrong choice here. It will not satisfy the need for
conditionals, which causes us to invent terrible hacks inside the JSON
like we had to do with the
On 08/22/2012 06:50 AM, Benjamin Smedberg wrote:
On 8/21/2012 7:36 PM, Gregory Szorc wrote:
On the other end of the spectrum, we could have the build manifest
files be Python "scripts." This solves a lot of problems around
needing functionality in the manifest files. But, it would be a
pot
On Thu, Aug 23, 2012 at 01:07:34AM +1200, Blair McBride wrote:
> On 22/08/2012 5:58 p.m., Mike Hommey wrote:
> >>+1 for Lua - it seems perfect for this. For #1, I find it far easier
> >>>to read (and write) than Gyp, when it comes to things like
> >>>conditionals. For #8, we could just ship the ent
On 8/21/2012 7:36 PM, Gregory Szorc wrote:
On the other end of the spectrum, we could have the build manifest
files be Python "scripts." This solves a lot of problems around
needing functionality in the manifest files. But, it would be a
potential foot gun. See requirement #7.
I don't think
Gregory Szorc schrieb:
We could go the route of GYP and shoehorn conditionals into a static
document (JSON) [3].
JSON is a good format for data for the most part, but IMHO we *really*
want comments in those files, and unfortunately JSON doesn't have those
and therefore probably must be thrown
On 22/08/2012 5:58 p.m., Mike Hommey wrote:
+1 for Lua - it seems perfect for this. For #1, I find it far easier
>to read (and write) than Gyp, when it comes to things like
>conditionals. For #8, we could just ship the entire runtime in the
>tree for Tier 1 platforms (its small enough!), then its
Gregory Szorc wrote:
Up until now, the focus has been on making Makefile.in's themselves
generic and data-driven. We would use pymake's API to parse, load, and
extract data from Makefile.in's to construct the build definition. In
the long run, we'd realize that using make files for data defini
On Wed, Aug 22, 2012 at 03:56:33PM +1200, Blair McBride wrote:
> On 22/08/2012 11:36 a.m., Gregory Szorc wrote:
> >I think Lua is perfect for this (it was invented to be a configuration
> >language after all). But, I'm not sure it satisfies #1 nor #8.
>
> +1 for Lua - it seems perfect for this. Fo
On 22/08/2012 11:36 a.m., Gregory Szorc wrote:
I think Lua is perfect for this (it was invented to be a configuration
language after all). But, I'm not sure it satisfies #1 nor #8.
+1 for Lua - it seems perfect for this. For #1, I find it far easier to
read (and write) than Gyp, when it comes
On 8/21/2012 6:12 PM, xunxun wrote:
于 2012/8/22 7:36, Gregory Szorc 写道:
Up until now, the focus has been on making Makefile.in's themselves
generic and data-driven [1]. We would use pymake's API to parse,
load, and extract data from Makefile.in's to construct the build
definition. In the long
于 2012/8/22 7:36, Gregory Szorc 写道:
tl;dr We're proposing moving away from Makefile's as the sole source
of the build system definition. This will lead to faster build times.
Bikeshedding^wFeedback on the file format is requested.
The existing build system is defined by Mak
tl;dr We're proposing moving away from Makefile's as the sole source of
the build system definition. This will lead to faster build times.
Bikeshedding^wFeedback on the file format is requested.
The existing build system is defined by Makefile.in's scattered around
the source
55 matches
Mail list logo
