Hi Peter,
I am looking into improving performance of build.webkit.org. After doing some
initial research I feel that buildbot isn't the most efficient in terms of
processing a large number of requests. I believe requests are processed
serially, and not in parallel. The buildbot documentation mentions a limit to
the number of slaves that can be supported, but doesn't give any specific
numbers. As webkit has grown, and the number of builds and slaves we are
supporting has increased, it seems the server is showing signs of reaching this
capacity. We may have reached a limit on the number of slaves we can handle
with our current configuration.
Some of the things I have been researching to increase build.webkit.org
performance include the following:
-Upgrades to hardware. It's possible faster hardware could alleviate some
problems. It is unclear if this is really the limiting factor, or how much
additional capacity this would provide.
-Updating the way files are transferred
(https://bugs.webkit.org/show_bug.cgi?id=73484)
-Upgrade the version of buildbot. (0.8.3 -> 0.8.5).
Version 0.8.4 and higher supposedly has a threading improvement, but
it's unclear how much it would benefit us since it looks to be mostly database
related.
-Upgrading the version of twisted running on the sever. We are currently at
version 8.2, and version 12 is now available.
-Investigating some kind of apache caching layer on top of buildbot
-Adding a second master to the buildbot system.
buildbot supports multi-master mode. We could potentially split out
build and test onto two separate masters.
build.webkit.org and test.webkit.org ?
I would be interested in hearing any other thoughts on increasing
build.webkit.org performance.
Lucas
On Feb 29, 2012, at 7:08 AM, Peter Beverloo wrote:
> Hi Bill,
>
> Would it be possible to get some more information on your/Lucas' plans on
> improving stability of the infrastructure? This was the third significant
> interruption within a week, and especially for us non-PST folks it's becoming
> quite inconvenient.
>
> Thanks,
> Peter
>
> On Wed, Feb 29, 2012 at 15:00, William Siegrist <[email protected]> wrote:
> This should be fixed now.
>
> -Bill
>
> On Feb 29, 2012, at 3:19 AM, David Barr wrote:
>
> >> Is it any trick to update my git-svn repo from svn?
> >>
> >> I set up my git svn repo about
> >> $http://trac.webkit.org/wiki/UsingGitWithWebKit :
> >> - git clone git://git.webkit.org/WebKit.git WebKit
> >> - git svn init --prefix=origin/ -T trunk
> >> http://svn.webkit.org/repository/webkit
> >> - git config --replace svn-remote.svn.fetch
> >> trunk:refs/remotes/origin/master
> >
> > Usually, the only missing step here is 'git svn fetch'.
> > However, when I run that now I get:
> >
> > RA layer request failed: OPTIONS of
> > 'http://svn.webkit.org/repository/webkit': could not connect to server
> > (http://svn.webkit.org) at /Users/davidbarr/libexec/git-core/git-svn
> > line 2139
> >
> > --
> > David Barr.
> > _______________________________________________
> > webkit-dev mailing list
> > [email protected]
> > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
> _______________________________________________
> webkit-dev mailing list
> [email protected]
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
>
> _______________________________________________
> webkit-dev mailing list
> [email protected]
> http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
_______________________________________________
webkit-dev mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
