It would be ideal if we could cover OSX in Jenkins, but the only solution that I'm aware of would require physical machines to be the workers. I would be weakly opposed to having physical servers running on PRs. The downsides that I see in order of importance:
- We can't autoscale physical hardware. If we find that the load is too high we have to buy more machines. - Security would be tricky, as they'd have to be connected to the internet and then to our Jekins master instance. Connecting via a wired network would probably not be possible on most corporate networks as these machines are by definition running arbitrary code from the internet. Many corporate sites have public wifi that this machine could potentially connect to, but then our PRs start failing if the wifi disconnects temporarily. To connect to the master we would need to setup a vpn solution with endpoints in our vpc on AWS. This is possible but would probably require a lot of security work. - We can't just create a simple startup script or yaml file that is checked into GitHub to manage the machine. Someone will actually have to physically administer the machine, apply updates, etc. which will make community ownership difficult. Specific to an OSX build: - We can't virtualize OSX which means we'd only be able to cover one OSX build environment per physical device. We couldn't target a matrix of OSX and Xcode versions as in Travis. -Kellen On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier <[email protected]> wrote: > So why Travis when we could possibly use Jenkins? > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu < > [email protected]> > wrote: > > > Yes that's correct, Chris. > > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" <[email protected] > >: > > > > > A quick google search seems to indicate that Mac can be used as a > Jenkins > > > slave. Is this correct? > > > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel < > [email protected]> > > > wrote: > > > > > > > +1 for #1 and #2 > > > > > > > > I’m working on getting a MacPro to add to CI system. > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland < > > > > [email protected]> wrote: > > > > > > > > > Background: TravisCI is a startup providing managed continuous > > > > > integration services with GitHub integration and YAML based > > > > configuration. > > > > > TravisCI is one of the few CI providers that will build a variety > of > > > > > OSX/MacOS builds for software projects. Their pricing ranges from > > Free > > > > > (for open source, 1 concurrent job, to $489 monthly for 10 > concurrent > > > > jobs). > > > > > > > > > > Problem: We’ve had a few OSX build issues slip into MXNet master in > > the > > > > > past few weeks. We’ve previously had a Travis CI based testing > > system > > > > that > > > > > would have caught these issues. > > > > > > > > > > Proposals so far: > > > > > > > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity check > on > > > OSX. > > > > > If we compile the program, and for example run C++ unit tests we’re > > > > > unlikely to run into problems with queued builds. The total build > > time > > > > > here should be less than 15 minutes. Configuration should be quite > > > > simple > > > > > and easy to maintain. Error messages should also be obvious to > > > > > contributors. > > > > > 2) Run clang in Linux with our current CI. Building with clang > > should > > > > > take less than 10 minutes, should flush out a large subset of the > > > issues > > > > > we’ve seen with OSX, and be quite easy to maintain. > > > > > 3) Run full test-suites in TravisCI, equaling the level of coverage > > we > > > > > provide to Linux in Jenkins. This could require us to subscribe > to a > > > > > monthly package with Travis to ensure our build queue doesn’t grow > to > > > an > > > > > unacceptable length. It may also require a volunteer to setup and > > > > maintain > > > > > long-term. > > > > > > > > > > I’d +1 #1 and #2 as I think those should be low-cost, low-maintence > > > > > solutions that should catch the majority of the problems we’ve seen > > > thus > > > > > far. > > > > > > > > > > -Kellen > > > > > > > > > > > > > > >
