Don’t get me wrong, I’m not saying this Mac OS Jenkins solution is doable but I feel like we should investigate because the payoff would be large.
On Tue, Dec 12, 2017 at 9:38 AM Chris Olivier <[email protected]> wrote: > Apple’s Darwin OS Is recently open-sourced. > https://github.com/PureDarwin/PureDarwin > > How to convert this into a non-GUI VM I am not sure but I am willing to > bet that people have done it already. > > On Tue, Dec 12, 2017 at 9:16 AM kellen sunderland < > [email protected]> wrote: > >> It might be technically possible, but I think it would violate the MacOS >> license: http://store.apple.com/Catalog/US/Images/MacOSX.htm >> >> "2. Permitted License Uses and Restrictions. >> A. This License allows you to install and use one copy of the Apple >> Software on a single Apple-labeled computer at a time. This License does >> not allow the Apple Software to exist on more than one computer at a >> time,and you may not make the Apple Software available over a network >> where >> it could be used by multiple computers at the same time. You may make one >> copy of the Apple Software (excluding the Boot ROM code) in >> machine-readable form for backup purposes only; provided that the backup >> copy must include all copyright or other proprietary notices contained on >> the original. " >> >> I could be wrong though, does anyone know the details of MacOS licensing / >> virtualization? >> >> On Tue, Dec 12, 2017 at 6:10 PM, Chris Olivier <[email protected]> >> wrote: >> >> > googling seems to be full of running OSX (and even open-sourced >> PureDarwin) >> > in VMs. One could conceivably run a VM on an EC2 instance, right? >> > >> > On Tue, Dec 12, 2017 at 9:01 AM kellen sunderland < >> > [email protected]> wrote: >> > >> > > It would be ideal if we could cover OSX in Jenkins, but the only >> solution >> > > that I'm aware of would require physical machines to be the workers. >> I >> > > would be weakly opposed to having physical servers running on PRs. >> The >> > > downsides that I see in order of importance: >> > > >> > > - We can't autoscale physical hardware. If we find that the load is >> > too >> > > high we have to buy more machines. >> > > - Security would be tricky, as they'd have to be connected to the >> > internet >> > > and then to our Jekins master instance. Connecting via a wired >> network >> > > would probably not be possible on most corporate networks as these >> > machines >> > > are by definition running arbitrary code from the internet. Many >> > corporate >> > > sites have public wifi that this machine could potentially connect to, >> > but >> > > then our PRs start failing if the wifi disconnects temporarily. To >> > connect >> > > to the master we would need to setup a vpn solution with endpoints in >> our >> > > vpc on AWS. This is possible but would probably require a lot of >> > security >> > > work. >> > > - We can't just create a simple startup script or yaml file that is >> > > checked into GitHub to manage the machine. Someone will actually >> have to >> > > physically administer the machine, apply updates, etc. which will make >> > > community ownership difficult. >> > > >> > > Specific to an OSX build: >> > > - We can't virtualize OSX which means we'd only be able to cover one >> OSX >> > > build environment per physical device. We couldn't target a matrix of >> > OSX >> > > and Xcode versions as in Travis. >> > > >> > > -Kellen >> > > >> > > On Tue, Dec 12, 2017 at 5:46 PM, Chris Olivier <[email protected] >> > >> > > wrote: >> > > >> > > > So why Travis when we could possibly use Jenkins? >> > > > >> > > > On Tue, Dec 12, 2017 at 7:59 AM Marco de Abreu < >> > > > [email protected]> >> > > > wrote: >> > > > >> > > > > Yes that's correct, Chris. >> > > > > >> > > > > Am 12.12.2017 4:46 nachm. schrieb "Chris Olivier" < >> > > [email protected] >> > > > >: >> > > > > >> > > > > > A quick google search seems to indicate that Mac can be used as >> a >> > > > Jenkins >> > > > > > slave. Is this correct? >> > > > > > >> > > > > > On Tue, Dec 12, 2017 at 7:42 AM Steffen Rochel < >> > > > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > +1 for #1 and #2 >> > > > > > > >> > > > > > > I’m working on getting a MacPro to add to CI system. >> > > > > > > On Tue, Dec 12, 2017 at 1:43 AM kellen sunderland < >> > > > > > > [email protected]> wrote: >> > > > > > > >> > > > > > > > Background: TravisCI is a startup providing managed >> continuous >> > > > > > > > integration services with GitHub integration and YAML based >> > > > > > > configuration. >> > > > > > > > TravisCI is one of the few CI providers that will build a >> > variety >> > > > of >> > > > > > > > OSX/MacOS builds for software projects. Their pricing >> ranges >> > > from >> > > > > Free >> > > > > > > > (for open source, 1 concurrent job, to $489 monthly for 10 >> > > > concurrent >> > > > > > > jobs). >> > > > > > > > >> > > > > > > > Problem: We’ve had a few OSX build issues slip into MXNet >> > master >> > > in >> > > > > the >> > > > > > > > past few weeks. We’ve previously had a Travis CI based >> testing >> > > > > system >> > > > > > > that >> > > > > > > > would have caught these issues. >> > > > > > > > >> > > > > > > > Proposals so far: >> > > > > > > > >> > > > > > > > 1) Use TravisCI in it’s free mode for a very minimal sanity >> > check >> > > > on >> > > > > > OSX. >> > > > > > > > If we compile the program, and for example run C++ unit >> tests >> > > we’re >> > > > > > > > unlikely to run into problems with queued builds. The total >> > > build >> > > > > time >> > > > > > > > here should be less than 15 minutes. Configuration should >> be >> > > quite >> > > > > > > simple >> > > > > > > > and easy to maintain. Error messages should also be >> obvious to >> > > > > > > > contributors. >> > > > > > > > 2) Run clang in Linux with our current CI. Building with >> clang >> > > > > should >> > > > > > > > take less than 10 minutes, should flush out a large subset >> of >> > the >> > > > > > issues >> > > > > > > > we’ve seen with OSX, and be quite easy to maintain. >> > > > > > > > 3) Run full test-suites in TravisCI, equaling the level of >> > > coverage >> > > > > we >> > > > > > > > provide to Linux in Jenkins. This could require us to >> > subscribe >> > > > to a >> > > > > > > > monthly package with Travis to ensure our build queue >> doesn’t >> > > grow >> > > > to >> > > > > > an >> > > > > > > > unacceptable length. It may also require a volunteer to >> setup >> > > and >> > > > > > > maintain >> > > > > > > > long-term. >> > > > > > > > >> > > > > > > > I’d +1 #1 and #2 as I think those should be low-cost, >> > > low-maintence >> > > > > > > > solutions that should catch the majority of the problems >> we’ve >> > > seen >> > > > > > thus >> > > > > > > > far. >> > > > > > > > >> > > > > > > > -Kellen >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
