Could you tell us what the problem is, e.g. do you get error messages, exceptions, ... ?
Maarten ----- Original Message ---- From: Uldis Karlovs-Karlovskis <uldis.karlovs-karlovs...@ctco.lv> To: ivy-u...@ant.apache.org Sent: Friday, March 20, 2009 4:50:09 PM Subject: RE: Running Ivy in parallel builds The parallel artifacts download unfortunately failed. Artifact-lock is not helping. However, this is much better than it was. Regards, Uldis, C. T. Co uldis.karlovs-karlovs...@ctco.lv -----Original Message----- From: Alan Chaney [mailto:a...@writingshow.com] Sent: piektdiena, 2009. gada 20. martā 17:47 To: ivy-u...@ant.apache.org Subject: Re: Running Ivy in parallel builds Hi Maarten Maarten Coene wrote: > Hi, > > I think you can solve it like this > - use the same cache for your artifacts > - use a different (per build?) cache for your resolution data > - use the "artifact-lock" locking strategy on your artifact cache > > More info about the caches is here: > http://ant.apache.org/ivy/history/latest-release/settings/caches.html > > Maarten > > I hope you don't consider the following too off-topic but I'm trying to set up a parallel build and I'm a little confused about the docs as they relate to caching. In http://ant.apache.org/ivy/history/latest-release/concept.html#cache it states: While there is always only one resolution cache, you can define multiple repository caches <http://ant.apache.org/ivy/history/latest-release/settings/caches.html>, each resolver <http://ant.apache.org/ivy/history/latest-release/settings/resolvers.html> being able to use a separate cache. But in the actual caches/cache description it says that you can define the cachedir within your ivysettings.xml. I assume that the 'only one resolution cache' means 'per ivy settings'. Is this correct? If you have multiple ivy settings defined in a project and you specify which settings you use you can presumably use different resolution caches for different aspects of the project? Regards Alan Chaney > > > ----- Original Message ---- > From: Uldis Karlovs-Karlovskis <uldis.karlovs-karlovs...@ctco.lv> > To: ivy-u...@ant.apache.org > Sent: Friday, March 20, 2009 11:45:21 AM > Subject: RE: Running Ivy in parallel builds > > OMG, this is totally unacceptable. We started to use Ivy to optimize space > for builds and automatize build tools setups. My Ivy cache stores about > 600Mb of different tools initially which are reused then for every build. So > using cache per build is a nightmare! Using cache per running parallel build > sounds better but developing this custom configuration can make a lot of > unexpected problems afterthen. > Ivy on his own should know if cache is used by another process and wait for > it. Does anybody is going to implement this functionality? Since I`m using > special Ivy wrapper I can try to do it on my own on my side but it is not a > good idea anyway. > > Regards, > Uldis, C. T. Co > uldis.karlovs-karlovs...@ctco.lv > > > -----Original Message----- > From: Martin Eigenbrodt [mailto:martineigenbr...@googlemail.com] > Sent: ceturtdiena, 2009. gada 19. marta 20:31 > To: ivy-u...@ant.apache.org > Subject: Re: Running Ivy in parallel builds > > I do run Ivy on Hudson and use one Cache per "Executor". So builds in > parallel have different caches but sequential builds may share > a cache. Hudson provides an Environment Variabel with the Executor Number > that I use within the ivysettings.xml to set different Cachedirs. A similair > setup may be possible in other environments. > A second concurency issue is that the buildnumber and deploy tasks are > seperated. There should be an atomic "deployWithNext Buildnumber" or you > will run in problems when the same project is build in parallel. > > Best regards, > > Martin2009/3/19 Sam Berlin <sber...@gmail.com> > > >> Yes, I have noticed significant problems when using ivy in a parallel >> build environment (essentially, any build-server), and we've only had >> Ivy in use on the buildserver for ~12 hours. The only solution I can >> think of is to make the cache local per build, which is pretty far >> from ideal. >> >> Any suggestions for how to make Ivy more robust against concurrent builds? >> >> Sam >> >> 2009/3/19 Uldis Karlovs-Karlovskis <uldis.karlovs-karlovs...@ctco.lv>: >> >>> And this is not all. Previous case was when artifact is already in >>> > cache. > >> If >> >>> Ivy needs to download new artifact then parallel execution gives >>> >> something >> >>> like this: >>> >>> --------------------------------------------------------------------- >>> | | modules || artifacts | >>> | conf | number| search|dwnlded|evicted|| number|dwnlded| >>> >>> --------------------------------------------------------------------- >>> | default | 4 | 0 | 0 | 0 || 6 | 0 | >>> >>> --------------------------------------------------------------------- >>> >>> :: problems summary :: >>> :::: WARNINGS >>> problem while downloading module descriptor: >>> >>> > http://server/download?name=somemodule&version=1.3.1.16&file=somemodule&file > >>> Type=xml: impossible to move part file to definitive one: >>> /root/.ivy2/cache/ctco/somemodule/ivy-1.3.1.16.xml.original.part -> >>> /root/.ivy2/cache/ctco/somemodule/ivy-1.3.1.16.xml.original (21ms) >>> >>> module not found: ctco#somemodule;1.3.1.16 >>> >>> ==== bones-repository: tried >>> >>> >>> >>> > http://server/download?name=somemodule&version=1.3.1.16&file=somemodule&file > >>> Type=xml >>> >>> :::::::::::::::::::::::::::::::::::::::::::::: >>> >>> :: UNRESOLVED DEPENDENCIES :: >>> >>> :::::::::::::::::::::::::::::::::::::::::::::: >>> >>> :: ctco#somemodule;1.3.1.16: not found >>> >>> :::::::::::::::::::::::::::::::::::::::::::::: >>> >>> >>> >>> :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS >>> command exit code: 1 >>> >>> Regards, >>> Uldis, C. T. Co >>> uldis.karlovs-karlovs...@ctco.lv >>> >>> >>> -----Original Message----- >>> From: Uldis Karlovs-Karlovskis [mailto:uldis.karlovs-karlovs...@ctco.lv] >>> Sent: ceturtdiena, 2009. gada 19. marta 17:37 >>> To: ivy-u...@ant.apache.org >>> Subject: Running Ivy in parallel builds >>> >>> Hello! >>> >>> >>> >>> Have You tested running Ivy in more than one build at the time? >>> >>> I see there are few files in cache which got updated every time I run >>> >> Ivy. >> >>> When my build system is running more than one process at the time >>> >> sometimes >> >>> I got this exception: >>> >>> :: retrieving :: lv.ctco#scm-bootstrap >>> >>> confs: [default] >>> >>> Caught: java.lang.RuntimeException: problem during retrieve of >>> <namespace>#<module>: java.text.ParseException: failed to parse report: >>> /root/.ivy2/cache/lv.ctco-scm-bootstrap-default.xml: Premature end of >>> >> file. >> >>> command exit code: 1 >>> >>> >>> >>> >>> >>> Uldis Karlovs - Karlovskis >>> >>> Lead SCM Engineer >>> >>> System Configuration Management >>> >>> C.T.Co >>> >>> Mobile: +371 29 345 210 >>> >>> >>> >>> >>> >>> > > > > > > !DSPAM:49c3796450102136417547! > >
