Sorry, I should explain further. The Data Import Handler had been running for a while retrieving only about 150000 records from the database. Both in development env (windows) and linux machine it took about 3 mins.
The query has been changed and we are now trying to retrieve about 10 million records. We do expect the time to increase. With the new query the time taken on windows machine is consistently around 40 mins. While the DIH is running queries slow down i.e. a query that typically took 60 msec takes 100 msec. The time taken on linux machine is consistently around 2.5 hours. While the DIH is running queries take about 200 to 400 msec. Thanks! On Mon, Feb 1, 2016 at 8:45 PM, Erick Erickson <erickerick...@gmail.com> wrote: > What happens if you run just the SQL query from the > windows box and from the linux box? Is there any chance > that somehow the connection from the linux box is > just slower? > > Best, > Erick > > On Mon, Feb 1, 2016 at 6:36 PM, Alexandre Rafalovitch > <arafa...@gmail.com> wrote: > > What are you importing from? Is the source and Solr machine collocated > > in the same fashion on dev and prod? > > > > Have you tried running this on a Linux dev machine? Perhaps your prod > > machine is loaded much more than a dev. > > > > Regards, > > Alex. > > ---- > > Newsletter and resources for Solr beginners and intermediates: > > http://www.solr-start.com/ > > > > > > On 2 February 2016 at 13:21, Troy Edwards <tedwards415...@gmail.com> > wrote: > >> We have a windows development machine on which the Data Import Handler > >> consistently takes about 40 mins to finish. Queries run fine. JVM > memory is > >> 2 GB per node. > >> > >> But on a linux machine it consistently takes about 2.5 hours. The > queries > >> also run slower. JVM memory here is also 2 GB per node. > >> > >> How should I go about analyzing and tuning the linux machine? > >> > >> Thanks >
