Re: DIH Issues

2017-04-25 Thread Sales
> On Apr 25, 2017, at 10:28 AM, AJ Lemke wrote: > > Thanks for the thought Alex! > The fields that have this happen most often are numeric and boolean fields. > These fields have real data (id numbers, true/false, etc.) > > AJ > We had an identical problem a few months ago, and there was no

Re: DIH Issues

2017-04-25 Thread Alexandre Rafalovitch
and boolean fields. >> These fields have real data (id numbers, true/false, etc.) >> >> AJ >> >> -Original Message- >> From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] >> Sent: Tuesday, April 25, 2017 8:27 AM >> To: solr-user >

Re: DIH Issues

2017-04-25 Thread Erick Erickson
se, etc.) > > AJ > > -Original Message- > From: Alexandre Rafalovitch [mailto:arafa...@gmail.com] > Sent: Tuesday, April 25, 2017 8:27 AM > To: solr-user > Subject: Re: DIH Issues > > Maybe the content gets simplified away between the database and the Solr >

RE: DIH Issues

2017-04-25 Thread AJ Lemke
: solr-user Subject: Re: DIH Issues Maybe the content gets simplified away between the database and the Solr schema. For example if your field contains just spaces and you have UpdateRequestProcessors to do trim and removal of empty fields? Schemaless mode will remove empty fields, but will not

Re: DIH Issues

2017-04-25 Thread Alexandre Rafalovitch
Maybe the content gets simplified away between the database and the Solr schema. For example if your field contains just spaces and you have UpdateRequestProcessors to do trim and removal of empty fields? Schemaless mode will remove empty fields, but will not trim for example. Regards, Alex. -

Re: DIH issues with 4.7.1

2014-04-26 Thread Mark Miller
bq. due to things like NTP, etc. The full sentence is very important. NTP is not the only way for this to happen - you also have leap seconds, daylight savings time, internet clock sync, a whole host of things that affect currentTimeMillis and not nanoTime. It is without question the way to go

Re: DIH issues with 4.7.1

2014-04-26 Thread Walter Underwood
NTP works very hard to keep the clock positive monotonic. But nanoTime is intended for elapsed time measurement anyway, so it is the right choice. You can get some pretty fun clock behavior by running on virtual machines, like in AWS. And some system real time clocks don't tick during a leap sec

Re: DIH issues with 4.7.1

2014-04-26 Thread Mark Miller
My answer remains the same. I guess if you want more precise terminology, nanoTime will generally be monotonic and currentTimeMillis will not be, due to things like NTP, etc. You want monotonicity for measuring elapsed times. --  Mark Miller about.me/markrmiller On April 26, 2014 at 11:25:16 AM,

Re: DIH issues with 4.7.1

2014-04-26 Thread Walter Underwood
NTP should slew the clock rather than jump it. I haven't checked recently, but that is how it worked in the 90's when I was organizing the NTP hierarchy at HP. It only does step changes if the clocks is really wrong. That is most likely at reboot, when other demons aren't running yet. wunder O

Re: DIH issues with 4.7.1

2014-04-26 Thread Mark Miller
System.currentTimeMillis can jump around due to NTP, etc. If you are trying to count elapsed time, you don’t want to use a method that can jump around with the results. --  Mark Miller about.me/markrmiller On April 26, 2014 at 8:58:20 AM, YouPeng Yang (yypvsxf19870...@gmail.com) wrote: Hi Rafał

Re: DIH issues with 4.7.1

2014-04-26 Thread YouPeng Yang
Hi Rafał Kuć I got it,the point is many operating systems measure time in units of tens of milliseconds,and the System.currentTimeMillis() is just base on operating system. In my case,I just do DIH with a crontable, Is there any possiblity to get in that trouble?I am really can not picture w

Re: DIH issues with 4.7.1

2014-04-26 Thread YouPeng Yang
Hi Mark Miller Sorry to get you in these discussion . I notice that Mark Miller report this issure in https://issues.apache.org/jira/browse/SOLR-5734 according to https://issues.apache.org/jira/browse/SOLR-5721,but it just happened with the zookeeper. If I just do DIH with JDBCDataSource ,I d

Re: DIH issues with 4.7.1

2014-04-26 Thread Rafał Kuć
Hello! Look at the javadocs for both. The granularity of System.currentTimeMillis() depend on the operating system, so it may happen that calls to that method that are 1 millisecond away from each other still return the same value. This is not the case with System.nanoTime() - http://docs.oracle.c

Re: DIH issues with 4.7.1

2014-04-26 Thread YouPeng Yang
Hi I have just compare the difference between the version 4.6.0 and 4.7.1. Notice that the time in the getConnection function is declared with the System.nanoTime in 4.7.1 ,while System.currentTimeMillis(). Curious about the resson for the change.the benefit of it .Is it neccessory? I hav

Re: DIH issues with 4.7.1

2014-04-25 Thread Shawn Heisey
On 4/25/2014 11:56 AM, Hutchins, Jonathan wrote: I recently upgraded from 4.6.1 to 4.7.1 and have found that the DIH process that we are using takes 4x as long to complete. The only odd thing I notice is when I enable debug logging for the dataimporthandler process, it appears that in the new ve

Re: DIH issues with 4.7.1

2014-04-25 Thread Alan Woodward
Hi Jonathan, It's a known bug: https://issues.apache.org/jira/browse/SOLR-5954. It'll be fixed in 4.8, which is being voted on now. Alan Woodward www.flax.co.uk On 25 Apr 2014, at 18:56, Hutchins, Jonathan wrote: > I recently upgraded from 4.6.1 to 4.7.1 and have found that the DIH > process