>>>
>>> The work being done is addressing the deletes, AIUI, but of course
>>> there are other things happening during shutdown, too.
>> There are no deletes to do. It was a clean index to begin with
>> and there were no duplicates.
>>
>
>I have not followed this thread, so forgive me if this has a
The work being done is addressing the deletes, AIUI, but of course
there are other things happening during shutdown, too.
There are no deletes to do. It was a clean index to begin with
and there were no duplicates.
I have not followed this thread, so forgive me if this has already
been sugg
>On Apr 2, 2009, at 9:23 AM, Fergus McMenemie wrote:
>
>> Grant,
>>
>>
>>
>>> I should note, however, that the speed difference you are seeing may
>>> not be as pronounced as it appears. If I recall during ApacheCon, I
>>> commented on how long it takes to shutdown your Solr instance when
>>> exit
On Apr 2, 2009, at 9:23 AM, Fergus McMenemie wrote:
Grant,
I should note, however, that the speed difference you are seeing may
not be as pronounced as it appears. If I recall during ApacheCon, I
commented on how long it takes to shutdown your Solr instance when
exiting it. That time it t
Grant,
>I should note, however, that the speed difference you are seeing may
>not be as pronounced as it appears. If I recall during ApacheCon, I
>commented on how long it takes to shutdown your Solr instance when
>exiting it. That time it takes is in fact Solr doing the work that
>was
On Apr 2, 2009, at 4:02 AM, Fergus McMenemie wrote:
Grant,
Hmmm, the big difference is made by &overwrite=false. But,
can you explain why &overwrite=false makes such a difference.
I am starting off with an empty index and I have checked the
content there are no duplicates in the uniqueKey field
>On Apr 1, 2009, at 9:39 AM, Fergus McMenemie wrote:
>
>> Grant,
>>
>> Redoing the work with your patch applied does not seem to
>
>>
>> make a difference! Is this the expected result?
>
>No, I didn't expect Solr 1095 to fix the problem. Overwrite = false +
>1095, does, however, AFAICT by your la
On Apr 1, 2009, at 9:39 AM, Fergus McMenemie wrote:
Grant,
Redoing the work with your patch applied does not seem to
make a difference! Is this the expected result?
No, I didn't expect Solr 1095 to fix the problem. Overwrite = false +
1095, does, however, AFAICT by your last line, righ
Grant,
Redoing the work with your patch applied does not seem to
make a difference! Is this the expected result?
I did run it again using the full file, this time using my Imac:-
643465took 22min 14sec 2008-04-01
734796 73min 58sec 2009-01-15
Grant,
I am messing with the script, and with your tip I expect I can
make it recurse over as many releases as needed.
I did run it again using the full file, this time using my Imac:-
643465took 22min 14sec 2008-04-01
734796 73min 58sec 2009-
Can you try adding &overwrite=false and running against the latest
version? My current working theory is that Solr/Lucene has changed
how deletes are handled such that work that was deferred before is now
not deferred as often. In fact, you are not seeing this cost paid (or
at least not n
svn co -r REV_NUM https://svn.apache.org/repos/asf/lucene/solr/trunk
solr-REV_NUM
-Grant
On Mar 30, 2009, at 4:55 PM, Fergus McMenemie wrote:
Can you verify that rev 701485 still performs reasonably well? This
is from October 2008 and I get similar results to the earlier rev.
Am now tryin
>Can you verify that rev 701485 still performs reasonably well? This
>is from October 2008 and I get similar results to the earlier rev.
>Am now trying some other versions between October and when you first
>reported the issue in November.
OK. Can you tell me how to get a hold of revisio
Grant,
After all my playing about at boot camp, I gave things a rest. It
was not till months later that got back to looking at solr again.
So after 643465 (2008-Apr-01) the next version I tried was 694377
from (2008-Sep-11). Nothing in between. Yep so 643465 is the latest
version I tried that st
Can you verify that rev 701485 still performs reasonably well? This
is from October 2008 and I get similar results to the earlier rev.
Am now trying some other versions between October and when you first
reported the issue in November.
-Grant
On Mar 30, 2009, at 3:37 PM, Grant Ingersol
Fregus,
Is rev 643465 the absolute latest you tried that still performs? i.e.
every revision after is slower?
-Grant
On Mar 30, 2009, at 12:45 PM, Grant Ingersoll wrote:
Fergus,
I think the problem may actually be due to something that was
introduced by a change to Solr's StopFilterFac
Fergus,
I think the problem may actually be due to something that was
introduced by a change to Solr's StopFilterFactory and the way it
loads the stop words set. See https://issues.apache.org/jira/browse/SOLR-1095
I am in the process of testing it out and will let you know.
-Grant
On Mar
Hey Fergus,
Finally got a chance to run your scripts, etc. per the thread:
http://www.lucidimagination.com/search/document/5c3de15a4e61095c/upgrade_from_1_2_to_1_3_gives_3x_slowdown_script#8324a98d8840c623
I can reproduce your slowdown.
One oddity with rev 643465 is:
On the old version, there
Yonik
>Another thought I just had - do you have autocommit enabled?
>
No; not as far as I know!
The solrconfig.xml from the two versions are equivalent as best I can tell,
also they are exactly as provided in the download. The only changes were
made by the attached script and should not affect co
Another thought I just had - do you have autocommit enabled?
A lucene commit is now more expensive because it syncs the files for
safety. If you commit frequently, this could definitely cause a
slowdown.
-Yonik
On Wed, Nov 26, 2008 at 10:54 AM, Fergus McMenemie <[EMAIL PROTECTED]> wrote:
> Hell
Hello Grant,
>
>Haven't forgotten about you, but I've been traveling and then into
>some US Holidays here.
Happy thanks giving!
>
>To confirm I am understanding, you are seeing a slowdown between 1.3-
>dev from April and one from September, right?
Yep.
Here are the MD5 hashes:-
fergus: md5 *.w
Hi Fergie,
Haven't forgotten about you, but I've been traveling and then into
some US Holidays here.
To confirm I am understanding, you are seeing a slowdown between 1.3-
dev from April and one from September, right?
Can you produce an MD5 hash of the WAR file or something, such that I
c
Hello Grant,
Not much good with Java profilers (yet!) so I thought I
would send a script!
Details... details! Having decided to produce a script to
replicate the 1.2 vis 1.3 speed problem. The required rigor
revealed a lot more.
1) The faster version I have previously referred to as 1.2,
On Nov 20, 2008, at 9:18 AM, Fergus McMenemie wrote:
Hello Grant,
Were you overwriting the existing index or did you also clean out the
Solr data directory, too? In other words, was it a fresh index, or
an
existing one? And was that also the case for the 22 minute time?
No in each case
Hello Grant,
>Were you overwriting the existing index or did you also clean out the
>Solr data directory, too? In other words, was it a fresh index, or an
>existing one? And was that also the case for the 22 minute time?
No in each case it was a new index. I store the indexes (the "data" d
Hi Fergus,
Were you overwriting the existing index or did you also clean out the
Solr data directory, too? In other words, was it a fresh index, or an
existing one? And was that also the case for the 22 minute time?
Would it be possible to profile the two instance and see if you notice
Hello,
I have a CSV file with 6M records which took 22min to index with
solr 1.2. I then stopped tomcat replaced the solr stuff inside
webapps with version 1.3, wiped my index and restarted tomcat.
Indexing the exact same content now takes 69min. My machine has
2GB of RAM and tomcat is running
27 matches
Mail list logo
