speaking about pagesizes, what is the optimum page size that should be
retrieved each time ??
i understand it depends upon the data you are fetching back fromeach hit
document ... but lets say when ever a document is hit am fetching back 100
bytes worth data from each of those docs in indexes (alon
There isn't an "optimum" page size that I know of, it'll vary with lots of
stuff, not the least of which is whatever servlet container limits there are.
But I suspect you can get quite a few (1000s) without
too much problem, and you can always use the JSON response
writer to pack in more pages wit
Though I think you could get the Dedupe feature to do this:
http://wiki.apache.org/solr/Deduplication
On Aug 13, 2011, at 11:52 , Erick Erickson wrote:
> If you mean just throw the new document on the floor if
> the index already contains a document with that key, I don't
> think you can do tha
thanks erick ... that means it depends upon the memory allocated to the JVM
.
going back queryCacheResults factor i have got this doubt ..
say, i have got 10 threads with 10 different queries ..and each of them in
parallel are searching the same index with millions of docs in it
(multisharded ) .
As many results will be cached as you ask. See solrconfig.xml,
the queryResultCache. This cache is essentially a map of queries
and result document IDs. The number of doc IDs cached for
each query is controlled by queryResultWindowSize in
solrconfig.xml
Best
Erick
On Sun, Aug 14, 2011 at 8:35 AM,
You either have to go to near real time (NRT), which is under
development, but not committed to trunk yet or just stop
warming up searchers and let the first user to open a searcher
pay the penalty for warmup, (useColdSearchers as I remember).
Although I'd also ask whether this is a reasonable req
my queryResultCache size =0 and queryResultWindowSize =50
does this mean that am not caching any results ?
On 14 August 2011 18:27, Erick Erickson wrote:
> As many results will be cached as you ask. See solrconfig.xml,
> the queryResultCache. This cache is essentially a map of queries
> and res
On Aug 14, 2011, at 9:03 AM, Erick Erickson wrote:
> You either have to go to near real time (NRT), which is under
> development, but not committed to trunk yet
NRT support is committed to trunk.
- Mark Miller
lucidimagination.com
Hi All,
I am trying to group by results using SolrJ. According to
https://issues.apache.org/jira/browse/SOLR-2637 the feature was added, so I
upgraded to SolrJ-3.4-Snapshot and I can see the necessary method for
grouping in QueryResponse, which is getGroupResponse(). The only thing left
that I don
Naveen:
You should try NRT with Apache Solr 3.3 and RankingAlgorithm. You can
update 10,000 documents / sec while also concurrently searching. You can
set commit freq to about 15 mins or as desired. The 10,000 document
update performance is with the MBArtists index on a dual core Linux
syste
Hi Omri,
SOLR-2637 was concerned with adding grouped response parsing. There is no
convenience method for grouping, but you can use the normal
SolrQuery#set(...) methods to enable grouping.
The following code should enable grouping via SolrJ api:
SolrQuery query = new SolrQuery();
query.set(GroupP
Thanks a lot!! Exactly what I was looking for..
That solved this!
On Sun, Aug 14, 2011 at 6:23 PM, Martijn v Groningen <
martijn.v.gronin...@gmail.com> wrote:
> Hi Omri,
>
> SOLR-2637 was concerned with adding grouped response parsing. There is no
> convenience method for grouping, but you can us
It is nothing special - just like this:
conn = Solr::Connection.new("http://#{LOCAL_SHARD}";,
{:timeout => 1000, :autocommit => :on})
options[:shards] = HA_SHARDS
response = conn.query(query, options)
Where LOCAL_SHARD points to a haproxy of a single shard and HA_SHARDS
Yep.
ResultWindowSize in
>> solrconfig.xml
>>
>> Best
>> Erick
>>
>> On Sun, Aug 14, 2011 at 8:35 AM, jame vaalet wrote:
>> > thanks erick ... that means it depends upon the memory allocated to the
>> JVM
>> > .
>> >
>> > going back queryCacheResults factor i have got this doubt ..
>> > say, i ha
Ah, thanks, Mark... I must have been looking at the wrong JIRAs.
Erick
On Sun, Aug 14, 2011 at 10:02 AM, Mark Miller wrote:
>
> On Aug 14, 2011, at 9:03 AM, Erick Erickson wrote:
>
>> You either have to go to near real time (NRT), which is under
>> development, but not committed to trunk yet
>
>
Hi Mark/Erick/Nagendra,
I was not very confident about NRT at that point of time, when we started
project almost 1 year ago, definitely i would try NRT and see the
performance.
The current requirement was working fine till we were using commitWithin 10
millisecs in the XMLDocument which we were p
It's somewhat confusing - I'll straighten it out though. I left the issue open
to keep me from taking forever to doc it - hasn't helped much yet - but maybe
later today...
On Aug 14, 2011, at 12:12 PM, Erick Erickson wrote:
> Ah, thanks, Mark... I must have been looking at the wrong JIRAs.
>
>
On 8/13/2011 9:59 AM, Michael Sokolov wrote:
Shawn, my experience with SolrJ in that configuration (no autoCommit)
is that you have control over commits: if you don't issue an explicit
commit, it won't happen. Re lifecycle: we don't use a static
instance; rather our app maintains a small poo
Does instantiating a Solr::Connection for each request make things better?
Erik
On Aug 14, 2011, at 11:34 , Ian Connor wrote:
> It is nothing special - just like this:
>
>conn = Solr::Connection.new("http://#{LOCAL_SHARD}";,
> {:timeout => 1000, :autocommit => :on})
>optio
It's worth noting that the fast commit rate is only an indirect part
of the issue you're seeing. As the error comes from cache warming - a
consequence of committing, it's not the fault of commiting directly.
It's well worth having a good close look at exactly what you're caches
are doing when they
Naveen:
NRT with Apache Solr 3.3 and RankingAlgorithm does need a commit for a
document to become searchable. Any document that you add through update
becomes immediately searchable. So no need to commit from within your
update client code. Since there is no commit, the cache does not have
How do you setup log4j to work with Loggly for SOLR logs?
Anyone have this set up?
Bill
OK,
I'll ask the elephant in the roomÂ.
What is the difference between the new UpdateHandler from Mark and the
SOLR-RA?
The UpdateHandler works with 4.0 does SOLR-RA work with 4.0 trunk?
Pros/Cons?
On 8/14/11 8:10 PM, "Nagendra Nagarajayya"
wrote:
>Naveen:
>
>NRT with Apache Solr 3.3 and Ra
Bill:
The technical details of the NRT implementation in Apache Solr with
RankingAlgorithm (SOLR-RA) is available here:
http://solr-ra.tgels.com/papers/NRT_Solr_RankingAlgorithm.pdf
(Some changes for Solr 3.x, but for most it is as above)
Regarding support for 4.0 trunk, should happen someti
OK. But SOLR has built-in caching. Do you not like the caching? What so
you think we should change to the SOLR cache?
Bill
On 8/10/11 9:16 AM, "didier deshommes" wrote:
>Consider putting a cache (memcached, redis, etc) *in front* of your
>solr slaves. Just make sure to update it when replicati
I understand.
Have you looked at Mark's patch? From his performance tests, it looks
pretty good.
When would RA work better?
Bill
On 8/14/11 8:40 PM, "Nagendra Nagarajayya"
wrote:
>Bill:
>
>The technical details of the NRT implementation in Apache Solr with
>RankingAlgorithm (SOLR-RA) is avai
Bill:
I did look at Marks performance tests. Looks very interesting.
Here is the Apacle Solr 3.3 with RankingAlgorithm NRT performance:
http://solr-ra.tgels.com/wiki/en/Near_Real_Time_Search_ver_3.x
Regards
- Nagendra Nagarajayya
http://solr-ra.tgels.org
http://rankingalgorithm.tgels.org
On
Regarding point b, i mean that when Slave server does a replication from
Master, it creates a lock-file in it's index directory. How to avoid that?
On Tue, Aug 9, 2011 at 2:56 AM, Markus Jelsma wrote:
> Hi,
>
> > Hi
> >
> > I am using Solr 1.4. and doing a replication process where my slave is
>
Hi All,
I am looking for a solution to filter out text which contains non english
words. Where my goal is to present my english speaking users with results in
their language.
any ideas?
thanks
Omri
29 matches
Mail list logo
