Look at the index with the Schema Browser in the Solr UI. This pulls
the terms for each field.
On Sun, Jul 15, 2012 at 8:38 PM, Giovanni Gherdovich
wrote:
> Hi all,
>
> are stopwords from the stopwords.txt config file
> supposed to be indexed?
>
> I would say no, but this is the situation I am
>
Ok i'm added the debug, there is the query from the response after executing
query :
facet=true,sort=publishingdate
desc,debugQuery=true,facet.mincount=1,q=service:1 AND
publicationstatus:LIVE,facet.field=pillar,wt=javabin,fq=(((pillar:10))),version=2}},response={numFound=2,start=0,docs=[SolrDocum
David,
Thanks for such a detailed response. The data volume I mentioned is the
total set of records we have - but we would never ever need to search the
entire base in one query; we would divide the data by region or zip code.
So, in that case I assume that for a single region, we would not have m
OK: that is helpful, thanks!
On 13 July 2012 15:44, Mark Miller wrote:
> It really comes down to you.
>
> Many people run a trunk version of Solr in production. Some never would.
> Generally, bugs are fixed quickly, and trunk is pretty stable. The main
> issue is index format changes and upgrade
Yes,
sorry Just a typo.
I meant
q=*:*&fq=&start=0&rows=10&qt=&wt=&explainOther=&fl=product:(if(show_product:true,
product, "")
thanks
On Sat, Jul 14, 2012 at 11:27 PM, Erick Erickson [via Lucene]
wrote:
> I think in 4.0 you can, but not 3.x as I remember. Your example has
> the fl as part
> of
On Mon, Jul 16, 2012 at 4:43 AM, maurizio1976
wrote:
> Yes,
> sorry Just a typo.
> I meant
> q=*:*&fq=&start=0&rows=10&qt=&wt=&explainOther=&fl=product:(if(show_product:true,
> product, "")
> thanks
Functions normally derive their values from the fieldCache... there
isn't currently a function t
"So you want to re-use same SQL sentence in many entities? "
Yes
is it necessary to deploy complete solr and lucene for this?
--
View this message in context:
http://lucene.472066.n3.nabble.com/DIH-include-Fieldset-in-query-tp3994798p3995228.html
Sent from the Solr - User mailing list archive at
Hi Giovanni,
you have entered the stopwords into stopword.txt file, right? But in the
definition of the field type you are referencing stopwords_FR.txt..
best regards,
Michael
On Mon, 16 Jul 2012 05:38:04 +0200, Giovanni Gherdovich
wrote:
> Hi all,
>
> are stopwords from the stopwords.txt conf
Hi all, thank you for your replies.
Lance:
> Look at the index with the Schema Browser in the Solr UI. This pulls
> the terms for each field.
I did it, and it was the first alarm I got.
After the indexing, I went on the schema browser hoping
to don't see any stopword in the top-terms, but...
they
Hi,
Is the any way to make grouping searches more efficient?
My queries look like:
/select?q=query&group=true&group.field=id&group.facet=true&group.ngroups=true&facet.field=category1&facet.missing=false&facet.mincount=1
For index with 3 mln documents query for all docs with group=true takes
almo
Okay... found the problem after some more debugging. I was using a wrong
datasource tag in the data-config.xml, may be Solr should validate the xml
against a schema so these kind of issues are caught upfront.
wrong: s*ource name="fieldSource" type="FieldReaderDataSource" />
correct: S*ource name=
Yes, This feature will solve the below problem very neatly.
All,
Is there any approach to achieve this for now?
--Rajani
On Sun, Jul 15, 2012 at 6:02 PM, Jack Krupansky wrote:
> The answer appears to be "No", but it's good to hear people express an
> interest in proposed features.
>
> -- Jac
You'll have to query the index for the fields and sift out the _s ones
and cache them or something.
On Mon, 2012-07-16 at 16:52 +0530, Rajani Maski wrote:
> Yes, This feature will solve the below problem very neatly.
>
> All,
>
> Is there any approach to achieve this for now?
>
>
> --Rajani
Andrew:
I'm not entirely sure that's your problem, but it's the first thing I'd try.
As for your config files, see the section "Replicating solrconfig.xml"
here: http://wiki.apache.org/solr/SolrReplication. That at least
allows you to centralize separate solrconfigs for master and
slave, maki
Ahhh, you need to look down another few lines. When you specify fq, there
should be a section of the debug output like
.
.
.
where the array is the parsed form of the filter queries. I was thinking about
comparing that with the parsed form of the "q" parameter in the non-filter
case to see
Hi Agnieszka ,
if you don't need number of groups, you can try leaving out
group.ngroups=true param.
In this case Solr apparently skips calculating all groups and delivers
results much faster.
At least for our application the difference in performance
with/without group.ngroups=true is significant
In this URL - https://issues.apache.org/jira/browse/SOLR-247
there are *patches *and one patch with name "*SOLR-247-FacetAllFields*"
Will that help me to fix this problem?
If yes, how do I add this to solr plugin ?
Thanks & Regards
Rajani
On Mon, Jul 16, 2012 at 5:04 PM, Darren Govoni
Hi Pavel,
I tried with group.ngroups=false but didn't notice a big improvement.
The times were still about 4000 ms. It doesn't solve my problem.
Maybe this is because of my index type. I have millions of documents but
only about 20 000 groups.
Cheers
Agnieszka
2012/7/16 Pavel Goncharik
> Hi
Michael,
Thanks for the response. Below is the stack trace.
Note: Our environment is 64 bit and the Initial Pool size is set to 4GB and
Max pool size is 12GB so it doesn't makes sense why it tries to allocate
24GB (even that is available as the total RAM is 64GB).
This issue doesn't come with SO
Hello, Bruno,
No, 4 simultaneous requests should not be a problem.
Have you checked the Tomcat logs or logged the data in the query
response object to see if there are any clues to what the problem
might be?
Michael Della Bitta
Appinions, Inc. --
samabhiK wrote
>
> David,
>
> Thanks for such a detailed response. The data volume I mentioned is the
> total set of records we have - but we would never ever need to search the
> entire base in one query; we would divide the data by region or zip code.
> So, in that case I assume that for a sin
On Jul 15, 2012, at 2:45 PM, Nick Koton wrote:
> I converted my program to use
> the SolrServer::add(Collection docs) method with 100
> documents in each add batch. Unfortunately, the out of memory errors still
> occur without client side commits.
This won't change much unfortunately - currentl
What are the RAM of your server and size of the data folder?
-Original Message-
From: Agnieszka Kukałowicz
To: solr-user
Sent: Mon, Jul 16, 2012 6:16 am
Subject: Re: Grouping performance problem
Hi Pavel,
I tried with group.ngroups=false but didn't notice a big improvement.
The time
Thinking more about this, the way to get a Lucene based system to scale to
the maximum extent possible for geospatial queries would be to get a
geospatial query to be satisfied by just one (usually) Lucene index segment.
It would take quite a bit of customization and work to make this happen. I
s
I have server with 24GB RAM. I have 4 shards on it, each of them with 4GB
RAM for java:
JAVA_OPTIONS="-server -Xms4096M -Xmx4096M"
The size is about 15GB for one shard (i use ssd disk for index data).
Agnieszka
2012/7/16
> What are the RAM of your server and size of the data folder?
>
>
>
> --
Thanks Erick,
I will look harder at our current configuration and how we're handling
config replication, but I just realized that a backup script was doing a
commit and an optimize on the slave prior to taking the backup. This
happens daily, after updates and replication from the master. This is
s
This is strange. We have data folder size 24Gb, RAM for java 2GB. We query
with grouping, ngroups and highlighting, do not query all fields and query
time mostly is less than 1 sec it rarely goes up to 2 sec. We use solr 3.6 and
tuned off all kind of caching.
Maybe your problem is with caching
Maybe try EdgeNgramFilterFactory
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters/#solr.EdgeNGramFilterFactory
On Mon, Jul 16, 2012 at 6:57 AM, santamaria2 wrote:
> I'm about to implement an autocomplete mechanism for my search box. I've
> read
> about some of the common approaches, b
Thank you,
I am already on 4alpha. Patch feels a little too unstable for my
needs/familiarity with the codes.
What about something around multiple cores? Could I have full-text
fields stored in a separate cores and somehow (again, minimum
hand-coding) do search against all those cores and get bac
Hello.
We are running Solr 3.5 multicore in master-slave mode.
-Our delta-import looks like:
/solr/core01/dataimport?command=delta-import&*optimize=false*
The size of the index in 1.18GB
When delta-import is going on, on the slave admin UI
8983/solr/core01/admin/replication/index.jsp
I can se
term component will be faster.
like below:
http://host:port/solr/terms?terms.fl=content&terms.prefix=sol
--
View this message in context:
http://lucene.472066.n3.nabble.com/Wildcard-query-vs-facet-prefix-for-autocomplete-tp3995199p3995378.html
Sent from the Solr - User mailing list archive at Nab
Hello Michael,
I will check the log, but today I think to another thing may be it's my
program that it losts some requests.
It's the first time where the download is so fast.
With Jetty, it's a little bit slower so may be for this reason my
program works fine.
Do you think I can use Jetty f
Hello Bruno,
Jetty is a legitimate choice. I do, however, worry that you might be
masking an underlying problem by making that choice, without a
guarantee that it won't someday hurt you even if you use Jetty.
A question: are you using a client to connect to Solr and issue your
queries? Something
Erick Erickson wrote
>
> Ahhh, you need to look down another few lines. When you specify fq, there
> should be a section of the debug output like
>
> .
> .
> .
>
>
> where the array is the parsed form of the filter queries. I was thinking
> about
> comparing that with the parsed form of
> That suggests you're running out of threads
Michael,
Thanks for this useful observation. What I found just prior to the "problem
situation" was literally thousands of threads in the server JVM. I have
pasted a few samples below obtained from the admin GUI. I spent some time
today using this ba
Any thought on this? Is the default Mmap?
Sent from my mobile device
720-256-8076
On Feb 14, 2012, at 7:16 AM, Bill Bell wrote:
> Does someone have an example of using unmap in 3.5 and chunksize?
>
> I am using Solr 3.5.
>
> I noticed in solrconfig.xml:
>
> class="${solr.directoryFactory:
Hi Bill,
Standard picks one for you. Otherwise, you can hardcode the
DirectoryFactory in your config file, or I believe if you specify
-Dsolr.solr.directoryFactory=solr.MMapDirectoryFactory
That will get you what you want.
Michael Della Bitta
Ap
Hi,
Our index is divided into two shards and each of them has 120M docs , total
size 75G in each core.
The server is a pretty good one , jvm is given memory of 70G and about same
is left for OS (SLES 11) .
We use all dynamic fields except th eunique id and are using long queries
but almost all of
5 min is ridiculously long for a query that used to take 65ms. That ought
to be a great clue. The only two things I've seen that could cause that
are thrashing, or GC. Hard to see how it could be thrashing, given your
hardware, so I'd initially suspect GC.
Aim VisualVM at the JVM. It shows how muc
We all know that MMapDirectory is fastest. However we cannot always
use it since you might run out of memory on large indexes right?
Here is how I got iSimpleFSDirectoryFactory to work. Just set
-Dsolr.directoryFactory=solr.SimpleFSDirectoryFactory.
Your solrconfig.xml:
You can check it with h
Yep.
-Dsolr.directoryFactory=solr.SimpleFSDirectoryFactory
or
-Dsolr.directoryFactory=solr.MMapDirectoryFactory
works great.
On Mon, Jul 16, 2012 at 7:55 PM, Michael Della Bitta
wrote:
> Hi Bill,
>
> Standard picks one for you. Otherwise, you can hardcode the
> DirectoryFactory in your confi
Thanks Brian. Excellent suggestion.
I haven't used VisualVM before but I am going to use it to see where CPU is
going. I saw that CPU is overly used. I haven't seen so much CPU use in
testing.
Although I think GC is not a problem, splitting the jvm per shard would be
a good idea.
On Mon, Jul 16,
Another thing you may wish to ponder is this blog entry from Mike
McCandless:
http://blog.mikemccandless.com/2011/04/just-say-no-to-swapping.html
In it, he discusses the poor interaction between OS swapping, and
long-neglected allocations in a JVM. You're on Linux, which has decent
control over sw
Hello
According to release notes from 4.0.0-ALPHA, SOLR-2396, I replaced
ICUCollationKeyFilterFactory with ICUCollationField in our schema. But this throws an
exception, see the following excerpt from the log:
Jul 16, 2012 5:27:48 PM org.apache.solr.common.
45 matches
Mail list logo
