Re: SolrCloud and exernal file fields

On Wed, Nov 21, 2012 at 7:08 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > On Wed, Nov 21, 2012 at 2:07 AM, Martin Koch wrote: > > > I'm not sure about the mmap directory or where that > > would be configured in solr - can you explain that? > > > > You can check it at Solr Admin/St

RE: Reduce QueryComponent prepare time

Hi Mikhail, Thanks for sharing your experiences. I'll look into the flexible query parser. Markus -Original message- > From:Mikhail Khludnev > Sent: Tue 20-Nov-2012 19:53 > To: solr-user@lucene.apache.org > Subject: Re: Reduce QueryComponent prepare time > > Markus, > > It seems yo

Re: user session id / cookie to record search query

Record? E.g. output the cookie value of a given name in the log? Provided you use Apache mod_proxy, we do this by a special log-format. paul Le 21 nov. 2012 à 09:50, Romita Saha a écrit : > Hi All, > > Do anyone have an idea how to use user session id / cookie to record > search query from th

Re: user session id / cookie to record search query

Hello! You want it to be written into logs ? If that is the case you can just add additional parameter, that is not recognized by Solr, for example 'userId' and send a query like this: http://localhost:8983/solr/select?q=*:*&userId=user1 In the logs you should see something like that: INFO: [col

Re: Solr defining Schema structure trouble.

isn't it possible to combine the document related values and page related values at query time? Book1 Page1 with ref to book1 Page2 with ref to book2 When querying making all pages (page1+book1) and (page2+book1) Or would this be hard to achieve. I'm pretty sure they wan't to search on book rel

Re: user session id / cookie to record search query

Hello Rafał Kuć Thanks a lot for you guidance. I am not quite sure how to i collect the logs. Could you please help. Romita From: Rafał Kuć To: solr-user@lucene.apache.org, Date: 11/21/2012 04:57 PM Subject:Re: user session id / cookie to record search query Hello! You

Re: SolrCloud and exernal file fields

On Wed, Nov 21, 2012 at 11:53 AM, Martin Koch wrote: > > I wasn't aware until now that it is possible to send a commit to one core > only. What we observed was the effect of curl > localhost:8080/solr/update?commit=true but perhaps we should experiment > with solr/coreN/update?commit=true. A quic

Re: user session id / cookie to record search query

Hello! What Solr are you using ? If not 4.0, information on logging can be found on wiki - http://wiki.apache.org/solr/SolrLogging -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > Hello Rafał Kuć > Thanks a lot for you guidance. I am not quite

Recip m parameter to take function value

Hi, We need the recip function's m-parameter to take other functions e.g. recip(dateField, div(1,prod(1,2)), 1,1) but ValueSourceParser want to read a float instead. How could we modifiy either Solr or Lucene as well to take functions for that parameter? I've been looking at the various extende

Re: user session id / cookie to record search query

Hi, Thanks a lot. Will follow the same. Thanks and regards, Romita From: Rafał Kuć To: solr-user@lucene.apache.org, Date: 11/21/2012 05:34 PM Subject:Re: user session id / cookie to record search query Hello! What Solr are you using ? If not 4.0, information on logging c

From Solr3.1 to SolrCloud

hello, We are using solr 3.1 for searching on our webpage right now. We want to use the nice features of solr 4: realtime search. Our current configuration looks like this: Master Slave1 Slave2 Slave3 We have 3 slaves and 1 master and the data is replication every night. In the future we want to

Re: Best way to retrieve 20 specific documents

On Tue, Nov 20, 2012 at 12:45 AM, Shawn Heisey wrote: > You can also use this query format: > > id:(123 OR 456 OR 789) > > This does get expanded internally by the query parser to the format that has > the field name on every clause, but it is sometimes easier to write code > that produces the abo

Re: Replication Backup

Hi Otis, It seems to me that I'm going to have to write a script anyway that takes handles the retention of the backups. Plus it doesn't seem optimal that I would run a solr instance on that server, taking up memory when I could probably write a script that would pull all the data directly using t

Re: [Solrj] How can I get unique field name?

That's right. Thank you Jack. Cordialement. __ Zakaria BENZIDALMAL mobile: 06 31 40 04 33 2012/11/20 Jack Krupansky > There is no absolute requirement that a Solr schema have a unique key > field, so you could get a null value for the field. > > -- Jack Krupansky > > -O

Re: From Solr3.1 to SolrCloud

> > - We aren't using shards because our index only contains 1 mil simple docs. > We only need multiple server because the amount of traffic. In the examples > of solrCloud i see only examples with shards. Is numshards=1 possible? One > big index is faster than multiple shards? I need 1 collection

Re: SolrCloud and exernal file fields

Mikhail, PSB On Wed, Nov 21, 2012 at 10:08 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > On Wed, Nov 21, 2012 at 11:53 AM, Martin Koch wrote: > > > > > I wasn't aware until now that it is possible to send a commit to one core > > only. What we observed was the effect of curl > > l

Single Tomcat Multiple Shards

Hey Guys, We are experimenting with solr cloud, this is what we want to set up as, 2 Machines each having have 8 master shards, so total of 16 shards. The assumption is we want to store approximately 4-5 TB data over a period of 1 year of so. Replication factor of 1 which are again distributed

SolrCloud and external Zookeeper ensemble

Hi, I have 4 solr collections, 2-3mn documents per collection, up to 100K updates per collection daily (roughly). I'm going to create SolrCloud4x on Amazon's m1.large instances (7GB mem,2x2.4GHz cpu each). The question is what about zookeeper? It's going to be external ensemble, but is it better t

Re: From Solr3.1 to SolrCloud

Thanks Tomás, I will use numshards=1. Are there some instructions on how to install only zookeeper on a separate server? Or do i have to install solr 4 on that server? How make the connection between the solr instances and the zk instance(server)? Thanks so far, Roy -- View this message in

[SolrCloud] is softcommit cluster-wide for the collection ?

Hello, We're working on integrating SolrCloud andwe're wondering whether issuing a softCommit via Solrj forces the soft commit : a) only on the receiving core or b) to the whole cluster and the receiving cores forwards the soft commit to all replicas. If the answer is a), what is the best prac

solr autocomplete

Hi all, I'am using the apache-solr4.0.0, and the autocomplete feature. In general it works fine, but I still have two problems which I can't solve. In general I need the autocomplete to show a movie titles. 1. The first thing is that the autocomplete search ignores all characters after the space.

Re: From Solr3.1 to SolrCloud

> > I will use numshards=1. Are there some instructions on how to install only > zookeeper on a separate server? Or do i have to install solr 4 on that > server? > You don't need to install Solr in that server. See http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html > > How make the conn

Solr 4 Admin UI Dashboard Not Populating

Our Admin UI Dashboard is not populating on one of our servers, not sure if it is a permission issue or what. We have three others that it is working on. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-

Re: Copying few field using copyField to non multiValued field

i would also like to copy a few fields to a single-valued field. my reasoning for this is to then perform exact-match search on the concatenated field. e.g. full_name = first_name + last+name i would then like to search: full_name:"john foo" but copyField is making the full_name field look like

Writing SOLR custom search component to search SOLR to fetch more documents

I have stored following documents in my solr schema.Just to tell in brief about the schema, the schema is about teacher, course and their relationship.Example data :|ID| Name| Type| FromID | ToID ||1 | t1 | Teacher ||||2 | t2 | Teacher ||||3 | c1 | Course ||

Pls help: Very long query - what to do?

my query is like this, see below. I use already POST request. i got a solr exception: org.apache.solr.client.solrj.SolrServerException: Server at http://server:7056/solr returned non ok status:400, message:Bad Request is there a way in order to prevent this? id:("ModuleImpl@20117" OR "ModuleImpl

Re: Pls help: Very long query - what to do?

Hi, you have to set maxHttpHeaderSize of the element in server.xml. The default is something about 8K. See it with more detail: http://serverfault.com/questions/56691/whats-the-maximum-url-length-in-tomcat Regards, Péter Király portal backend developer http://europeana.eu 2012/11/21 uwe72 : > m

Re: Pls help: Very long query - what to do?

Hello! If you really need a query that long, than one of the things is increase the allowed header length in Jetty. Add the following to your Jetty connector configuration: 16384 -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > my query is lik

Re: Pls help: Very long query - what to do?

i have already: -- View this message in context: http://lucene.472066.n3.nabble.com/Pls-help-Very-long-query-what-to-do-tp4021606p4021619.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Pls help: Very long query - what to do?

Hello, Do not forget to increase maxBooleanClauses. Regards, - Luis Cappa. 2012/11/21 uwe72 > i am using tomcat > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Pls-help-Very-long-query-what-to-do-tp4021606p4021620.html > Sent from the Solr - User mailing list

Re: [SolrCloud] is softcommit cluster-wide for the collection ?

On Nov 21, 2012, at 9:11 AM, GIROLAMI Philippe wrote: > Hello, > We're working on integrating SolrCloud andwe're wondering whether issuing a > softCommit via Solrj forces the soft commit : > > a) only on the receiving core or > b) to the whole cluster and the receiving cores forwards the sof

Re: SolrCloud and external Zookeeper ensemble

Separate is generally nice because then you can restart Solr nodes without consideration for ZooKeeper. Performance-wise, I doubt it's a big deal either way. - Mark On Nov 21, 2012, at 8:54 AM, Marcin Rzewucki wrote: > Hi, > > I have 4 solr collections, 2-3mn documents per collection, up to

Re: Using SolrCloud for update often lose response and get 503 error

Have you looked at the logs? - Mark On Nov 21, 2012, at 1:07 AM, Qun Wang wrote: > Hello, > > Does anyone get the error of 503 when update by using SolrCloud? In my test I > found that if update too frequency Solr often get 503 error and all servers > inaccessible. Could someone provide any

RE: [SolrCloud] is softcommit cluster-wide for the collection ?

Hi Mark, Thanks for the details >> If the answer is b), what would happen on a 1-replica setup if one commit >> succeeded and the replica commit failed ? >What's the reason the commit failed? Either a really bad problem and that node >will need to be restarted and either won't answer requests or

Re: Out Of Memory =( Too many cores on one server?

On 11/21/2012 12:36 AM, stockii wrote: okay. i will try out more RAM. i am using not much caching because of "near-realt-time"-search. in this case its better to increase xmn or only xmx and xms? I have personally found that increasing the size of the young generation (Eden) is beneficial to

Re: [SolrCloud] is softcommit cluster-wide for the collection ?

On Nov 21, 2012, at 11:00 AM, GIROLAMI Philippe wrote: > Hi Mark, > Thanks for the details >>> If the answer is b), what would happen on a 1-replica setup if one commit >>> succeeded and the replica commit failed ? >> What's the reason the commit failed? Either a really bad problem and that

Re: SolrCloud and external Zookeeper ensemble

Hello! Zookeeper by itself is not demanding, but if something happens to your nodes that have Solr on it, you'll loose ZooKeeper too if you have them installed side by side. However if you will have 4 Solr nodes and 3 ZK instances you can get them running side by side. -- Regards, Rafał Kuć S

Re: Single Tomcat Multiple Shards

On Nov 21, 2012, at 8:32 AM, Cool Techi wrote: > Hey Guys, > > We are experimenting with solr cloud, this is what we want to set up as, > > 2 Machines each having have 8 master shards, so total of 16 shards. The > assumption is we want to store approximately 4-5 TB data over a period of 1 >

group.facet=true performances

Hi, I'm trying to use field collapsing, and i'm facing performance issue where using group.facet=true. On a small index (100.000 documents), query with group=true and group.facet=false takes 20ms, and group.facet=true take 800ms Maybe i miss some configuration option? Best regards, Mickael

Re: Out Of Memory =( Too many cores on one server?

> I have personally found that increasing the size of the young generation > (Eden) is beneficial to Solr, I've seen the same thing - I think it's because requests create a lot of short lived objects and if the eden is not large enough, a lot of those objects will make it to the tenured space, wh

Re: Pls help: Very long query - what to do?

Check the Solr log to see what the actual error (Solr vs. SolrJ) message was. -- Jack Krupansky -Original Message- From: uwe72 Sent: Wednesday, November 21, 2012 10:31 AM To: solr-user@lucene.apache.org Subject: Pls help: Very long query - what to do? my query is like this, see below

Re: Pls help: Very long query - what to do?

On 11/21/2012 8:53 AM, Luis Cappa Banda wrote: Do not forget to increase maxBooleanClauses. I believe this is the culprit right here. I counted 1576 instances of "OR" in the long query, which is rather a lot higher than the default maxBooleanClauses value of 1024. I think that the maxBoole

Re: Solr 4 Admin UI Dashboard Not Populating

Richard >From what i see from the Screen, the Javascript stopped executing because of >an error .. my first guess would be, that if you request >"http://solr-host:port/solr/production/admin/system?wt=json"; manually - you'll >not see a "host" property in the "core"-object, right? Normally that

Re: Pls help: Very long query - what to do?

You can increase that limit in your solrconfig.xml: 1024 Don't go wild with it, but upping it to 2000 or 5000 shouldn't be a big deal considering that hardware performance has increased significantly since Lucene was started. -- Jack Krupansky -Original Message- From: Shawn

Suggester for numbers

Hello guys, Please i need help.. im using the suggest search component for autocomplete in Solr 3.6.1, i have an autocomplete field wich contains two other fields: an conteiner_name and conteiner_id just like this: When i search for username in my suggester handler i

Re: Pls help: Very long query - what to do?

Yes it works when i increase the maxBooleanClauses But any case i have to think how i redesign the document structure. i have big problems do the relations between documents. also a document can be changed, then i have to update many documents which has a relation to the modified one. -- View

Re: Solr 4 Admin UI Dashboard Not Populating

I was able to figure it out, I ran solr/admin/system?wt=xml and noticed that the host entry was blank. Our servers are Linux so I looked at /etc/hosts file and noticed it was messed up. I made the change and everything is populating now. -- View this message in context: http://lucene.472066.n

Re: Solr defining Schema structure trouble.

You could implement a custom search component that takes the pages found by the query and then re-queries to find the book-level documents and adds them to the search results. Or, you could even have a query/parameter that found the pages but then discarded them and only kept the book metadata.

Re: Pls help: Very long query - what to do?

My design is like this at the moment: Documents in general has a relation to each other. So, a document has a id, some attributes and a multivalue-field "navigateTo". E.g. Document1: id1, some attributes, naviagteToAllDocumentsWhenColor:red, navigateTo: id2, id3 Document2: id2, some attribute

Re: Inconsistent search results.

Try the Solr Admin Analysis page and see how your failing examples analyze for both index and query. Also, if you experiment with analyzer settings, be sure to FULLY reindex your documents since a mismatch between how the documents were ORIGINALLY analyzed and the latest query analysis can cau

Re: SolrCloud and exernal file fields

Hi Martin, thanks for sharing your experience with EFF and saving me a lot of time figuring it out myself, I was afraid of exactly this kind of problems. Mikhail, thanks for expanding the thread with even more useful informations! Simone 2012/11/20 Martin Koch > Solr 4.0 does support using EF

Re: Inconsistent search results.

Hello! I suggest you to try PatternTokenizer with a regex that includes "." and blank spaces, for example, in Query and Index analyzers for that fieldType. The expression will be tokenized by that regex expression and you will success querying. Unfortunately, you will have to reindex all if you ch

Re: Weird Behaviour on Solr 5x (SolrCloud)

I'm not sure - I guess I'll have to look into it - could you file a JIRA issue with these details? - Mark On Wed, Nov 21, 2012 at 1:19 AM, deniz wrote: > well... i find a way to avoid this... i dont know if it is the correct way or > i am simply bypassing the problem instead of fixing it.. > > w

Re: Pls help: Very long query - what to do?

Uwe, Do you think BlockJoin can help you http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html? Do your docs forms parent-child blocks? How often you need to reindex particular doc? On Wed, Nov 21, 2012 at 9:54 PM, uwe72 wrote: > My design is like this at the moment: >

Re: is there a way to prevent abusing rows parameter

Does that 'someone' has direct access to Solr endpoint? Is that a right thing to do in a first place? But assuming they do (e.g. intranet), you could build on Jack's suggestion and create a couple of query-handler end-points that are only different in invariant raw count value. So, your default se

Re: SolrCloud and exernal file fields

Martin, I don't think solrconfig.xml shed any light on. I've just found what I didn't get in your setup - the way of how to explicitly assigning core to collection. Now, I realized most of details after all! Ball is on your side, let us know whether you have managed your cores to commit one by one

Re: SolrCloud and external Zookeeper ensemble

First of all: thank you for your answers. Yes, I meant side by side configuration. I think the worst case for ZKs here is to loose two of them. However, I'm going to use 4 availability zones in same region so at least this will reduce the risk of loosing both of them at the same time. Regards. On

Re: Solr 4 Admin UI Dashboard Not Populating

Glad it worked Richard, i've created an issue anyway: https://issues.apache.org/jira/browse/SOLR-4102 On Wednesday, November 21, 2012 at 6:40 PM, richardg wrote: > Thanks Stefan host was the issue, I responded to my post before I saw yours. > > > > -- > View this message in context: > http

Using payloads to encode part-of-speech in Solr 4.0.0

Dear list members, I am trying to figure out how to configure schema.xml in solr 4.0.0 so that it takes into account part-of-speech (PoS) tags to index documents and filter queries, all of it by using payloads. The schema.xml file includes a payloads field in Solr 4.0.0. From the comments I have

Re: SolrCloud and external Zookeeper ensemble

Hello! As I told I wouldn't use the Zookeeper that is embedded into Solr, but rather setup a standalone one. -- Regards, Rafał Kuć Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > First of all: thank you for your answers. Yes, I meant side by side > configuration.

Re: SolrCloud and external Zookeeper ensemble

Yes, I meant the same (not -zkRun). However, I was asking if it is safe to have zookeeper and solr processes running on the same node or better on different machines? On 21 November 2012 21:18, Rafał Kuć wrote: > Hello! > > As I told I wouldn't use the Zookeeper that is embedded into Solr, but >

Partial results with not enough hits

In all of my queries I have timeAllowed parameter. My application is ready for partial results. However, whenever Solr returns partial result it is a very bad result. For example, I have a test query and here its execution log with the strict time allowed: WARNING: Query: ; Elapsed time: 1

Re: Partial results with not enough hits

It could be that the time to get set up to return even the first result is high and then each additional document is a minimal increment in time. Do a query with &rows=1 (or even 0) and see what the minimum query time is for your query, index, and environment. -- Jack Krupansky -Original

Re: Weird Behaviour on Solr 5x (SolrCloud)

Mark Miller-3 wrote > I'm not sure - I guess I'll have to look into it - could you file a > JIRA issue with these details? sure... but before that could it be because of using RAM dir? because basically when you restart solr the ram is gone and it tries to checks the old folder that it had used..

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

after putting the port information to solr.xml too, it seems properly... i dont know why this thing only happens on remote machines not on local, but could this be a minor bug related with solr? basically if we are giving the port information in the starting command, then we shouldnt be dealing wit

How to use CloudSolrServer in multi threaded indexing program

I am a newbie to SolrCloud. I have setup a SolrCloud of n leaders, n replicas and a zookeeper ensemble. I have a client that uses SolrJ and has access to millions of docs. This client program runs on a separate machine. Since I want these docs to be indexed as fast as possible, I would like to spa

Re: SolrCloud(5x) - Detects all of the Solr insrances on a machine

Limitation of web containers. There is not a clean way to get the port without making some request. If you pass the port as a sys prop on the cmd line and use jetty, it works out of the box. If you don't do that, there is config necessary. - Mark On Nov 21, 2012, at 8:34 PM, deniz wrote: > a