Re: Not getting results when searching a term from Solr Admin

Hi Jack, Thanks for the quick response. Actually i have configured Apache Nutch 1.6 along with Apache Solr 4.3.0 After crawling a website(For example www.bing.com) I use the following command to do a Solr Index bin/nutch solrindex http://127.0.0.1:8983/solr/ crawl/crawldb -linkdb crawl/linkdb cra

Re: reg: efficient querying using solr

This might help (indirectly): http://svn.apache.org/repos/asf/lucene/dev/trunk/dev-tools/size-estimator-lu cene-solr.xls From: "gururaj kosuru" Sent: Wednesday, June 12, 2013 12:28 AM To: "solr-user" Subject: Re: reg: efficient querying using solr Thank

Re: Are there any plans to change example directory layout?

If you have a good idea... Just do it. Open an issue On Jun 11, 2013 9:34 PM, "Alexandre Rafalovitch" wrote: > I think it is quite hard for beginners that basic solr example > directory is competing for attention with other - nested - examples. I > see quite a lot of questions on which directory

Are there any plans to change example directory layout?

I think it is quite hard for beginners that basic solr example directory is competing for attention with other - nested - examples. I see quite a lot of questions on which directory inside 'example' to pay attention to and which to ignore, etc. Actually, this is so confusing, I am not even sure ho

Re: reg: efficient querying using solr

Thanks Walter, Shawn and Otis for the assistance, I will look into tuning the parameters by experimenting as seems to be the only way to go. On 11 June 2013 19:17, Shawn Heisey wrote: > On 6/11/2013 12:15 AM, gururaj kosuru wrote: > > How can one calculate an ideal max shard size for a solr cor

Re: upgrading 1hr autoCommit behavior

: > However, we are wondering how to best setup autoCommit/autoSoftCommit on : > masters to preserve the old behavior. It seems that setting autoCommit to : > 1hr (openSearcher=true) without any autoSoftCommit preserves our previous : > setup - is this correct? Wil the transaction log make masters

Re: Best way to concatenate 2 array pairs - DIH?

Ok I wrote a custom Java transformer as below. Can someone confirm if this is the right way? import java.util.ArrayList; import java.util.List; import java.util.Map; import org.apache.solr.handler.dataimport.Context; import org.apache.solr.handler.dataimport.DataImporter; import org.apache.solr.h

Re: solr facet query on multiple search term

: Actual requirement is to get day wise total no. of counts for multiple : terms. the approach you described in your original mail (using multiple fqs to identify your terms, and using multiple facet.range requests on the date field with differnet exclusions) will work fine. the confusing part

Re: Error indexing string field

: I have a field declared as type string, so should it care whats inside the string? : : Caused by: java.lang.NumberFormatException: For input string: "1835-1910." you haven't given us any information we can use to help you... schema? high level error that wrapped that NFE? full stack trace of

Re: Does from term in join query undergo query time analysis?

: ...&q={!join+from%3DconceptId+to%3Did+fromIndex%3DSearchProducts0}parentProductId:PROD_Hs01_00179501 : But, I have a client who asked if it would be possible for the parent : product ID to be specified in a case insensitive manner, with or w/o the : underbar characters. So, I figured I coul

ConstantScoreQuery

Hi, I have imported the money.xml doc in lucene - https://github.com/normann/apache-solr/blob/master/example/exampledocs/money.xml I tried the query: price_1:1 and got back a result containing only USD as the search hit. But the money.xml has Euro, pound and NOK too. Do we know why they are not r

Re: upgrading 1hr autoCommit behavior

On 6/11/2013 4:07 PM, Michael Tsadikov wrote: Thanks for the quick reply, Shawn. I am less worried about long transaction-log replays after crashes because in normal life (long uptimes, rare orderly restarts) this should never happen, and even if it does, uses will not be affected. I am more wo

Error indexing string field

I have a field declared as type string, so should it care whats inside the string? Caused by: java.lang.NumberFormatException: For input string: "1835-1910." Thanks -Peri

Re: upgrading 1hr autoCommit behavior

Thanks for the quick reply, Shawn. I am less worried about long transaction-log replays after crashes because in normal life (long uptimes, rare orderly restarts) this should never happen, and even if it does, uses will not be affected. I am more worried about potential increase in heap usage due

Re: upgrading 1hr autoCommit behavior

On 6/11/2013 3:45 PM, Michael Tsadikov wrote: We're upgrading our Solr 3.1 distributed masters/slaves setup to 4.3. In 3.1 we used autoCommit every hour on masters, each commit is replicated to slaves, and all searches are done on slaves. 1hr visibility is ok - we don't need NRT. In 4.3 we enab

Re: Filtering of results based on associated field values

Give us an example of one of your multivalued fields. The semantics of a query or filter query on a multivalued field are simply does ANY of the multiple values pass the query/filter conditions. If so, if ANY matched, the document is a match. If none of the values in the multiple values passes

Best way to concatenate 2 array pairs - DIH?

I am trying to combine latitude and longitude data extracted from text file using data import handler.. One document can contain multiple latitudes / longitudes... My data would be of format [lat,lat], [long,long] Example: [33.7209548950195, 34.474838],[-117.176193237305, -117.573463] I am cur

upgrading 1hr autoCommit behavior

We're upgrading our Solr 3.1 distributed masters/slaves setup to 4.3. In 3.1 we used autoCommit every hour on masters, each commit is replicated to slaves, and all searches are done on slaves. 1hr visibility is ok - we don't need NRT. In 4.3 we enable transaction logs, for durability in case of c

Filtering of results based on associated field values

I have some unusual requirements for a Solr 4.3 based search service I am developing: * I am required to filter the results by a set of fields, some of which are multivalued * Each 'filter' field value has an associated begin and end date/time field, in the case of the multivalued fields, different

Re: Not getting results when searching a term from Solr Admin

Sorry, but a Solr search does not return indexed terms - it returns documents and displays stored (not indexed) values. Please state more specifically what the actual problem is. Have you verified that the field you are searching in actually has the query terms indexed? -- Jack Krupansky --

Not getting results when searching a term from Solr Admin

Hello All, I have configured Apache Solr 4.3.0 When i search for *.* then i get all the indexed term. But when i searh for something specific then i dont get any result. Here is my schema.xml file schema.xml Please suggest me. Rega

Re: How to ignore folder collection1 when running single instance of SOLR?

What you are doing by removing solr.xml is reverting to the old Solr 3.x 'single core' setup. Erick is suggesting that this is best considered deprecated, and will make life harder for you with future releases. If you don't like the reference to 'collection1', rename it to something else. Stop Sol

statistics about segments

In the Solr UI, there is the number of segments in the core. Is there information in Solr (in the UI or in xml responses) about their histogram? I.e., their size, how many are there in each size level? Are there any that are currently being merged, etc.? Thanks, Yoni Confidentiality: This commun

Re: Filtering down terms in suggest

Aloke, If you do not have a factorial problem in the combination of userid and groupid (which I can imagine you might) you could consider creating a field for each combination (u1g1, u2g2) which can easily be done via dynamic fields. Use CopyField to get data into these various constructs (aga

Re: Dataless nodes in SolrCloud?

"Just try it" seems like the method we'd all trust the most :) If you want to mod Solr a bit yourself and add some timers around aggregation, you could do that. You could use something like this: * Modify Solr code and add Coda Metrics timers around aggregation code [1] * Use SPM Reporter for Cod

Does from term in join query undergo query time analysis?

Hello: I've been able to find little regarding my specific question by searching around. The bottom line is that I'm issuing a cross index join query such as the following (from the log file): ...&q={!join+from%3DconceptId+to%3Did+fromIndex%3DSearchProducts0}parentProductId:PROD_Hs01_00179501

Re: How to ignore folder collection1 when running single instance of SOLR?

Erick, Thanks a lot for your response. Just to confirm if I am right, I need to use solr.xml even if I change the folder structure as below. Am I right? Do you have any idea when "discovery-based" core enumeration feature would be released? -- View this message in context: http://lucene.472

Re: [ANN] Solr book: Instant Apache Solr for Indexing Data How-to

: #1: done. Somebody should probably sort them in reasonable (latest : version covered?) order go for it (or maybe group them ... put older additions as sub-bullets of current editions?) : But who do I submit a patch to? A new Solr JIRA? An Infra Jira? SOLR Jira. -Hoss

QTime > prepare+process

Hi, I'm wondering why some of our queries take more than a second and started to debug with debugQuery=true. QTime is 2.028s Measured response time on client is 2.0566s But timing only sums up to 204ms: prepare org.apache.solr.handler.component.QueryComponent: 0

Re: Query-node+shard stickiness?

We did this at Infoseek about fifteen years ago. It really helps. We hashed each query, then took modulo the number of servers, and sent it to that server. That means that the same query always goes to the same server. The tricky part is handling a server failure. You really do not want to chan

Re: Filtering down terms in suggest

I suppose you can use fq with SpellCheckComponent but I haven't tried it yet. https://issues.apache.org/jira/browse/SOLR-2010 -- View this message in context: http://lucene.472066.n3.nabble.com/Filtering-down-terms-in-suggest-tp4069627p4069690.html Sent from the Solr - User mailing list arch

Re: SOLR-4641: Schema now throws exception on illegal field parameters.

I think if you use validate=false in schema.xml, field or dynamicField level, Solr will not disable validation. I think this only works in solr 4.3 and above.. -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4641-Schema-now-throws-exception-on-illegal-field-parameters-

Re: [ANN] Solr book: Instant Apache Solr for Indexing Data How-to

Thanks Hoss, #1: done. Somebody should probably sort them in reasonable (latest version covered?) order #2: Found where the book images are: https://svn.apache.org/repos/asf/lucene/cms/trunk/templates/solr-slides.html But who do I submit a patch to? A new Solr JIRA? An Infra Jira? Thanks, Ale

Re: Field Names

On 11 June 2013 16:48, PeriS wrote: > > Nothing wrong and would work too, but I wanted to see if I could restrict > it more as per the schema at http://www.loc.gov/marc/bibliographic/. [...] As per your other messages, constructing the query is under your control. So, why do you care about such r

Re: index merge question

Right - but that sounds a little different than what we were talking about. You had brought up the core admin merge cmd that let's you merge an index into a running Solr cluster. We are calling that the golive option in the map reduce indexing code. It has the limitations we have discussed. Ho

RE: index merge question

FWIW, the Solr included with Cloudera Search, by default, "ignores all but the most recent document version" during merges. The conflict resolution is configurable however. See the documentation for details. http://www.cloudera.com/content/support/en/documentation/cloudera-search/cloudera-search

Re: [ANN] Solr book: Instant Apache Solr for Indexing Data How-to

: I have just published a new mini-book with Packt Publishing, called : "Instant Apache Solr for Indexing Data How-to": : http://www.packtpub.com/apache-solr-for-indexing-data/book . The : examples from the book are available from Packt or from : https://github.com/arafalov/solr-indexing-book . A

Re: Documents with multiple values in a certain field

The Solr Javadoc has an example: http://lucene.apache.org/solr/4_3_0/solr-core/org/apache/solr/update/processor/CountFieldValuesUpdateProcessorFactory.html I have a more complete example (plus others) in the book: Example - Store a count of the values of a named multivalued field in another fi

Re: index merge question

>From my experience the lucene mergeTool and the one invoked by coreAdmin is a pure lucene implementation and does not understand the concepts of a unique Key(solr land concept) http://wiki.apache.org/solr/MergingSolrIndexes has a cautionary note at the end we do frequent index merges for which

Re: external zookeeper with SolrCloud

On Jun 11, 2013, at 10:15 AM, "Joshi, Shital" wrote: > Thanks Mark. > > Looks like this bug is fixed in Solr 4.4. Do you have any date for official > release of 4.4? Looks like it might come out in a couple of weeks. > Is there any instruction available on how to build Solr 4.4 from SVN > r

Re: Why there is no getter method for defaultCollection at CloudSolrServer?

On Jun 11, 2013, at 4:51 AM, Furkan KAMACI wrote: > Why there is no getter method for defaultCollection at CloudSolrServer? Want to create a JIRA issue to add it? - Mark

Re: index merge question

Yeah, you have to carefully manage things if you are map/reduce building indexes *and* updating documents in other ways. If your 'source' data for MR index building is the 'truth', you also have the option of not doing incremental index merging, and you could simply rebuild the whole thing eve

Re: Adding pdf/word file using JSON/XML

"is it possible to index the file + metadata with a JSON/XML request?" You still aren't being clear as to what you are really trying to achieve here. I mean, just write a shell script that does the curl command, or write a Java program or application layer that uses SolrJ to talk to Solr and a

RE: external zookeeper with SolrCloud

Thanks Mark. Looks like this bug is fixed in Solr 4.4. Do you have any date for official release of 4.4? Is there any instruction available on how to build Solr 4.4 from SVN repository? -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Monday, June 10, 2013 8:05

Re: Adding pdf/word file using JSON/XML

Jan, Thanks for the answer. Concerning the usage of /extract, If I understand correctly how works the interface, it seems that the Document is recreated every time the url is called. That would means that all metadata must be provided along the file every time we want to update the related docume

Re: reg: efficient querying using solr

On 6/11/2013 12:15 AM, gururaj kosuru wrote: > How can one calculate an ideal max shard size for a solr core instance if I > am running a cloud with multiple systems of 4GB? That question is impossible to answer without experimentation, but here's a good starting point. That's all it is, a starti

Solr cloud: how to deploy multiple solr instances on the same tomcat server

Using ver 4.3 Solr cloud: how to deploy multiple solr instances on the same tomcat server couldn't understand how to do that It is possible in non cloud version but when I try to use solr cloud I don't understand how do it I have big indexes and I prefer several solr instances and not several core

Re: Adding pdf/word file using JSON/XML

Hi, You can let your web application where people upload the files take care of extracting the text, e.g. using Apache Tika. Once you have the text of the PDF, you can add that to your Solr document along with all the rest of the metadata, and post it to Solr as JSON, XML or whatever you like. Y

[ANN] Solr book: Instant Apache Solr for Indexing Data How-to

Hello, I have just published a new mini-book with Packt Publishing, called "Instant Apache Solr for Indexing Data How-to": http://www.packtpub.com/apache-solr-for-indexing-data/book . The examples from the book are available from Packt or from https://github.com/arafalov/solr-indexing-book . It i

Re: Request to be added to the contributors list

Done, thanks! On Tue, Jun 11, 2013 at 8:06 AM, Barry Chesterman wrote: > Sorry I forgot to include my username, it's BarryChesterman > > > On Tue, Jun 11, 2013 at 1:00 PM, Barry Chesterman > wrote: > >> Hi, I would like to ask if it's possible for myself to be added to the >> contributors list. >

SOLR-4641: Schema now throws exception on illegal field parameters.

Is there a way to tell solr, that it should not check these parameters? Because we added our own parameters, which we load on runtime for other proposes. Thans in advance! -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4641-Schema-now-throws-exception-on-illegal-fiel

Re: Request to be added to the contributors list

Sorry I forgot to include my username, it's BarryChesterman On Tue, Jun 11, 2013 at 1:00 PM, Barry Chesterman wrote: > Hi, I would like to ask if it's possible for myself to be added to the > contributors list. > > I want to update some information provided on the Jetty logging page for > Solr,

Request to be added to the contributors list

Hi, I would like to ask if it's possible for myself to be added to the contributors list. I want to update some information provided on the Jetty logging page for Solr, there is now out dated information about logging properties (as jetty has now moved from mortbay to eclipse which means that the

Re: Adding pdf/word file using JSON/XML

We are working on an application that allows some users to add files (pdf, ms word, odt, etc), located on their local hard disk, to our internal system and allows other users to search for them. So we are considering Solr for the indexing and search functionalities of the system. Along with the fil

RE: Replication not working

This is the log when the slave is polling. But it's onlythe last two times. As you can see, the time in between is ~ 2 minutes. It it's from interest, I can post complete log from fresh restart on. -Original Message- From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:noble.p...@gmail.com] Sent: Tue

Re: Documents with multiple values in a certain field

I believe the usual solution is to pre-calculate that by using UpdateRequestProcessor (probably: CountFieldValuesUpdateProcessorFactory ). Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that

Filtering down terms in suggest

Hi, Trying to find a way to filter down the suggested terms set based on the term value of another indexed field? Let's say we have the following documents indexed in Solr: userid:1, groupid:1, content:"alpha beta gamma" userid:2, groupid:1, content:"alternate better garden" userid:3, groupid:2,

Re: Replication not working

I mean , the log when polling happens when from slave. Not when you issue a command. On Tue, Jun 11, 2013 at 5:28 PM, wrote: > Log on slave: > > 2013-06-11 13:19:08,477 8385607 INFO [org.apache.solr.core.SolrCore] > (http-0.0.0.0-31006-1:) [contacts] webapp=/solr path=/replication > params={in

AW: Replication not working

Log on slave: 2013-06-11 13:19:08,477 8385607 INFO [org.apache.solr.core.SolrCore] (http-0.0.0.0-31006-1:) [contacts] webapp=/solr path=/replication params={indent=true&command=indexversions&wt=json+} status=0 QTime=0 2013-06-11 13:19:08,477 8385607 DEBUG [org.apache.solr.servlet.SolrDispatch

Re: Replication not working

You said polling is happening and nothing is replicated What do the logs say on slave (Set level to INFO) ? On Tue, Jun 11, 2013 at 4:54 PM, wrote: > Calling indexversion on master gives: > > > 00 > > 1370612995391 > 53 > > > On Slave: > >0 >

AW: Replication not working

Calling indexversion on master gives: 00 1370612995391 53 On Slave: 0 0 0 1 > pollInterval is set to 2 minutes. It is usually long I know ;-) -Ursprüngliche Nachricht- Von: Noble Paul നോബിള്‍ नोब्ळ् [mailto:noble.p.

Re: Field Names

And besides, the user is not going to be aware of the actual field names to query on - its abstracted to some simple names like author which could be a combination of 100a,b which the application would know how to construct based on an external configuration etc… You see what i mean. On Jun 11,

Re: Field Names

Nothing wrong and would work too, but I wanted to see if I could restrict it more as per the schema at http://www.loc.gov/marc/bibliographic/. On Jun 10, 2013, at 10:09 PM, Gora Mohanty wrote: > What is wrong with, e.g., > stored="true"/> > > Regards, > Gora

Re: How to ignore folder collection1 when running single instance of SOLR?

That'll work, but I would recommend you put an intermediate name in there. It doesn't have to be "collection1", and it could even be "solr", but I wouldn't put the conf, bin, and index directories as immediate children of SOLR. You could do something like SOLR lib solr conf data

Re: Replication not working

can you check with the indexversion command on both mater and slave? pollInterval is set to 2 minutes. It is usually long . So you may need to wait for 2 mins for the replication to kick in On Tue, Jun 11, 2013 at 3:21 PM, wrote: > Hi all, > > > > we have a setup with multiple cores, loaded vi

Re: Field Names

So what I m trying to do is index MARC21 fields which have a schema defined at http://www.loc.gov/marc/bibliographic/. Now the reason I wanted to use dynamic fields is so that once they are defined I don't have to touch the schema.xml and control indexed fields configuration from outside i.e. a

Re: Query-node+shard stickiness?

Hmmm, that does cut the problem down to size... I guess I wonder how often the same query actually comes into the system but that's a nit. I can certainly see how such a thing would improve those situations that do have lots of repeats Erick On Mon, Jun 10, 2013 at 6:15 PM, Otis Gospodnetic

Re: reg: efficient querying using solr

You can't calculate it precisely, but you can test it. This is where EC2 is handy. Otis Solr & ElasticSearch Support http://sematext.com/ On Jun 11, 2013 2:16 AM, "gururaj kosuru" wrote: > How can one calculate an ideal max shard size for a solr core instance if I > am running a cloud with multi

Re: Advice on Solr 4.0 index backups

This sounds pretty complete to me. Otis Solr & ElasticSearch Support http://sematext.com/ On Jun 11, 2013 4:21 AM, "Cosimo Streppone" wrote: > Hi, > > I'd like your advice on this backup plan. > It's my first Solr deployment (4.0). > > Production consists of 1 master and n frontend slaves > plac

Very bad search performance with group=true

Hi, I'm indexing pdf documents to use full text search with solr. To get the number of the page where the result was found, I save every page separately and group the results with a field called doc_id. (See this topic: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201303.mbox/%3c

Replication not working

Hi all, we have a setup with multiple cores, loaded via DataImportHandlers. Works fine so far. Now we are trying to get the replication working (for one core so far). But the automated replication is never happening. Manually triggered replication works! Environment: Solr 4.1 (also trie

Documents with multiple values in a certain field

Hello, is there a way to get all documents where a certain field has multiple entries? best, Silvio

Re: LIMIT on number of OR in fq

Thanks Alok One need to add headerBufferSize to connector section in jetty.xml file as shown below * 5 1500 false 65536 * Regards Kamal On Mon, Jun 10, 2013 at 12:20 P

Re: How to Get Cloud Statistics and Why It is Permitted to Use CloudSolrServer and LukeRequest?

I think that I should use HttpSolrServer for such kind of purposes. It would be nice to do it within CloudSolrServer if somebody send a LukeRequest it would be nice CloudSolrServer collects distributed metrics. i.e. num of docs, max docs within all shards etc. Last modification date should be the l

Why there is no getter method for defaultCollection at CloudSolrServer?

Why there is no getter method for defaultCollection at CloudSolrServer?

Advice on Solr 4.0 index backups

Hi, I'd like your advice on this backup plan. It's my first Solr deployment (4.0). Production consists of 1 master and n frontend slaves placed in different datacenters, replicating through HTTP. Only master is backed up. Frontend slaves can die anytime or go stale for a while and that's ok. Bac