RE: Solr-4.0.0-Beta Bug with "Load Term Info" in Schema Browser

2012-08-24 Thread Fuad Efendi
Any news? CC: Dev -Original Message- Subject: Solr-4.0.0-Beta Bug with "Load Term Info" in Schema Browser Hi there, "Load term Info" shows 3650 for a specific term "MyTerm", and when I execute query "channel:MyTerm" it shows 650 documents foundŠ possibly bugŠ it happens after I commit

Re: How do I represent a group of customer key/value pairs

2012-08-24 Thread Sheldon P
Thanks for the prompt reply Jack. Could you point me towards any code examples of that technique? On Fri, Aug 24, 2012 at 4:31 PM, Jack Krupansky wrote: > The general rule in Solr is simple: denormalize your data. > > If you have some maps (or tables) and a set of keys (columns) for each map >

Re: How do I represent a group of customer key/value pairs

2012-08-24 Thread Jack Krupansky
The general rule in Solr is simple: denormalize your data. If you have some maps (or tables) and a set of keys (columns) for each map (table), define fields with names like _, such as "map1_name", "map2_name", "map1_field1", "map2_field1". Solr has dynamic fields, so you can define "_*" to hav

More debugging DIH - URLDataSource

2012-08-24 Thread Carrie Coy
I'm trying to write a DIH to incorporate page view metrics from an XML feed into our index. The DIH makes a single request, and updates 0 documents. I set log level to "finest" for the entire dataimport section, but I still can't tell what's wrong. I suspect the XPath. http://localhost:80

How do I represent a group of customer key/value pairs

2012-08-24 Thread Sheldon P
I've just started to learn Solr and I have a question about modeling data in the schema.xml. I'm using SolrJ to interact with my Solr server. It's easy for me to store key/value paris where the key is known. For example, if I have: title="Some book title" author="The authors name" I can repre

Re: ngroups question

2012-08-24 Thread Erick Erickson
I think the memory size is about the (number of groups) * ((size of key) + (a little memory for the bucket to hold members of that group). This latter is (I'm guessing here) quite small. Sure, you can have all 500.000 groups consume memory, quite easily. q=*:* (OK, that one wouldn't be scored, but

Re: Query-side Join work in distributed Solr?

2012-08-24 Thread Erick Erickson
Not as I understand it. All that allows is a pluggable assignment of documents to shards in SolrCloud. There's nothing tying that JIRA to distributed joins or field collapsing. Distributed grouping is already in place as of Solr 3.5, see: https://issues.apache.org/jira/browse/SOLR-2066 Best Erick

Re: Query-side Join work in distributed Solr?

2012-08-24 Thread Pavel Goncharik
Do I understand correctly that once https://issues.apache.org/jira/browse/SOLR-2592 is resolved, it will make both distributed joins and field collapsing work? Best regards, Pavel On Fri, Aug 24, 2012 at 6:01 PM, Erick Erickson wrote: > Right, there hasn't been any action on that patch in a whil

Solr 4.0 beta deadlock / file descriptor spike

2012-08-24 Thread Casey Callendrello
Hi there, I have been doing some load testing with Solr 4 beta (now, trunk). My configuration is fairly simple - two servers, replicating via SolrCloud. SolrCloud is configured as recommended in the wiki: Twice now I've seen sudden thread and file-descriptor spikes along

Re: turning up logging using the web UI, can't get more than INFO

2012-08-24 Thread Kevin Goess
On Fri, Aug 24, 2012 at 10:23 AM, Ahmet Arslan wrote: > > We have a pretty standard > > out-of-the-box solr/jetty setup. Using the web > > UI at /solr/admin/logging, for WARNING or SEVERE we get less > > logging, but > > none of CONFIG, FINE or FINEST result in any *more* logging > > than just a

Solr-4.0.0-Beta Bug with "Load Term Info" in Schema Browser

2012-08-24 Thread Fuad Efendi
Hi there, "Load term Info" shows 3650 for a specific term "MyTerm", and when I execute query "channel:MyTerm" it shows 650 documents foundŠ possibly bugŠ it happens after I commit data too, nothing changes; and this field is single-valued non-tokenized string. -Fuad -- Fuad Efendi 416-993-2060

Re: Is SpellCheck Case Sensitive in Solr3.6.1?

2012-08-24 Thread Kiran Jayakumar
You are missing query analyzer field type: add this line in your search component. *spell* ... On Fri, Aug 24, 2012 at 5:31 AM, mechravi25 wrote: > Hi, > > Im using solr 3.6.1 version now and I configured spellcheck by making > following changes > > Solrconfig.xml: > > > > solr.IndexBa

Re: turning up logging using the web UI, can't get more than INFO

2012-08-24 Thread Ahmet Arslan
> We have a pretty standard > out-of-the-box solr/jetty setup.  Using the web > UI at /solr/admin/logging, for WARNING or SEVERE we get less > logging, but > none of CONFIG, FINE or FINEST result in any *more* logging > than just at > INFO. > > Is there another place to look for something that mig

Re: What are the available parameters in field tag in schema.xml, and data-config.xml ?

2012-08-24 Thread Ahmet Arslan
> For example, in schema.xml, field tag is having a parameter > called > 'default', what else are available? Here is the full list: http://wiki.apache.org/solr/SchemaXml#Common_field_options > For example, in data-config.xml, field tag is having a > parameter called > 'splitBy', what else are ava

What are the available parameters in field tag in schema.xml, and data-config.xml ?

2012-08-24 Thread srinalluri
I want to know XSD of schema.xml and data-config.xml. Basically I want to know available parameters in field tag in schema.xml, and data-config.xml. For example, in schema.xml, field tag is having a parameter called 'default', what else are available? For example, in data-config.xml, field tag is

turning up logging using the web UI, can't get more than INFO

2012-08-24 Thread Kevin Goess
We have a pretty standard out-of-the-box solr/jetty setup. Using the web UI at /solr/admin/logging, for WARNING or SEVERE we get less logging, but none of CONFIG, FINE or FINEST result in any *more* logging than just at INFO. Is there another place to look for something that might be controlling

Re: Bitmap field in solr

2012-08-24 Thread Erick Erickson
There are a couple of open JIRAs, but native bitwise support isn't in the code yet. See SOLR-1913 and SOLR-1918 Best Erick On Thu, Aug 23, 2012 at 4:31 PM, Andy Lester wrote: > > On Aug 23, 2012, at 2:54 PM, Rohit Harchandani wrote: > >> Hi all, >> Is there any way to have a bitmap field in Solr

Re: Porting Lucene Index to Solr: ERROR:SCHEMA-INDEX-MISMATCH

2012-08-24 Thread Erick Erickson
Trie fields index "extra" information to aid in ranges etc. So if you indexed your data as non-trie, then asked Solr to read them as trie fields, it's bound to be unfortunate. Or if you changed the precisionstep. Or. Your schema has to exactly reflect what your lucene program did for indexing,

Re: Query regarding multi core search

2012-08-24 Thread Erick Erickson
Why do you have 4 cores in the first place? The usual use-case is that cores aren't for similar documents But the easiest thing to do would be to include the sort field in the response and have the app (or whatever is aggregating the 4 responses) sort the responses, essentially merging the 4 s

Re: Boosting documents matching in a specific shard

2012-08-24 Thread Erick Erickson
Well, the simplest would be to include the shard ID in the document when you index it, then just boost on that field... Best Erick On Thu, Aug 23, 2012 at 8:33 AM, Husain, Yavar wrote: > I am aware that IDF is not distributed. Suppose I have to boost or give > higher rank to documents which are

Re: Can't extract Outlook message files

2012-08-24 Thread Erick Erickson
Hmmm, it kind of looks like your file doesn't have an "id" field, but that's just guessing based on your statement hat providing an ID "works just fine". Does it work if you take the definition out of your schema.xml (and you'll also have to remove the 'required="true" ' from the id field)? But t

Re: Group count in SOLR 3.3

2012-08-24 Thread Erick Erickson
3.6 has a getNGroups, does that do what you want? Best Erick On Thu, Aug 23, 2012 at 2:23 AM, Roman Slavík wrote: > Hi guys, > > we are using SOLR 3.3 with Solrj inside our java project. In actual version > we had to add some grouping support, so we add parameters into SolrQuery > object like th

Re: Index version & generation for Solr 3.5

2012-08-24 Thread Erick Erickson
This is quite possible if you have multiple commits between replications. You should _not_ depend on the version number of an index changing in a pre-defined way, it'll increase on a commit, but that's about all you can really count on... The slaves do not increment the index version, they just ge

Re: Query-side Join work in distributed Solr?

2012-08-24 Thread Erick Erickson
Right, there hasn't been any action on that patch in a while... Best Erick On Wed, Aug 22, 2012 at 12:18 PM, Timothy Potter wrote: > Just to clarify that query-side joins ( e.g. {!join from=id > to=parent_signal_id_s}id:foo ) do not work in a distributed mode yet? > I saw LUCENE-3759 as unresolv

Re: Debugging DIH

2012-08-24 Thread Ahmet Arslan
> That is not completely true. If the columns have the same > names as the fields, the mapping is redundant. Nevertheless, > it might be the problem. What I've experienced with Oracle, > at least, is that the columns would be returned in uppercase > even if my alias would be in lowercase. You migh

Re: Solr Index problem

2012-08-24 Thread Michael Della Bitta
Have you investigated the logs of your servlet container? There's probably some explanation for why the documents weren't submitted in there. Michael Della Bitta Appinions | 18 East 41st St., Suite 1806 | New York, NY 10017 www.appinions.com Where

Re: Debugging DIH

2012-08-24 Thread Chantal Ackermann
> > I don't see that you have anything in the DIH that tells what columns from > the query go into which fields in the index. You need something like > > > > > That is not completely true. If the columns have the same names as the fields, the mapping is redundant. Nevertheless, it might b

Re: Solr Index problem

2012-08-24 Thread Chantal Ackermann
> Are you committing? You have to commit for them to be actually added…. If DIH says it did not add any documents ("added 0 documents") committing won't help. Likely, there is a problem with the mapping between DIH and the schema so that none of the fields make it into the index. We would need

Re: Unmatched quotes

2012-08-24 Thread Jack Krupansky
1. You can look at the Solr log file and see this exception: Caused by: org.apache.lucene.queryParser.ParseException: Cannot parse 'title:"cycle with 24': Lexical error at line 1, column 21. Encountered: after : "\"cycle with 24" at org.apache.lucene.queryParser.QueryParser.parse(Quer

Re: Solr - Unique Key Field Should Apply on q search or fq search

2012-08-24 Thread Jack Krupansky
A query such as "q=myTextFeild:politics programme" will search for "programme" in the default search field, which may not have any hits. An explicit field name applies to only the immediately successive term or parenthesized sub-query. The second and third queries work because the default oper

RES: Problem to start solr-4.0.0-BETA with tomcat-6.0.20

2012-08-24 Thread Claudio Ranieri
Hi Vadim, No, I used the entire apache-solr-4.0.0-BETA\example\solr (schema.xml, solrconfig.xml ...) -Mensagem original- De: Vadim Kisselmann [mailto:v.kisselm...@gmail.com] Enviada em: sexta-feira, 24 de agosto de 2012 07:26 Para: solr-user@lucene.apache.org Assunto: Re: Problem to sta

Re: Query expansion by taxonomy

2012-08-24 Thread Jack Krupansky
The "More Like This" feature may give you what you want: http://wiki.apache.org/solr/MoreLikeThis http://wiki.apache.org/solr/MoreLikeThisHandler The basic idea is that you do your query on your primary field(s), then you take term(s) from some secondary field (your category) and re-query and ad

Re: Debugging DIH

2012-08-24 Thread Andy Lester
On Aug 24, 2012, at 9:17 AM, Hasan Diwan wrote: > > url="jdbc:h2:tcp://192.168.1.6/finance" user="sa" /> > > > > > > and I've added the appropriate fields to schema.xml: > > > > > There's nothing in my index and 343 rows in my table. What is going on? -- H I don't

Re: Debugging DIH

2012-08-24 Thread Hasan Diwan
On 24 August 2012 07:17, Hasan Diwan wrote: > I have some data in an H2 database that I'd like to move to SOLR. I > probably should/could extract and post the contents as 1 new document per > record, but I'd like to configure the data import handler and am having > some difficulty doing so. Follo

Debugging DIH

2012-08-24 Thread Hasan Diwan
I have some data in an H2 database that I'd like to move to SOLR. I probably should/could extract and post the contents as 1 new document per record, but I'd like to configure the data import handler and am having some difficulty doing so. Following the wiki instructions[1], I have the following in

ngroups question

2012-08-24 Thread reikje
I have a question regarding expected memory consumption when using field collapsing with the ngroups parameter. We have indexed a forum with 500.000 threads. Each thread is a group, so we can have max. 500.000 groups. I read somewhere that for each group a org.apache.lucene.util.ByteRef is created

Query expansion by taxonomy

2012-08-24 Thread Nicholas Ding
Hello, I want do query expansion on Solr, I have a taxonomy index like this. Is that possible to do one search to get a list contains searched keywords and their siblings under same category? For example, search for "Ford" whose category is "Car Dealer", the results is not limited to "Ford",

Is SpellCheck Case Sensitive in Solr3.6.1?

2012-08-24 Thread mechravi25
Hi, Im using solr 3.6.1 version now and I configured spellcheck by making following changes Solrconfig.xml: solr.IndexBasedSpellChecker ./spellchekerIndex spell true and added the following in the standard handler to include the spellcheck spe

Re: Solr search – Tika extracted text from PDF not return highlighting snippet

2012-08-24 Thread Lance Norskog
There are two different sets of readers for binary and character-mode data, and I don't remember which is which. You may be reading the PDF binary blob as a character blob. On Wed, Aug 22, 2012 at 1:34 AM, anarchos78 wrote: > Thanks for your reply, > I had tryied many things (copy field etc) with

Re: Indexing and querying BLOBS stored in Mysql

2012-08-24 Thread Alexandre Rafalovitch
I think it would greatly help if you say specifically where you are stuck. Otherwise, there are too many directions to guess. The configuration snippet you have is a little too large to 'parse'. I believe DataImportHandler has some definition for nested processors, have you tried using those and h

Re: Data Import Handler - Could not load driver - com.microsoft.sqlserver.jdbc.SQLServerDriver - SOLR 4 Beta

2012-08-24 Thread Lance Norskog
Does this class exist in the driver jar? com.microsoft.sqlserver.jdbc.SQLServerDriver On Thu, Aug 23, 2012 at 9:09 AM, awb3667 wrote: > Hello, > > I was able to get the DIH working in SOLR 3.6.1 (placed the sqljdbc4.jar > file in the lib directory, etc). Everything worked great. Tried to get > ev

Re: Unmatched quotes

2012-08-24 Thread Ahmet Arslan
> If I execute the following query, with unmatched quotes, I > get an error from Solr - as I haven't escaped the middle ". > > But the error message appears to simply be "400 null". Is it > possible to get Solr to return a more informative error > message? > > http://myhost/solr/myapp/select?q=ti

Re: Solr 4.0 Beta missing example/conf files?

2012-08-24 Thread Lance Norskog
bin/ usually goes in the collection/ directory, but nobody uses the programs in bin/. They are all for the old rsync replicator. lib/ can go next to solr.xml, or in a collection. In the top directory, lib/ jars are visible to all collections. Inside a collection, lib/ jars are only visible to that

Re: Solr - Unique Key Field Should Apply on q search or fq search

2012-08-24 Thread Ahmet Arslan
> For. e. g. if i search with below url, then it return > results me as 0 rows , > where as such record exist. > > http://localhost:8080/solr/core0/select?q=myTextFeild:politics > programme AND > myuniquekey:193834 > > but if i modify my search with any of below mentioned search > query it works

Unmatched quotes

2012-08-24 Thread Peter Kirk
Hi, If I execute the following query, with unmatched quotes, I get an error from Solr - as I haven't escaped the middle ". But the error message appears to simply be "400 null". Is it possible to get Solr to return a more informative error message? http://myhost/solr/myapp/select?q=title:"cycl

Re: Problem to start solr-4.0.0-BETA with tomcat-6.0.20

2012-08-24 Thread Vadim Kisselmann
a presumption: do you use your "old" solrconfig.xml files from older installations? when yes, compare the default config and yours. 2012/8/23 Claudio Ranieri : > I made this instalation on a new tomcat. > With Solr 3.4.*, 3.5.*, 3.6.* works with jars into > $TOMCAT_HOME/webapps/solr/WEB-INF/lib,

Re: Indexing and querying BLOBS stored in Mysql

2012-08-24 Thread Alexey Serba
I would recommend to create a simple data import handler to test tika parsing for large BLOBs, i.e. remove not related entities, remove all the configuration for delta imports and keep just entity that retrieves blobs and entity that parses binary content (fieldReader/TikaEntityProcessor). Some co