Re: DIH and UTF-8

2010-12-27 Thread Lance Norskog
Also if you can show your DIH script. Does the Chinese text come out ok when you do a database query? http://wiki.apache.org/solr/DataImportHandler#Troubleshooting On Mon, Dec 27, 2010 at 4:38 PM, Glen Newton wrote: > Is it possible your browser is not set up to properly display the > chinese ch

Re: DIH and UTF-8

2010-12-27 Thread Glen Newton
Is it possible your browser is not set up to properly display the chinese characters? (I am assuming you are looking at things through your browser) Do you have any problems viewing other chinese documents properly in your browser? Using mysql, can you see these characters properly? What happens w

Re: DIH and UTF-8

2010-12-27 Thread Mark
Just like the user of that thread... i have my database, table, columns and system variables all set but it still doesnt work as expected. Server version: 5.0.67 Source distribution Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> SHOW VARIABLES LIKE 'collation%'; +

Re: exception with xml file processing

2010-12-27 Thread Lance Norskog
Tomcat needs a flag that tells it to use UTF-8. If you don't set that various problems happen, including this one. Look on the solr wiki for Tomcat and UTF-8. Also, there can't be any blank lines at the top of the XML file before the XML header. Can you post a very short XML file that has this pr

Re: Custom match scoring

2010-12-27 Thread Lance Norskog
The Similarity class in Lucene is the low-level place where documents and fields are rated. You might be able to do this change there. On Mon, Dec 27, 2010 at 3:15 AM, Nelson Branco wrote: > Yes, I already looked dismax which I'm  using for other purposes, the big > deal for this problem is havin

Re: DIH and UTF-8

2010-12-27 Thread Glen Newton
Try this in mysql: mysql> SHOW VARIABLES LIKE 'character_set%'; From: http://forums.mysql.com/read.php?45,362257,362257 >From the same page: do you have utf8 set in your 'my.cnf' (shouldn't effect JDBC but will effect the server)? > [client] > default-character-set = utf8 > [mysql] > default-char

Re: DIH and UTF-8

2010-12-27 Thread Mark
I tried both of those with no such luck. On 12/27/10 2:49 PM, Glen Newton wrote: 1 - Verify your mysql is set up using UTF-8 2 - Does your JDBC connect string contain: useUnicode=true&characterEncoding=UTF-8 See: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-charsets.html Glen ht

Re: DIH and UTF-8

2010-12-27 Thread Glen Newton
1 - Verify your mysql is set up using UTF-8 2 - Does your JDBC connect string contain: useUnicode=true&characterEncoding=UTF-8 See: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-charsets.html Glen http://zzzoot.blogspot.com/ On Mon, Dec 27, 2010 at 5:15 PM, Mark wrote: > Solr: 1.4

Re: DIH and UTF-8

2010-12-27 Thread Mark
Solr: 1.4.1 JDBC driver: Connector/J 5.1.14 Looks like its the JDBC driver because It doesn't even work with a simple java program. I know this is a little off subject now, but do you have any clues? Thanks again On 12/27/10 1:58 PM, Erick Erickson wrote: More data please. Which jdbc drive

Re: DIH and UTF-8

2010-12-27 Thread Erick Erickson
More data please. Which jdbc driver? Have you tried just printing out the results of using that driver in a simple Java program? Solr should handle UTF-8 just fine, but the servlet container may have to have some settings tweaked, which one of those are you using? What version of Solr? Best Eri

DIH and UTF-8

2010-12-27 Thread Mark
Seems like I am missing some configuration when trying to use DIH to import documents with chinese characters. All the documents save crazy nonsense like "这是测试" instead of actual chinese characters. I think its at the JDBC level because if I hardcode one of the fields within data-confi

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 7:03 PM, Ahmet Arslan wrote: f d Yes exactly you said it. It worked Thanks Ahmet Here is the working code 'sort'=>array('packageId desc, sortId desc') Thanks dhanesh s.r

Re: exception with xml file processing

2010-12-27 Thread Erick Erickson
This often happens if there is some character at the very beginning of the XML document, outside of any tags, here: character ''' (code 39) in prolog; expected '<' at [row,col {unknown-source}]: [1,1] But you indicate that this is happening for every document? If that's the case, it may be an en

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
> Yes it was sint and now i changed to tint > Restarted the solr server > Remove the data from the solr server > Added fresh data for testing. > But not working the sorting. I just noticed, you are supplying multiple sort parameters. That's the problem. There should be only one sort parameter.

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 6:19 PM, Ahmet Arslan wrote: Its an intiger like 1293452485. I changed the solr schema as you specified Restarted the solr server. First sort field sortId desc is working but the second sort field packageId desc is not working Then make packageId tint too, like sortId. By t

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
> Its an intiger like 1293452485. I changed the solr schema > as you specified > > stored="true" required="true"/> > > Restarted the solr server. > > First sort field sortId desc is working but the second sort > field packageId desc is not working Then make packageId tint too, like sortId.

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 5:22 PM, Ahmet Arslan wrote: Its an intiger like 1293452485. I changed the solr schema as you specified required="true"/> Restarted the solr server. First sort field sortId desc is working but the second sort field packageId desc is not working Here is the query $addition

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
> required="false" /> > What is the domain of id set? Integers? Letters? if it is integers use this: if not use: if it is integers use this: Re-start servlet container, re-index.

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 5:15 PM, Ahmet Arslan wrote: id field is the unique field ( id) Okey, but what is its type? There should be something like

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
> id field is the unique field  ( > id) Okey, but what is its type? There should be something like

Re: sort multiple fields using solr not working

2010-12-27 Thread dhanesh
On 12/27/2010 5:09 PM, Ahmet Arslan wrote: Here is the query $additionalParameters = array( 'facet'=>'true', 'facet.field'=>array('category','subcategory','district'), 'fl'=>'name,category,package,district,score,city,description,id', 'fq'=>array('ca

Re: sort multiple fields using solr not working

2010-12-27 Thread Ahmet Arslan
> Here is the query > >     $additionalParameters  >    = array( >         'facet'=>'true', >         > 'facet.field'=>array('category','subcategory','district'), >         > 'fl'=>'name,category,package,district,score,city,description,id', >         > 'fq'=>array('category:'.$_GET['searchCategory

RE: Custom match scoring

2010-12-27 Thread Nelson Branco
Yes, I already looked dismax which I'm using for other purposes, the big deal for this problem is having summed only the best match of each field. In dismax it sum all matches on each field. Thanks anyway. -- Nelson Branco SAPO Mapas/GIS -Original Message- From: Erick Erickson [mailto:

sort multiple fields using solr not working

2010-12-27 Thread dhanesh
Hi Can anybody help me to sort multiple fields using solr Here is the query $additionalParameters = array( 'facet'=>'true', 'facet.field'=>array('category','subcategory','district'), 'fl'=>'name,category,package,district,score,city,description,id', 'fq'=>

Re: Optimizing to only 1 segment

2010-12-27 Thread Li Li
oh, you mean lucene 4 trunk. LogByteSizeMergePolicy's default size is 2048MB(2GB) I did like this LogByteSizeMergePolicy mp=new LogByteSizeMergePolicy(); mp.setMaxMergeMB(100); 2010/12/27 Rok Rejc : > Okej the same thing happens if i run optimize in java: > >        File

Re: Optimizing to only 1 segment

2010-12-27 Thread Rok Rejc
Okej the same thing happens if i run optimize in java: File file = new File("e:\\myIndex\\index"); Directory directory = FSDirectory.open(file); Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_40); IndexWriterConfig config = new IndexWriterConfig(Version.LU