Reg:- StrField Analyzer Issue
Hi , I am using Apache Solr for do advanced searching with my Big Data. When I am creating Solr core , then by default for text field , it is coming as TextField data type and class. Can you please tell me how to change TextField to StrField. My table contains record into English as well as Chinese . Please guide me for correct StrField. Thanks.
Re: Reg:- StrField Analyzer Issue
Thanks Erick For best Explanation. The issue with My data is as below. :- I have few data on my books table. cqlsh:nandan> select * from books; id | author | date | isbn | solr_query | title --+--+--+--++--- 3910b29d-c957-4312-9b8b-738b1d0e25d0 | Chandan | 2015 | 1asd33s | null | Solr d7534021-80c2-4315-8027-84f04bf92f53 | 现在有货 | 2015 | 现在有货 | null | Solr 780b5163-ca6b-40bf-a523-af2c075ef7df | 在有货 | 2015 | 在有货 | null | Solr e6229268-d0fd-485b-ad89-bbde73a07ed6 | 货 | 2015 | 现有货 | null | Solr 76461e7e-6c31-4a4b-8a36-0df5ce746d50 | Nandan | 2017 |1 | null | Datastax 9a9c66c2-cd34-460e-a301-6d8e7eb14e55 | Kundan | 2016 | 12ws | null | Cassandra 7e87dc3a-5e4e-4653-84cc-3d83239708d4 | 现有货 | 2015 | 现有货 | null | Solr 6971976e-2528-4956-94a8-345deefe5796 | 现货 | 2015 | 现货 | null | Solr When I am trying to select from table based on author as:- cqlsh:nandan> SELECT * from books where solr_query = 'author:现有货'; id | author | date | isbn | solr_query | title --+--+--+--++--- d7534021-80c2-4315-8027-84f04bf92f53 | 现在有货 | 2015 | 现在有货 | null | Solr 7e87dc3a-5e4e-4653-84cc-3d83239708d4 | 现有货 | 2015 | 现有货 | null | Solr 6971976e-2528-4956-94a8-345deefe5796 | 现货 | 2015 | 现货 | null | Solr 780b5163-ca6b-40bf-a523-af2c075ef7df | 在有货 | 2015 | 在有货 | null | Solr It should return me one value , but I am getting other records also, But when I am trying to retrive another way, then it is returning me 0 rows as :- cqlsh:nandan> SELECT * from books where solr_query = 'author:*现有货*'; id | author | date | isbn | solr_query | title ++--+--++--- (0 rows) cqlsh:nandan> SELECT * from books where solr_query = 'author:*现有货'; id | author | date | isbn | solr_query | title ++--+--++--- (0 rows) cqlsh:nandan> SELECT * from books where solr_query = 'author:现有货*'; id | author | date | isbn | solr_query | title ++--+--++--- (0 rows) In Some cases, I am getting correct data but in some case, I am getting wrong data. Please check. Thanks Nandan On Thu, Jun 15, 2017 at 11:47 AM, Erick Erickson wrote: > Back up a bit and tell us why you want to use StrField, because what > you're trying to do is somewhat confused. > > First of all, StrFields are totally unanalyzed. So defining an > as part of a StrField type definition is totally > unsupported. I'm a bit surprised that Solr even starts up. > > Second, you can't search a StrField unless you search the whole thing > exactly. That is, if your title field is "My dog has fleas", there > only a few ways to match anything in that field > > 1> search "My dog has fleas" exactly. Even "my dog has fleas" wouldn't > match because of the capitalization. "My dog has fleas." would also > fail because of the period. StrField types are intended for data that > should be invariant and not tokenized. > > 2> prefix search as "My dog*" > > 3> pre-and-postfix as "*dog*" > > <2> is actually reasonable if you have more than, say, 3 or 4 "real" > characters before the wildcard. > > <3> performs very poorly at any kind of scale. > > A search for "dog" would not match. A search for "fleas" wouldn't > match. You see where this is going. > > If those restrictions are OK, just use the already-defined "string" type. > > As for the English/Chinese that's actually kind of a tough one. > Splitting Chinese up into searchable tokens is nothing like breaking > English up. There are examples in the managed-schema file that have > field definitions for Chinese, but I know of no way to have a single > field type shard the two different analysis chains. One solution > people have used is to have a title_ch and title_en field and search > both. Or search one or the other preferentially if the input is in one > language or the other. > > I strongly advise you use the admin UI>>analysis page to understand > the effects of tokenization, it's the heart of searching. > > Best, > Erick > > On Wed, Jun 14, 2017 at 6:23 PM, @Nandan@ > wrote: > > Hi , > > > > I am using Apache Solr for do advanced searching with my Big Data. > > > > When I am creating Solr core , then by default for text field , it is > > coming
Solr Reindex Issue - Can't able to Reindex Old Data
Hi , I am using Apache Solr with Cassandra Database. In my table, I have 20 rows. Due to some changes, I changed my Solr schema and Reindex schema with below option as *reindex=true and deleteAll=false* After Reindexing my Solr Schema, I am not able to do reindex my old data which are present in my table before. I am only able to retrieve newly added data which is done after reindexing. Please help in this issue. Thanks
Reg:- Indexing MySQL data with Solr
Hi , I am working on an Ecommerce database . I have more then 40 tables and around 20GB of data. I want to index data with solr for more effective search feature. Please tell me how to index MySQL data with Apache solr. Thanks in advance Nandan Priyadarshi
Reg:- Create Solr Core Using Command Line
Hi , This question might be very basic, but need to clarify my basic understanding. I am using Solr Version 7.2.1 I have one CSV file named as books_data.csv which contains 2 records. Now I want to create solr core to start my basic search using Solr UI. Steps which I Follow :- 1) Go to bin directory and start solr C:\solr\bin>solr start -p 8983 2) books_data.csv is in C:\solr location 3) Now I try to create solr core. C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No Conf Sub folder or Solrconfig.xml file present. 4) Then I created folder "books_data" in C:\solr location and Created conf subfolder under books_data folder and put solrconfig.xml inside conf subfolder. 5) Again start to execute query C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got Error :- Already core existed. When I checked Solr Admin UI , showing error message as SolrCore Initialization Failures - *books_data:* org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not load conf for core books_data: Error loading solr config from C:\solr\bin\books_data\conf\solrconfig.xml Please tell me where am I doing wrong? Thanks.
Re: Reg:- Create Solr Core Using Command Line
Hi Sadiki, I checked Sample Techproduct Conf folder. Inside that folder, there are numerous files. So Again my question will be how those files came. I want to create core from Scratch and want to check and create each and every config files from my sides. Then only I can able to understand what and which files needs in different solr search function. I hope you can understand my query. Thanks On Tue, Feb 6, 2018 at 11:48 AM, Sadiki Latty wrote: > If I'm not mistaken the command requires that the books_data folder > already exists with a conf folder inside and the various required files > (solrconfig.xml, solr.xml,etc). To get an idea of what you should have in > your conf folder you can check out the included configsets > (sample_techproducts_configs for example). These configsets have the > required files and you can copy and modify to accommodate your own needs. I > am not 100% sure where to find them on a windows installation but I believe > it would be C:\solr\server\configsets\ or another subfolder of the server > folder. > > -Original Message- > From: @Nandan@ [mailto:nandanpriyadarshi...@gmail.com] > Sent: Monday, February 5, 2018 9:46 PM > To: solr-user@lucene.apache.org > Subject: Reg:- Create Solr Core Using Command Line > > Hi , > This question might be very basic, but need to clarify my basic > understanding. > I am using Solr Version 7.2.1 > I have one CSV file named as books_data.csv which contains 2 records. > Now I want to create solr core to start my basic search using Solr UI. > Steps which I Follow :- > 1) Go to bin directory and start solr > C:\solr\bin>solr start -p 8983 > 2) books_data.csv is in C:\solr location > 3) Now I try to create solr core. > C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No Conf > Sub folder or Solrconfig.xml file present. > 4) Then I created folder "books_data" in C:\solr location and Created conf > subfolder under books_data folder and put solrconfig.xml inside conf > subfolder. > 5) Again start to execute query > C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got Error > :- Already core existed. > When I checked Solr Admin UI , showing error message as SolrCore > Initialization Failures > >- *books_data:* >org.apache.solr.common.SolrException:org.apache.solr. > common.SolrException: >Could not load conf for core books_data: Error loading solr config from >C:\solr\bin\books_data\conf\solrconfig.xml > > > Please tell me where am I doing wrong? > Thanks. >
Re: Reg:- Create Solr Core Using Command Line
Hi Abhishek, In my conf sub directory, I put only solrconfig.xml file as per error I was getting. I didn't put any other file . Please check . Thanks On Tue, Feb 6, 2018 at 12:57 PM, abhi Abhishek wrote: > Hello, > I followed the steps outlined in your mail. i was able to get a running > core up fine. only thing I can think of in your case is the config > directory having all the required files for the SOLR Core to get > initialized. can you check if you have all the SOLR config files in the > config directory ( i.e, schema.xml, solrconfig.xml, also various supporting > files referred in the schema.xml) you specified on the command line? > > can you share the conf directory, if you can? > > Cheers! > Abhishek > > On Tue, Feb 6, 2018 at 9:30 AM, @Nandan@ > wrote: > > > Hi Sadiki, > > I checked Sample Techproduct Conf folder. Inside that folder, there are > > numerous files. So Again my question will be how those files came. > > I want to create core from Scratch and want to check and create each and > > every config files from my sides. Then only I can able to understand what > > and which files needs in different solr search function. > > I hope you can understand my query. > > > > Thanks > > > > On Tue, Feb 6, 2018 at 11:48 AM, Sadiki Latty wrote: > > > > > If I'm not mistaken the command requires that the books_data folder > > > already exists with a conf folder inside and the various required files > > > (solrconfig.xml, solr.xml,etc). To get an idea of what you should have > in > > > your conf folder you can check out the included configsets > > > (sample_techproducts_configs for example). These configsets have the > > > required files and you can copy and modify to accommodate your own > > needs. I > > > am not 100% sure where to find them on a windows installation but I > > believe > > > it would be C:\solr\server\configsets\ or another subfolder of the > > server > > > folder. > > > > > > -Original Message- > > > From: @Nandan@ [mailto:nandanpriyadarshi...@gmail.com] > > > Sent: Monday, February 5, 2018 9:46 PM > > > To: solr-user@lucene.apache.org > > > Subject: Reg:- Create Solr Core Using Command Line > > > > > > Hi , > > > This question might be very basic, but need to clarify my basic > > > understanding. > > > I am using Solr Version 7.2.1 > > > I have one CSV file named as books_data.csv which contains 2 > records. > > > Now I want to create solr core to start my basic search using Solr UI. > > > Steps which I Follow :- > > > 1) Go to bin directory and start solr > > > C:\solr\bin>solr start -p 8983 > > > 2) books_data.csv is in C:\solr location > > > 3) Now I try to create solr core. > > > C:\solr\bin>solr create_core -c books_data -d C:\solr Got Error :- No > > Conf > > > Sub folder or Solrconfig.xml file present. > > > 4) Then I created folder "books_data" in C:\solr location and Created > > conf > > > subfolder under books_data folder and put solrconfig.xml inside conf > > > subfolder. > > > 5) Again start to execute query > > > C:\solr\bin>solr create_core -c books_data -d C:\solr\books_data Got > > Error > > > :- Already core existed. > > > When I checked Solr Admin UI , showing error message as SolrCore > > > Initialization Failures > > > > > >- *books_data:* > > >org.apache.solr.common.SolrException:org.apache.solr. > > > common.SolrException: > > >Could not load conf for core books_data: Error loading solr config > > from > > >C:\solr\bin\books_data\conf\solrconfig.xml > > > > > > > > > Please tell me where am I doing wrong? > > > Thanks. > > > > > >
Re: Reg:- Create Solr Core Using Command Line
Hi ,
I created core name as "films". Now I am trying to insert my csv file by
below step:-
C:\solr>curl "http://localhost:8983/solr/films/update?commit=true";
--data-binary @example/books_data.csv -H 'Content-type:application/csv'
Got Below result.
{
"responseHeader":{
"status":0,
"QTime":279}}
But in Solr Admin UI, can't able to see any data.
please tell me where am i wrong ?
Thanks
On Tue, Feb 6, 2018 at 1:42 PM, Shawn Heisey wrote:
> On 2/5/2018 10:39 PM, Shawn Heisey wrote:
>
>> In order for this solr script command to work, the argument to the -d
>> option (which you have as C:\solr) would have to be a config directory,
>> containing a minimum of solrconfig.xml and the schema.
>>
> Replying to myself because I made an error here.
>
> The directory provided with -d needs to contain a "conf" subdirectory,
> which in turn must contain the files that I mentioned.
>
> Thanks,
> Shawn
>
Re: Reg:- Create Solr Core Using Command Line
Hi Abhishek,
Even I tried this one before also. And everytime getting below result.
> C:\solr\bin>post -c films example/books_data.csv
'post' is not recognized as an internal or external command,
operable program or batch file.
C:\solr\bin>cd ..
C:\solr>bin/post -c films example/books_data.csv
'bin' is not recognized as an internal or external command,
operable program or batch file.
I google it but not able to find exact solution.
Please check and Provide me solution.
Thanks
On Tue, Feb 6, 2018 at 10:15 PM, abhi Abhishek wrote:
> you can try using the post Tool.
>
> https://lucene.apache.org/solr/guide/6_6/post-tool.html
>
> bin/post -c film example/books_data.csv
>
>
> Cheers!
> Abhishek
>
>
> On Tue, Feb 6, 2018 at 1:22 PM, @Nandan@
> wrote:
>
> > Hi ,
> > I created core name as "films". Now I am trying to insert my csv file by
> > below step:-
> > C:\solr>curl "http://localhost:8983/solr/films/update?commit=true";
> > --data-binary @example/books_data.csv -H 'Content-type:application/csv'
> > Got Below result.
> > {
> > "responseHeader":{
> > "status":0,
> > "QTime":279}}
> >
> > But in Solr Admin UI, can't able to see any data.
> > please tell me where am i wrong ?
> > Thanks
> >
> >
> > On Tue, Feb 6, 2018 at 1:42 PM, Shawn Heisey
> wrote:
> >
> > > On 2/5/2018 10:39 PM, Shawn Heisey wrote:
> > >
> > >> In order for this solr script command to work, the argument to the -d
> > >> option (which you have as C:\solr) would have to be a config
> directory,
> > >> containing a minimum of solrconfig.xml and the schema.
> > >>
> > > Replying to myself because I made an error here.
> > >
> > > The directory provided with -d needs to contain a "conf" subdirectory,
> > > which in turn must contain the files that I mentioned.
> > >
> > > Thanks,
> > > Shawn
> > >
> >
>
Re: Reg:- Create Solr Core Using Command Line
Hi , Thanks you all for quality reply . Finally I am able to below from Scratch:- 1) Create Core by CMD line. 2) Index CSV file by CMD Below are my steps , and I hope other basic beginners can able to learn easily. Basic Information about Solr :- Installation folder :- C:\solr Solr bin folder location :- C:\solr\bin *Create Solr Core [With Default Setting]:-* 1) Go to Solr bin folder 2) Open cmd 3) Enter Command C:\solr\bin>solr create_core –c films If you will check SOLR UI , then you will able to see core name as “films”. *Index CSV file into Existed Core [Windows 10] :-* 1) Please check where is post.jar file. By default , post.jar file present in solr_location/example/exampledocs location. 2) Open cmd in post.jar location. Once you will open cmd , it seems like below location . C:\solr\example\exampledocs> 3) Please check csv file location. In my case, CSV file located into below location. C:\solr\example\books_data.csv 4) Check Core name in which you want to index csv file. In my case, Core name is “films” 5) Use below command to Index csv file in Solr core. Command Format:- C:\solr\example\exampledocs>java -Dtype=text/csv -Dc= -jar post.jar Example:- C:\solr\example\exampledocs>java -Dtype=text/csv -Dc=films -jar post.jar C:\solr\example\books_data.csv Thanks... On Wed, Feb 7, 2018 at 1:55 PM, Shawn Heisey wrote: > On 2/6/2018 6:00 PM, @Nandan@ wrote: > >> Even I tried this one before also. And everytime getting below result. >> >> C:\solr\bin>post -c films example/books_data.csv >>> >> >> 'post' is not recognized as an internal or external command, >> operable program or batch file. >> >> C:\solr\bin>cd .. >> >> >> C:\solr>bin/post -c films example/books_data.csv >> >> 'bin' is not recognized as an internal or external command, >> operable program or batch file. >> >> I google it but not able to find exact solution. >> Please check and Provide me solution. >> > > The post tool is a shell script. There is currently no equivalent for > Windows. > > You can look at the "post" file in the bin directory to see the Java > commandline that is used to call the post tool. A similar java commandline > can be called by hand on Windows. > > Thanks, > Shawn >
Index data from mysql DB to Solr - From Scratch
Hi Team, I am working on one e-commerce project in which my data is storing into MySQL DB. As currently we are using mysql search but planning to implement Solr search to provide our customers more facilities. Just for development purpose ,I am trying to do experiments into localhost. Please guide me how can I Achieve it. Please provide some information links which I can refer to learn more in details from scratch. Thanks and Best Regards, Nandan Priyadarshi
Re: Index data from mysql DB to Solr - From Scratch
Hi David , Thanks for your reply. My few questions are :- 1) I have to denormalize my MySQL data manually or some process is there. 2) is it like when Data will insert into my MySQL , it has to auto index into solr ? Please explain these . Thanks On Feb 18, 2018 1:51 AM, "David Hastings" wrote: > Your first step is to denormalize your data into a flat data structure. > Then index that into your solr instance. Then you’re done > > On Feb 17, 2018, at 12:16 PM, @Nandan@ <mailto:nandanpriyadarshi...@gmail.com>> wrote: > > Hi Team, > I am working on one e-commerce project in which my data is storing into > MySQL DB. > As currently we are using mysql search but planning to implement Solr > search to provide our customers more facilities. > Just for development purpose ,I am trying to do experiments into localhost. > Please guide me how can I Achieve it. Please provide some information links > which I can refer to learn more in details from scratch. > > Thanks and Best Regards, > Nandan Priyadarshi >
Re: Index data from mysql DB to Solr - From Scratch
Thanks Rick. Is it possible to get some demo learning video link or web links from where I can get overview with real example? By which I can able to know in more details. On Feb 18, 2018 4:11 AM, "Rick Leir" wrote: > Nandan > Work backwards from your results screen. When a user has done a search, > what information would you like to appear on the screen? > > That tells you what your Solr document needs to contain. How will you get > that information into the Solr document? You will do the SQL select(s) as > necessary, get the info from MySQL, build a flat JSON record containing > all that info for one document, and POST it to Solr. Repeat for all > documents. Do a commit. Sorry, I Left out all the details! Cheers -- Rick > > On February 17, 2018 12:56:59 PM EST, "@Nandan@" < > nandanpriyadarshi...@gmail.com> wrote: > >Hi David , > >Thanks for your reply. > >My few questions are :- > >1) I have to denormalize my MySQL data manually or some process is > >there. > >2) is it like when Data will insert into my MySQL , it has to auto > >index > >into solr ? > > > >Please explain these . > >Thanks > > > >On Feb 18, 2018 1:51 AM, "David Hastings" wrote: > > > >> Your first step is to denormalize your data into a flat data > >structure. > >> Then index that into your solr instance. Then you’re done > >> > >> On Feb 17, 2018, at 12:16 PM, @Nandan@ > > >> <mailto:nandanpriyadarshi...@gmail.com>> wrote: > >> > >> Hi Team, > >> I am working on one e-commerce project in which my data is storing > >into > >> MySQL DB. > >> As currently we are using mysql search but planning to implement Solr > >> search to provide our customers more facilities. > >> Just for development purpose ,I am trying to do experiments into > >localhost. > >> Please guide me how can I Achieve it. Please provide some information > >links > >> which I can refer to learn more in details from scratch. > >> > >> Thanks and Best Regards, > >> Nandan Priyadarshi > >> > > -- > Sorry for being brief. Alternate email is rickleir at yahoo dot com
Re: Rename solrconfig.xml
You can change into core config file and then you can use any name . As i used as table_solrconfig.xml Same concept will applicable with schema.xml file too. On Feb 27, 2018 11:11 AM, "Zheng Lin Edwin Yeo" wrote: Hi Alexandre, Thanks for your reply. Will this cause other issues with the functionality if it is renamed? Regards, Edwin On 27 February 2018 at 07:15, Alexandre Rafalovitch wrote: > I believe this can be set with "config" property in the > core.properties file: > https://lucene.apache.org/solr/guide/7_2/defining-core- > properties.html#defining-core-properties > > Whether it is a good idea longer term, is a different question. > > Regards, >Alex. > > On 23 February 2018 at 18:06, Zheng Lin Edwin Yeo > wrote: > > Hi, > > > > Would like to check, how can we rename solrconfig.xml to something else? > > For example, I want to rename it to myconfig.xml. Is this possible? > > > > I'm using Solr 6.5.1, and planning to upgrade to Solr 7.2.1. > > > > Regards, > > Edwin >
Re: Add UUID on the schema
Here is some basic question based on that answers vary.
1) Are you going to use UUID column as unique column.
2) what do you with that column.
Process :-
Column is :-
For Add this :-
And then need to add in uniqueid section.
id
On Thu, May 17, 2018 at 5:00 PM, msaunier wrote:
> Hello,
>
>
>
> I use SolrCloud 7.0 and I don't have find how to add an UID field on the
> schema. I have add UUIDField type and add a field but it do not work.
>
>
>
>
>
> {
>
> "name": "uid",
>
> "class": "solr.UUIDField",
>
> "indexed": true,
>
> "stored": true
>
> },
>
>
>
> Thanks for your help.
>
>
>
>
>
>
Re: Uppercase Zip/Postal Code
In case of lowercase search, you need to apply analyzer with filter..
Hope this will help you
On Tue, Jun 12, 2018 at 1:52 AM, ron773 wrote:
> I'm using solr to query a database to find UK postal codes. However, I can
> only find zip codes when they use capital letters, and I can't find them
> with lowercase letters. I was wondering if it's possible to set solr to be
> case-insensitive for zip codes or if not, is it possible to set the query
> so
> that it forces uppercase onto search.
>
> Here's an example of the code.
>
> [Display(Name = "Zip / Postcode")]
> public string ZipCode { get; set; }
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
