Data Import Handler OR Solrj to index existing data
Hi All, It has been found that DIH might not suitable to index existing data from DB due to security reasons as like the security people WILL NOT “just open the database for the IP address of the Solr indexer. Is it true, what is preferable. Please let me know on this. Thanks in advance. -- View this message in context: http://lucene.472066.n3.nabble.com/Data-Import-Handler-OR-Solrj-to-index-existing-data-tp4082101.html Sent from the Solr - User mailing list archive at Nabble.com.
Category and Subcategory handling in 4.4 version
Hi All, Our web application (e commerce ) requires primary and secondary categories in items. Based on this requirement I have following queries : 1) How category and subcategory are handled in solr version 4.4. I have used apache-solr-1.3.0 previously, but facets have undergone many big changes since then so just wanted to know how this can be achieved efficiently now. 2) Does category and subcategories should be saved just in database and should be referred as fields in documents only for navigation, we will require categories for inventory count and as search criteria. Let me know about this. Thanks in advance. -- View this message in context: http://lucene.472066.n3.nabble.com/Category-and-Subcategory-handling-in-4-4-version-tp4083188.html Sent from the Solr - User mailing list archive at Nabble.com.
Document generation from database and partially from other source for the same item
Hi all, We have a requirement in the ecommerce site that, Keywords string for items is required but just for searching purpose. Since keywords will be long and only used for searching thus we just want to be indexed and don't need them to persist in DB. Keywords will be there is the spreadsheet initially along with rest of the item data. So everything excluding the keywords will be going to persist in database and documents will be generated for the items from db data. Now I just want the keywords from the spreadsheet to be updated in those documents for those items. How this can be achieved ? -- View this message in context: http://lucene.472066.n3.nabble.com/Document-generation-from-database-and-partially-from-other-source-for-the-same-item-tp4083191.html Sent from the Solr - User mailing list archive at Nabble.com.
Handling categories( level one and two) based navigation
Hi All, Our web application (e commerce ) requires primary and secondary categories in items. Based on this requirement I have following queries : 1) How category and subcategory are handled in solr version 4.4. I have used apache-solr-1.3.0 previously, but facets have undergone many big changes since then so just wanted to know how this can be achieved efficiently now. 2) Does category and subcategories should be saved just in database and should be referred as fields in documents only for navigation, we will require categories for inventory count and as search criteria. Let me know about this. Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Handling-categories-level-one-and-two-based-navigation-tp4083259.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Handling categories( level one and two) based navigation
Hi Eric, Yeah a document can belong to multiple subcategory hierarchies. Also we will be having multi-level categorization unlike the 2 level I previously mentioned. like : Electronics > Phones > Google Nexus ... Also since solr does not support relational join, so shall I fetch the categories and subcategories from DB directly and then use Facet.pivot feature to do category navigation and searching of documents using solr4.4.0 version ? -- View this message in context: http://lucene.472066.n3.nabble.com/Handling-categories-level-one-and-two-based-navigation-tp4083259p4084387.html Sent from the Solr - User mailing list archive at Nabble.com.
Applying Stop words for Field Type String
We have the facility of applying stop words for Field Type text during Query analyzer in schema.xml as shown below : Question : Is it possible to do the same for String type or not, since the XML result we are getting while doing a search on admin is showing the contents of String type as follows , Examle : I searched Salman Rushdie on my Solr Admin , the XML result i recieved was as shown below : - - 0 16 - 10 0 on salman 2.2 - - productIndex 6000 - productIndex 999 Salman Rushdie Harper Lee Gurcharan Das Jules Verne these Salman Rushdie Harper Lee Gurcharan Das Jules Verne 6000 6000 http://sapient1.com in Stock 499 Description No 1 Description No 1 Description No 1 Description No 1 Description No 1 999 - Salman Rushdie Harper Lee Gurcharan Das Jules Verne - Salman Rushdie Harper Lee Gurcharan Das Jules Verne these Description No 1 Description No 1 Description No 1 Description No 1 Description No 1 499 http://sapient1.com in Stock Please explain the steps to apply stop words for string type if possible. -- View this message in context: http://www.nabble.com/Applying-Stop-words-for-Field-Type-String-tp19722176p19722176.html Sent from the Solr - User mailing list archive at Nabble.com.
How Synonyms work in Solr
Hi,
Please explain that how the below mentioned synonyms patterns work in Solr
Search as there exists several seperators for synonym patterns:
1.
#Explicit mappings match any token sequence on the LHS of "=>"
#and replace with all alternatives on the RHS. These types of mappings
#ignore the expand parameter in the schema.
#Examples:
i-pod, i pod => ipod,
sea biscuit, sea biscit => seabiscuit
2.
#Equivalent synonyms may be separated with commas and give
#no explicit mapping. In this case the mapping behavior will
#be taken from the expand parameter in the schema. This allows
#the same synonym file to be used in different synonym handling strategies.
#Examples:
ipod, i-pod, i pod
foozball , foosball
universe , cosmos
3.
# If expand==true, "ipod, i-pod, i pod" is equivalent to the explicit
mapping:
ipod, i-pod, i pod => ipod, i-pod, i pod
# If expand==false, "ipod, i-pod, i pod" is equivalent to the explicit
mapping:
ipod, i-pod, i pod => ipod
4.
#multiple synonym mapping entries are merged.
foo => foo bar
foo => baz
#is equivalent to
foo => foo bar, baz
5.
Explain the meaning of this pattern:
a\=>a => b\=>b
a\,a => b\,b
Questions:
A) Among the following what all characters works as delimeters :
Whitespace(" ") comma(",") "=>" "\" "/"
B) Also, please let us know whether there exists certain other patterns
apart from the above mentioned ones.
C) In the pattern : ipod, i-pod, i pod
Here how we will determine that "i pod" has to be treated as a single
word though it contains Whitespace.
--
View this message in context:
http://www.nabble.com/How-Synonyms-work-in-Solr-tp20014192p20014192.html
Sent from the Solr - User mailing list archive at Nabble.com.
Getting Status 400 - org.apache.lucene.queryParser.ParseException on submitting a search query having ^ symbol through Web Application
Hi,
I am getting the following error while triggering a search query through the
web application :
HTTP Status 400 - org.apache.lucene.queryParser.ParseException: Cannot parse
'Sigma Survey for Police
Officers&field=Index_Type_s:productIndex&field=productType_s:product&field=Index_Type_s:productIndex&field=productType_s:program&field=Index_Type_s:productIndex&field=productType_s:course^1000&qt=dismaxrequest&hl=true':
Encountered ":" at line 1, column 84. Was expecting one of: ...
... ... "+" ... "-" ... "(" ... "*" ... "^" ... ...
... ... ... ... "[" ... "{" ...
...
Search Query URL :
http://localhost:8080/apache-solr-1.3.0/core51043/select/?
q=The Rough Guide to Film Musicals
1&field=Index_Type_s:productIndex&field=Index_Type_s:productIndex&field=productType_s:course^1000&qt=dismaxrequest&hl=true
(The same query returns proper result when hit on the Apache Solr server
directly)
In the application I m applying URLEncoding on the search string thus the
entire search string gets converted into :
http://localhost:8080/apache-solr-1.3.0/core51043/select/?
q=Sigma+Survey+for+Police+Officers%26field%3DIndex_Type_s%3AproductIndex%26field%3DproductType_s%3Aproduct%26field%3DIndex_Type_s%3AproductIndex%26field%3DproductType_s%3Aprogram%26field%3DIndex_Type_s%3AproductIndex%26field%3DproductType_s%3Acourse%5E1000%26qt%3Ddismaxrequest%26hl%3Dtrue
The error comes only when ^1000 gets appended into the search string , issue
persists even if even URLEncoding is applied.
Please provide suggestion for the same.
Regards,
Payal
--
View this message in context:
http://www.nabble.com/Getting-Status-400---org.apache.lucene.queryParser.ParseException-on-submitting-a-search-query-having-%5E-symbol-through-Web-Application-tp20419668p20419668.html
Sent from the Solr - User mailing list archive at Nabble.com.
Sorting is not taking place on the search results when query consists of "qt " parameter
Hi, I am trying to sort the search results on the basis of some fields of my XML document. SAMPLE RESULT XML document : productIndex 5000 productIndex 799 baraaa 5000 5000 sapient123 in Stock 4999 In Search of the Shape of the Universe 799 baraaa In Search of the Shape of the Universe 4999 sapient123 in Stock 2008-10-22T01:35:05.952Z productIndex 6000 - productIndex 799 coffee 6000 6000 http://sapient1.com in Stock 4999 Object 799 coffee Object 4999 http://sapient1.com in Stock 2008-10-22T01:35:05.983Z Sorting works fine when i give it simply like (index field used for sorting : "prdMainTitle_s") : http://delpearsonwebapps:8080/apache-solr-1.3.0/core51043/select/?q=sapient;prdMainTitle_s%20asc Issue : As i provide the parameter qt within the query , search response result comes out to be none. Sample query with qt parameter : http://delpearsonwebapps:8080/apache-solr-1.3.0/core51043/select/?q=sapient;prdMainTitle_s%20asc&qt=dismaxrequest requestHandler name="dismaxrequest" class="solr.DisMaxRequestHandler" has neen mentioned below within the solrConfig file content where all the fields which are indexed are mentioned within the tag :: explicit explicit 0.01 statusName_s^1.0 productId_s^1.0 iSBN10_s^1.0 iSBN13_s^1.0 prdMainTitle_s^1.0 prdKeywords_s^1.0 productDescription_s^1.0 prdMainSubTitle_s^1.0 contentTypeId_s^1.0 english^90 hindi^123 Glorious^2000 highlighting^1000 maths^100 ab^12 erer^4545 *,score dismax explicit 0.01 2<-1 5<-2 6<90% 100 *:* text features name 0 name regex Please let me know whether I need to provide some configuration changes in the solrConfig.XML for sorting related settings or Have to provide the indexed fields name in this file on which I want to perform sorting. Thanks in advance. -- View this message in context: http://www.nabble.com/Sorting-is-not-taking-place-on-the-search-results-when-query-consists-of-%22qt-%22-parameter-tp20481589p20481589.html Sent from the Solr - User mailing list archive at Nabble.com.
To associate sort criteria with the search query having fields of type String( required to be treated as numeric)
Hi, I m trying to apply sort on the search query which includes fields of type string, but i want some fields to be treated as of type float/integer by solr since the field basicly contains numeric value but have been defined as of type string in solrCofig.XML Currently sorting on these fileds occur on Lexicographical pattern whereby 3888 is considered to be smaller than 450 thus wrong results are getting displayed. Is there some way / syntax which can be given in the search query so that the string type field can be treated as numeric and the correct results can be displayed. Thanks in advance. -- View this message in context: http://www.nabble.com/To-associate-sort-criteria-with-the-search--query-having-fields-of-type-String%28-required-to-be-treated-as-numeric%29-tp20481627p20481627.html Sent from the Solr - User mailing list archive at Nabble.com.
Issue with Search when using wildcard(*) in search term.
Hi All, I am searching a term on Solr by using wildcard character "*" like this : http://delpearsonwebapps:8080/apache-solr-1.3.0/core51043/select/?q=alle* here the search term(word) is : alle* This query gives me proper result , but as i give dismaxrequest as parameter in the query , no results are returned , query with dismax parameter goes like this : http://delpearsonwebapps:8080/apache-solr-1.3.0/core51043/select/?q=alle*&qt=dismaxrequest Can anybody let me know the reason behind this behavior, also do I need to make any changes in my SolrConfig.XML in order to make the query run with both Wildcard as well as dismaxrequest. Thanks in advance. Payal -- View this message in context: http://www.nabble.com/Issue-with-Search-when-using-wildcard%28*%29-in-search-term.-tp20914102p20914102.html Sent from the Solr - User mailing list archive at Nabble.com.
Can we extract contents from two Core folders
Hi All, Issue: Need to fetch the data available in different core folders. Scenario: We are storing the information on different core folders specific to website ids (such as CoreUSA,CoreUK,CoreIndia ..). Thus information specific to any region get store in specific core folder. for e.g. for india specific information, CoreIndia folder is used. Now the requirement is that, we have to access the information stored in multiple cores that is CoreUSA and CoreUK folders simultaneously. Is it possible to do so and if what is the mechanism. Thanks in advance Payal -- View this message in context: http://www.nabble.com/Can-we-extract-contents-from-two-Core-folders-tp20933745p20933745.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Can we extract contents from two Core folders
Hi, Will you please explain what exactly you mean by : Distributed search over the cores. Please provide some context around this. Thanks markrmiller wrote: > > payalsharma wrote: >> Hi All, >> >> Issue: Need to fetch the data available in different core folders. >> Scenario: >> We are storing the information on different core folders specific to >> website >> ids (such as CoreUSA,CoreUK,CoreIndia ..). Thus information specific to >> any >> region get store in specific core folder. for e.g. for india specific >> information, CoreIndia folder is used. >> >> Now the requirement is that, we have to access the information stored in >> multiple cores that is CoreUSA and CoreUK folders simultaneously. >> Is it possible to do so and if what is the mechanism. >> >> Thanks in advance >> Payal >> > Try distributed search over the cores. > > -- View this message in context: http://www.nabble.com/Can-we-extract-contents-from-two-Core-folders-tp20933745p20937150.html Sent from the Solr - User mailing list archive at Nabble.com.
Master Slave Solr Replication Automation
We have a requirement of replicating data from one Solr set on a Linux Box to
Second Solr on another Linux box. In order to achieve the same we will use
the SolrCollectionDistributionScripts(snapshooter, snappuller etc) and rsync
utility.
Configurations:
1. Apache Solr 1.3.0
2. Machines : Linux
3. Master Slave : 1 Master and 1 slave
Settings done at our end:
Solr on the both Linux boxes contains multiple cores. We have disintegrated
the data to be indexed among multicores, sample solr path of data folder for
the same is like:
Path :: {SOLR_HOME}/solr/multicore//data
Sample :
/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/data
* SOLR_HOME :: /apache-solr-1.3.0/example/
** multi_corename :: CORE_WWW.ABCD.COM
Thus we will be going to have multiple cores on master as well as slaves
servers
As mentioned on http:// http://wiki.apache.org/solr/CollectionDistribution
:For the Solr distribution scripts, the name of the index directory can be
defined by the environment variable data_dir in the configuration file
conf/scripts.conf
Example conf/scripts.conf file on slave solr server :
user=
solr_hostname=localhost
solr_port=8080
rsyncd_port=18983
data_dir=${SOLR_HOME}/solr/muticore/CORE_WWW.ABCD.COM /data
webapp_name=solr
master_host=10.x.xx.xxx
master_data_dir=${SOLR_HOME}/solr/muticore/CORE_WWW.ABCD.COM/data
master_status_dir=${SOLR_HOME}/solr/muticore/CORE_WWW.ABCD.COM /status
The index directory name mentioned above should match the value used by the
Solr server which is defined in solr/conf/solrconfig.xml.
Following are few queries:
1. Please confirm whether the tag entry :
In solrconfig.xml should match for the Slave solr server / master solr
server in accordance to the scripts.conf configuration settings.
2. Also let us know whether some specific handling has to be done in case of
using multi cores during replication.
3. Are there any pitfalls in using the solr distribution scripts and rsync
utility.
Please throw some light on the queries.
--
View this message in context:
http://www.nabble.com/Master-Slave-Solr-Replication-Automation-tp23158672p23158672.html
Sent from the Solr - User mailing list archive at Nabble.com.
Snapinstaller on slave solr server | Can not connect to solr server issue
Hi All, I m facing an issue while running snapinstaller script on the Slave server, scripts installs the latest snapshot , but creates issue while making connectivity to the solr server , logs for the same from snapinstaller.log : 2009/04/28 18:48:03 command: /opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/bin/snapinstaller -u webuser 2009/04/28 18:48:16 installing snapshot /opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/data/snapshot.20090428180619 2009/04/28 18:48:21 notifing Solr to open a new Searcher 2009/04/28 18:48:21 failed to connect to Solr server 2009/04/28 18:48:21 snapshot installed but Solr server has not open a new Searcher 2009/04/28 18:48:21 failed (elapsed time: 18 sec) I ensured that slave solr server was in running state before calling ... snappuller and snapinstaller scripts. As a result of this issue Slave server's Collection was not displaying the indexes of latest installed snapshot, As a temporary solution, I restarted the Slave server and Collection got refreshed. Can anybody let me know the probable reason of this behavior. -- View this message in context: http://www.nabble.com/Snapinstaller-on-slave-solr-server-%7C-Can-not-connect-to-solr-server-issue-tp23278187p23278187.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Snapinstaller on slave solr server | Can not connect to solr server issue
To add to that :
This issue was coming because of the commit script called internally by
snapinstaller . Commit script creates the solr url to do the comit as shown
below:
curl_url=http://${solr_hostname}:${solr_port}/${webapp_name}/update
commitscript logs:
2009/04/28 18:48:21 started by root
2009/04/28 18:48:21 command:
/opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.PUFFIN.CO.UK/bin/commit
2009/04/28 18:48:21 commit request to Solr at
http://delpearsondm:8080/apache-solr-1.3.0/update failed:
2009/04/28 18:48:21 Apache Tomcat/6.0.18 - Error
reportHTTP Status 400 - Missing solr core name in
pathtype Status reportmessage
Missing solr core name in pathdescription The request sent
by the client was syntactically incorrect (Missing solr core name in
path).Apache
Tomcat/6.0.18
2009/04/28 18:48:21 failed (elapsed time: 0 sec)
Solr server set at our end contains multi cores, thus forms the URL like :
http://:8080/apache-solr-1.3.0/CORE_WWW.ABCD.COM/update
The Core name is not getting appended in the commit script.
Please let me know whether I need to change the commit script to accomodate
the core name in URL formed, or there is some alternate way to achieve the
same without modifying the script.
Thanks,
Payal
payalsharma wrote:
>
> Hi All,
>
> I m facing an issue while running snapinstaller script on the Slave
> server, scripts installs the latest snapshot , but creates issue while
> making connectivity to the solr server , logs for the same from
> snapinstaller.log :
>
> 2009/04/28 18:48:03 command:
> /opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/bin/snapinstaller
> -u webuser
> 2009/04/28 18:48:16 installing snapshot
> /opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/data/snapshot.20090428180619
> 2009/04/28 18:48:21 notifing Solr to open a new Searcher
> 2009/04/28 18:48:21 failed to connect to Solr server
> 2009/04/28 18:48:21 snapshot installed but Solr server has not open a new
> Searcher
> 2009/04/28 18:48:21 failed (elapsed time: 18 sec)
>
> I ensured that slave solr server was in running state before calling ...
> snappuller and snapinstaller scripts.
>
> As a result of this issue Slave server's Collection was not displaying the
> indexes of latest installed snapshot,
> As a temporary solution, I restarted the Slave server and Collection got
> refreshed.
>
>
> Can anybody let me know the probable reason of this behavior.
>
--
View this message in context:
http://www.nabble.com/Snapinstaller-on-slave-solr-server-%7C-Can-not-connect-to-solr-server-issue-tp23278187p23279140.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Snapinstaller on slave solr server | Can not connect to solr server issue
To add to that :
This issue was coming because of the commit script called internally by
snapinstaller . Commit script creates the solr url to do the comit as shown
below:
curl_url=http://${solr_hostname}:${solr_port}/${webapp_name}/update
commitscript logs:
2009/04/28 18:48:21 started by root
2009/04/28 18:48:21 command:
/opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/bin/commit
2009/04/28 18:48:21 commit request to Solr at
http:///apache-solr-1.3.0/update failed:
2009/04/28 18:48:21 Apache Tomcat/6.0.18 - Error
reportHTTP Status 400 - Missing solr core name in
pathtype Status reportmessage
Missing solr core name in pathdescription The request sent
by the client was syntactically incorrect (Missing solr core name in
path).Apache
Tomcat/6.0.18
2009/04/28 18:48:21 failed (elapsed time: 0 sec)
Solr server set at our end contains multi cores, thus forms the URL like :
http://:8080/apache-solr-1.3.0/CORE_WWW.ABCD.COM/update
The Core name is not getting appended in the commit script.
Please let me know whether I need to change the commit script to accomodate
the core name in URL formed, or there is some alternate way to achieve the
same without modifying the script.
Thanks,
Payal
payalsharma wrote:
>
> Hi All,
>
> I m facing an issue while running snapinstaller script on the Slave
> server, scripts installs the latest snapshot , but creates issue while
> making connectivity to the solr server , logs for the same from
> snapinstaller.log :
>
> 2009/04/28 18:48:03 command:
> /opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/bin/snapinstaller
> -u webuser
> 2009/04/28 18:48:16 installing snapshot
> /opt/apache-solr-1.3.0/example/solr/multicore/CORE_WWW.ABCD.COM/data/snapshot.20090428180619
> 2009/04/28 18:48:21 notifing Solr to open a new Searcher
> 2009/04/28 18:48:21 failed to connect to Solr server
> 2009/04/28 18:48:21 snapshot installed but Solr server has not open a new
> Searcher
> 2009/04/28 18:48:21 failed (elapsed time: 18 sec)
>
> I ensured that slave solr server was in running state before calling ...
> snappuller and snapinstaller scripts.
>
> As a result of this issue Slave server's Collection was not displaying the
> indexes of latest installed snapshot,
> As a temporary solution, I restarted the Slave server and Collection got
> refreshed.
>
>
> Can anybody let me know the probable reason of this behavior.
>
--
View this message in context:
http://www.nabble.com/Snapinstaller-on-slave-solr-server-%7C-Can-not-connect-to-solr-server-issue-tp23278187p23279184.html
Sent from the Solr - User mailing list archive at Nabble.com.
