Solr PingQuery
Hi, I use SolrPingResponse.getStatus method to start indexing to solr. I use SolrCloud with external zookeeper If i send it to the Zookeeper, if zookeeper is down, it returns NOTOK. But if one of my solr is up and second solr is down, the Ping returns OK status. Is this the behavior? Thanks, Prasi
MySQL Data import handler
Hi, I am new to solar and trying for MySQL data import handler. I have 3 tables in mysql. 1) user 2) mast_data 3) child_data The child_data table has foreign key ref of master_data and user. In child_data table, i have the following columns. 1) date 2) fee 3) location 4) type 5) text 6) user_id 7) master_data_id In the user table, display name field might get changed at any time. In the master table certain fields must be indexed in solar. So combining these 3 tables and trying to index . this is for search page ... the data will get added to these tables at any time and also the fields might get changed at any time I am not sure, how to keep the solr updated whenever there is any change in the table ... Also, looking for deleting the index in solr based on the date field Please help me on this Thanks, Baskar.S
Re: MySQL Data import handler
On 14 September 2013 18:46, Baskar Sikkayan wrote: > Hi, > I am new to solar and trying for MySQL data import handler. > > I have 3 tables in mysql. [...] Your question is unclear, and you would probably benefit by doing some basic homework on Solr and importing data into it. Please start from the Solr Wiki: http://wiki.apache.org/solr/ For indexing from a database, you can use the DataImportHandler: http://wiki.apache.org/solr/DataImportHandler http://wiki.apache.org/solr/DIHQuickStart What you want should be pretty straightforward. After setting up basic data import, please refer to delta-import for adding new data to Solr: http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command Regards, Gora
Re: MySQL Data import handler
Hi Gora, Thanks a lot for your reply. My requirement is to combine 3 tables in mysql for search operation and planning to sync these 3 tables( not all the columns ) in Apache Solr. Whenever there is any change( adding a new row, deleting a row, modifying the column data( any column in the 3 tables ) ), the same has to updated in solr. Guess, for this requirement, instead of going with delta-import, Apachae Solar java client will be of useful. Could you please share your view? Thanks, Baskar.S On Sat, Sep 14, 2013 at 7:25 PM, Gora Mohanty wrote: > On 14 September 2013 18:46, Baskar Sikkayan wrote: > > Hi, > > I am new to solar and trying for MySQL data import handler. > > > > I have 3 tables in mysql. > [...] > > Your question is unclear, and you would probably benefit > by doing some basic homework on Solr and importing > data into it. Please start from the Solr Wiki: > http://wiki.apache.org/solr/ > > For indexing from a database, you can use the DataImportHandler: > http://wiki.apache.org/solr/DataImportHandler > http://wiki.apache.org/solr/DIHQuickStart > What you want should be pretty straightforward. After > setting up basic data import, please refer to delta-import > for adding new data to Solr: > http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command > > Regards, > Gora >
Re: MySQL Data import handler
On 14 September 2013 20:07, Baskar Sikkayan wrote: > Hi Gora, > Thanks a lot for your reply. > My requirement is to combine 3 tables in mysql for search operation and > planning to sync these 3 tables( not all the columns ) in Apache Solr. > Whenever there is any change( adding a new row, deleting a row, modifying > the column data( any column in the 3 tables ) ), the same has to updated in > solr. Guess, for this requirement, instead of going with delta-import, > Apachae Solar java client will be of useful. [...] Yes, if you are comfortable with programming in Java, the Solr client would be a good alternative, though the DataImportHandler can also do what you want. Regards, Gora
Re: Solr Patent
On 9/13/2013 9:14 PM, Zaizen Ushio wrote: Hello I have a question about patent. I believe Apache license is protecting Solr developers from patent issue in Solr community. But is there any case that Solr developer or Solr users are alleged by outside of Solr Community? Is there any cases somebody experienced? Any advice is appreciated. Thanks, Zaizen Zaizen - I doubt you will get legal advice from this community. If you do get any other advice than to consult a lawyer, you should ignore it and consult a lawyer. Or move to New Zealand - I hear they outlawed software patents there. See that's just the sort of unhelpful legal advice you're likely to get here :) -Mike
Re: Solr Patent
You can ask on this site http://patents.stackexchange.com/ On Sat, Sep 14, 2013 at 10:03 AM, Michael Sokolov wrote: > On 9/13/2013 9:14 PM, Zaizen Ushio wrote: >> >> Hello >> I have a question about patent. I believe Apache license is protecting >> Solr developers from patent issue in Solr community. But is there any case >> that Solr developer or Solr users are alleged by outside of Solr Community? >> Is there any cases somebody experienced? Any advice is appreciated. >> >> Thanks, Zaizen >> >> >> > Zaizen - I doubt you will get legal advice from this community. If you do > get any other advice than to consult a lawyer, you should ignore it and > consult a lawyer. Or move to New Zealand - I hear they outlawed software > patents there. See that's just the sort of unhelpful legal advice you're > likely to get here :) > > -Mike -- sk...@alum.mit.edu (604) 446-2460
Re: Re: Unable to getting started with SOLR
If you're using the default jetty container, there's no log unless you set it up, the content is echoed to the screen. About a zillion people have downloaded this and started it running without issue, so you need to give us the exact steps you followed. If you checked the code out from SVN, you need to build it, go into /solr and execute ant example dist the "dist" bit isn't strictly necessary, but it builds the jars that you link to if you try to develop custom plugins etc. Best, Erick On Fri, Sep 13, 2013 at 3:56 AM, Rah1x wrote: > I have the same issue can anyone tell me if they found a solution? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Unable-to-getting-started-with-SOLR-tp3497276p4089761.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: what does "UnInvertedField; UnInverted multi-valued field" means and how to fix it
This is totally weird. Can you give us the exact
command you are using?
Best
Erick
On Fri, Sep 13, 2013 at 8:15 AM, Raheel Hasan wrote:
> Hi guyz,
>
> I have an issue here in between Solr Core and Data Indexing:
>
> When I build some index from fresh setup, everything is fine: all queries
> and additional/update indexing, everything runs is fine. But when I reload
> the Core, the solr stops from that point onward forever.
>
> All i get is this line as the last line of the solr log after the issue as
> occurred:
>
> UnInvertedField; UnInverted multi-valued field
>
> {field=prod_cited_id,memSize=4880,tindexSize=40,time=4,phase1=4,nTerms=35,bigTerms=4,termInstances=36,uses=0}
>
> Furthermore, the only way to get things working again, would be to delete
> the "data" folder inside "solr/{myCore}/"...
>
>
> So can anyone help me beat this issue and get things working again? I cant
> afford this issue when the system is LIVE..
>
> Thanks a lot.
>
> --
> Regards,
> Raheel Hasan
>
Re: Solr wildcard search
Also be aware that some analysis steps may not be performed on wildcards. The filter has to be MultTermAware. See: https://wiki.apache.org/solr/MultitermQueryAnalysis and http://searchhub.org/2011/11/29/whats-with-lowercasing-wildcard-multiterm-queries-in-solr/ Best, Erick On Fri, Sep 13, 2013 at 12:12 PM, Jack Krupansky wrote: > Wildcard applies only to a single term. The escaped space suggests that > you are trying to match a wildcard on multiple terms. > > Try the contrib complex phrase query parser. > > -- Jack Krupansky > > -Original Message- From: Prasi S > Sent: Friday, September 13, 2013 6:37 AM > To: solr-user@lucene.apache.org > Subject: Solr wildcard search > > > Hi all, > I am working with wildcard queries and few things are confusing. > > 1. Does a wildcard search omit the analysers on a particular field? > > 2. I have searched for > q=google\ technology - >gives result > q=google technology -> Gives results > q=google tech* -> gives results > q=google\ tech* -> 0 results. The debug Query for the last query is name="parsedquery_toString">**text:google tech* > > Why does this happen. > > > Thanks, > Prasi >
Re: Committing when indexing in parallel
First, there is no blocking, your P1 and P2 indexing processes should continue on just fine. That nit aside... No matter how many processes are indexing to Solr, the documents are treated as though they came in serial requests as far as committing is concerned. Let's say you send the data to Solr from P1 and P2 interleaved and let's claim they're perfectly interleaved as P1.1 P2.1 P1.2 P2.2 commit from p1. All the docs are committed from both processes Committing from the indexing side is simply flushing anything currently in memory to the current segments and opening a new segment where new updates will be written. There's a tiny bit of delay while this happens I guess.. HTH Erick On Fri, Sep 13, 2013 at 5:21 PM, Phani Chaitanya wrote: > > I'm wondering what happens to commit while we are indexing in parallel in > Solr. Are the indexing update requests blocked until the commit finishes ? > > Lets say I've a process P1 which issued a commit request and there is > another process P2 which is still indexing to the same index. What happens > to the index in that scenario. Are the P2 indexing requests blocked until > P1 > commit request finishes ? > > I'm just wondering about what is the behavior of Solr in the above case. > > > > - > Phani Chaitanya > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Committing-when-indexing-in-parallel-tp4089953.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Committing when indexing in parallel
Thanks Erick. I completely did not get the point you are trying to make w.r.t my question. I'll add it again according to your example. In the example you gave w.r.t P1.1, P2.1, P1.2, P2.2 and P1 issues a commit, I understand that all documents are committed. Now what happens when P1 issues a commit and P2 sends another document, say P2.3, to index at the same time ? As you said the requests are treated as if they are serial - if P1.commit is the first one among P1.commit & P2.3, P2.3 will be indexed into a new segment ? Regards, Phani. - Phani Chaitanya -- View this message in context: http://lucene.472066.n3.nabble.com/Committing-when-indexing-in-parallel-tp4089953p4090120.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Java Client
Hi,
I am new to Solr and trying to use Solr java client instead of using the
Data handler.
Is there any configuration i need to do for this?
I got the following sample code.
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
server.commit(); // periodically flush
I am confused here. I am going to index 3 different tables for 3 different
kind of searches. Here i dont have any option to differentiate 3 kind of
indexes.
Am i missing anything here. Could anyone please shed some light here?
Thanks,
Baskar.S
Re: MySQL Data import handler
Hi, If i am supposed to go with Java client, should i still do any configurations in solrconfig.xml or schema.xml. Thanks, Baskar.S On Sat, Sep 14, 2013 at 8:46 PM, Gora Mohanty wrote: > On 14 September 2013 20:07, Baskar Sikkayan wrote: > > Hi Gora, > > Thanks a lot for your reply. > > My requirement is to combine 3 tables in mysql for search operation and > > planning to sync these 3 tables( not all the columns ) in Apache Solr. > > Whenever there is any change( adding a new row, deleting a row, modifying > > the column data( any column in the 3 tables ) ), the same has to updated > in > > solr. Guess, for this requirement, instead of going with delta-import, > > Apachae Solar java client will be of useful. > [...] > > Yes, if you are comfortable with programming in Java, > the Solr client would be a good alternative, though the > DataImportHandler can also do what you want. > > Regards, > Gora >
Re: Committing when indexing in parallel
Hi, As per my knowledge, any number of requests can be issued in parallel for index the documents. Any commit request will write them to index. So if P1 issued a commit then all documents of P2 those are eligible get committed and remaining documents will get committed on other commit request. Rgds AJ On 14-Sep-2013, at 2:51, Phani Chaitanya wrote: > > I'm wondering what happens to commit while we are indexing in parallel in > Solr. Are the indexing update requests blocked until the commit finishes ? > > Lets say I've a process P1 which issued a commit request and there is > another process P2 which is still indexing to the same index. What happens > to the index in that scenario. Are the P2 indexing requests blocked until P1 > commit request finishes ? > > I'm just wondering about what is the behavior of Solr in the above case. > > > > - > Phani Chaitanya > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Committing-when-indexing-in-parallel-tp4089953.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: MySQL Data import handler
Hi Baskar, Just create a single schema.xml which should contains required fields from 3 tables. Add a status column to child table.i.e 1 = add 2 = update 3 = delete 4 = indexed Etc Write a program using solrj which will read the status and do thing accordingly. Rgds AJ On 15-Sep-2013, at 5:46, Baskar Sikkayan wrote: > Hi, > If i am supposed to go with Java client, should i still do any > configurations in solrconfig.xml or schema.xml. > > Thanks, > Baskar.S > > > On Sat, Sep 14, 2013 at 8:46 PM, Gora Mohanty wrote: > >> On 14 September 2013 20:07, Baskar Sikkayan wrote: >>> Hi Gora, >>>Thanks a lot for your reply. >>> My requirement is to combine 3 tables in mysql for search operation and >>> planning to sync these 3 tables( not all the columns ) in Apache Solr. >>> Whenever there is any change( adding a new row, deleting a row, modifying >>> the column data( any column in the 3 tables ) ), the same has to updated >> in >>> solr. Guess, for this requirement, instead of going with delta-import, >>> Apachae Solar java client will be of useful. >> [...] >> >> Yes, if you are comfortable with programming in Java, >> the Solr client would be a good alternative, though the >> DataImportHandler can also do what you want. >> >> Regards, >> Gora >>
Re: Solr Java Client
Add a field called "source" in schema.xml and value would be your table names.
Rgds
AJ
On 15-Sep-2013, at 5:38, Baskar Sikkayan wrote:
> Hi,
> I am new to Solr and trying to use Solr java client instead of using the
> Data handler.
> Is there any configuration i need to do for this?
>
> I got the following sample code.
>
> SolrInputDocument doc = new SolrInputDocument();
>
> doc.addField("cat", "book");
> doc.addField("id", "book-" + i);
> doc.addField("name", "The Legend of the Hobbit part " + i);
> server.add(doc);
> server.commit(); // periodically flush
>
> I am confused here. I am going to index 3 different tables for 3 different
> kind of searches. Here i dont have any option to differentiate 3 kind of
> indexes.
> Am i missing anything here. Could anyone please shed some light here?
>
> Thanks,
> Baskar.S
Re: Solr Java Client
How to add index to 3 diff tables from java ...
On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha wrote:
> Add a field called "source" in schema.xml and value would be your table
> names.
>
>
>
> Rgds
> AJ
>
> On 15-Sep-2013, at 5:38, Baskar Sikkayan wrote:
>
> > Hi,
> > I am new to Solr and trying to use Solr java client instead of using the
> > Data handler.
> > Is there any configuration i need to do for this?
> >
> > I got the following sample code.
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> > server.commit(); // periodically flush
> >
> > I am confused here. I am going to index 3 different tables for 3
> different
> > kind of searches. Here i dont have any option to differentiate 3 kind of
> > indexes.
> > Am i missing anything here. Could anyone please shed some light here?
> >
> > Thanks,
> > Baskar.S
>
Re: Solr PingQuery
On 9/14/2013 6:57 AM, Prasi S wrote: > I use SolrPingResponse.getStatus method to start indexing to solr. I use > SolrCloud with external zookeeper > > If i send it to the Zookeeper, if zookeeper is down, it returns NOTOK. > > But if one of my solr is up and second solr is down, the Ping returns OK > status. If your zookeeper is completely down or does not have quorum, then SolrCloud isn't going to work right, so a ping response of NOTOK is correct. A fully redundant zookeeper ensemble is at least three machines, preferably an odd number. You can run zookeeper on the same hardware as Solr, but it is recommended that it be a standalone process. You should not run the solr embedded zookeeper (-DzkRun) for production, because when you shutdown or restart Solr, the embedded zookeeper also goes down. With three machines in the zookeeper ensemble, you can have one of them go down and everything keeps working perfectly. If you want to know why an odd number is recommended, consider a scenario with four zookeepers instead of three. In order to have quorum, you need to have half the servers plus one available. On a four-server ensemble, that works out so that three of them have to be running. You are no better off than if you have three servers, because in either scenario you can only have one failure. On top of that, you have an extra possible point of failure and you're using more resources, like switchports and power. With five servers, two can go down and quorum will be maintained. If you only have two zookeepers, they both must be operational in order to have quorum. If one of them were to fail, quorum would be lost and SolrCloud would stop working correctly. SolrCloud itself is also designed to deal with a failure of a single machine. A replicationFactor of at least two is required for that to work correctly. Thanks, Shawn
Re: Solr Java Client
Question is not clear to me. Please be more elaborative in your query. Why do
u want to store index to DB tables?
Rgds
AJ
On 15-Sep-2013, at 7:20, Baskar Sikkayan wrote:
> How to add index to 3 diff tables from java ...
>
>
> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha wrote:
>
>> Add a field called "source" in schema.xml and value would be your table
>> names.
>>
>>
>>
>> Rgds
>> AJ
>>
>> On 15-Sep-2013, at 5:38, Baskar Sikkayan wrote:
>>
>>> Hi,
>>> I am new to Solr and trying to use Solr java client instead of using the
>>> Data handler.
>>> Is there any configuration i need to do for this?
>>>
>>> I got the following sample code.
>>>
>>> SolrInputDocument doc = new SolrInputDocument();
>>>
>>> doc.addField("cat", "book");
>>> doc.addField("id", "book-" + i);
>>> doc.addField("name", "The Legend of the Hobbit part " + i);
>>> server.add(doc);
>>> server.commit(); // periodically flush
>>>
>>> I am confused here. I am going to index 3 different tables for 3
>> different
>>> kind of searches. Here i dont have any option to differentiate 3 kind of
>>> indexes.
>>> Am i missing anything here. Could anyone please shed some light here?
>>>
>>> Thanks,
>>> Baskar.S
>>
Re: Solr Java Client
Solr does not have tables, and you can't add an index.
Solr's data model is flat, like a single table with lots of columns. Think
about creating a view where each row has all the information for one search
result. It includes everything that is searched (indexed=true in the schema)
and returned (stored=true in the schema).
wunder
On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> Question is not clear to me. Please be more elaborative in your query. Why
> do u want to store index to DB tables?
>
> Rgds
> AJ
>
> On 15-Sep-2013, at 7:20, Baskar Sikkayan wrote:
>
>> How to add index to 3 diff tables from java ...
>>
>>
>> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha wrote:
>>
>>> Add a field called "source" in schema.xml and value would be your table
>>> names.
>>>
>>>
>>>
>>> Rgds
>>> AJ
>>>
>>> On 15-Sep-2013, at 5:38, Baskar Sikkayan wrote:
>>>
Hi,
I am new to Solr and trying to use Solr java client instead of using the
Data handler.
Is there any configuration i need to do for this?
I got the following sample code.
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
server.commit(); // periodically flush
I am confused here. I am going to index 3 different tables for 3
>>> different
kind of searches. Here i dont have any option to differentiate 3 kind of
indexes.
Am i missing anything here. Could anyone please shed some light here?
Thanks,
Baskar.S
>>>
Re: Committing when indexing in parallel
You really are barking up the wrong tree here. Solr is a search engine, designed for batch update and "eventual consistency". This fantasy you have of knowing exactly when a document is committed is completely inappropriate with Solr. Sure, you can in fact do a hard commit at any time to guarantee that recent updates are immediately searchable, but that is strongly discouraged for performance reasons, since Solr is batch update oriented. And the level of detail you are requesting is merely how Solr happens to work today and is not necessarily guaranteed for future releases - since the guaranteed model is only for commits with eventual consistency. It appears that you are trying to imagine Solr as a traditional, transaction-based database, when that is not the case. I've asked you before to disclose what problem you are really trying to solve, and so far you have not yet let us in on your secret. You are certainly welcome to peruse the Solr and Lucene source code if your goal is merely idle curiosity, but you really should not be designing a Solr-based application around this non-guaranteed level of detail. Soft commit, commit within, hard commit, real time get, and eventual consistency are the proper tools to design a Solr-based application around. And if you wish to have multiple, uncoordinated update streams, you need to relax your requirements for eventual consistency even further. In short, to summarize again, Solr is not a transaction-based database, but instead is a batch-oriented search engine with eventual consistency. Focus on exploiting Solr's strengths, not trying to treat Solr as something that it is not. -- Jack Krupansky -Original Message- From: Phani Chaitanya Sent: Saturday, September 14, 2013 7:36 PM To: solr-user@lucene.apache.org Subject: Re: Committing when indexing in parallel Thanks Erick. I completely did not get the point you are trying to make w.r.t my question. I'll add it again according to your example. In the example you gave w.r.t P1.1, P2.1, P1.2, P2.2 and P1 issues a commit, I understand that all documents are committed. Now what happens when P1 issues a commit and P2 sends another document, say P2.3, to index at the same time ? As you said the requests are treated as if they are serial - if P1.commit is the first one among P1.commit & P2.3, P2.3 will be indexed into a new segment ? Regards, Phani. - Phani Chaitanya -- View this message in context: http://lucene.472066.n3.nabble.com/Committing-when-indexing-in-parallel-tp4089953p4090120.html Sent from the Solr - User mailing list archive at Nabble.com.
requested url solr/update/extract not available on this server
I am working on Solr 4.2 on Windows 7. I am trying to index pdf files.I referred Solr Cookbook 4. Tomcat is using 8080 port number. I get this error:requested url solr/update/extract not available on this server When my curl is : curl "http://localhost:8080/solr/update/extract?literal.id=1&commit=true"; -F "myfile=@cookbook.pdf" There is no entry in log files. Please help. -- View this message in context: http://lucene.472066.n3.nabble.com/requested-url-solr-update-extract-not-available-on-this-server-tp4090153.html Sent from the Solr - User mailing list archive at Nabble.com.
