RE: One document inserted but nothing showing up ? SOLR 1.3
Even that doesn't work, How can I check properly, I did insert one document but I can't get it back ??? Feak, Todd wrote: > > Unless "q=ALL" is a special query I don't know about, the only reason you > would get results is if "ALL" showed up in the default field of the single > document that was inserted/updated. > > You could try a query of "*:*" instead. Don't forget to URL encode if you > are doing this via URL. > > -Todd > > > -Original Message- > From: sunnyfr [mailto:[EMAIL PROTECTED] > Sent: Thursday, October 23, 2008 9:17 AM > To: solr-user@lucene.apache.org > Subject: One document inserted but nothing showing up ? SOLR 1.3 > > > Hi > > Can somebody help me ? > How can I see all my documents, I just did a full import : > > Indexing completed. Added/Updated: 1 documents. Deleted 0 documents. > > > and when I do :8180/solr/video/select/?q=ALL, I've no result ? > > − > > 0 > 0 > − > > ALL > > > > > > Thanks a lot, > > -- > View this message in context: > http://www.nabble.com/One-document-inserted-but-nothing-showing-up---SOLR-1.3-tp20134357p20134357.html > Sent from the Solr - User mailing list archive at Nabble.com. > > > > -- View this message in context: http://www.nabble.com/One-document-inserted-but-nothing-showing-up---SOLR-1.3-tp20134357p20145343.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to search a DataImportHandler solr index
Hi Paul, thanks for the answer but unfortunately it doesn't work. I have the following: I have defined banner_type and size as: Now when I do a search with Solr, I get: 1 Campaign Name flash gif 50x50 100x100 50x50 100x100 While I was expecting that the size tags were inside the banner_type tags, something like: flash 50x50 100x100 gif 50x50 100x100 Am I doing something wrong or is it just not possible? Because with the output it generates now I cann't accurately find a campaign that has a flash banner of size 50x50 for example, because the size 50x50 could as well be from a gif banner. With the nested structure, I think it would be possible. At least if Solr can search this type of structure. Any tips are welcome. Thanks. Kind regards, Nick -- View this message in context: http://www.nabble.com/How-to-search-a-DataImportHandler-solr-index-tp20120698p20145974.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to search a DataImportHandler solr index
probably u can paste your data-config.xml with the queries etc --Noble On Fri, Oct 24, 2008 at 1:33 PM, Nick80 <[EMAIL PROTECTED]> wrote: > > Hi Paul, > > thanks for the answer but unfortunately it doesn't work. I have the > following: > > > > > > > > > > > > > > > I have defined banner_type and size as: > > multiValued="true" omitNorms="true" termVectors="true" /> > > Now when I do a search with Solr, I get: > > > > 1 > Campaign Name > > > flash > gif > > > > 50x50 > 100x100 > 50x50 > 100x100 > > > > > While I was expecting that the size tags were inside the banner_type tags, > something like: > > > flash > > 50x50 > 100x100 > > > gif > > 50x50 > 100x100 > > > > > Am I doing something wrong or is it just not possible? Because with the > output it generates now I cann't accurately find a campaign that has a flash > banner of size 50x50 for example, because the size 50x50 could as well be > from a gif banner. With the nested structure, I think it would be possible. > At least if Solr can search this type of structure. Any tips are welcome. > Thanks. > > Kind regards, > > Nick > -- > View this message in context: > http://www.nabble.com/How-to-search-a-DataImportHandler-solr-index-tp20120698p20145974.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
delta-import for XML files, Solr statistics
Hello, I have some questions about DataImportHandler and Solr statistics... 1.) I'm using the DataImportHandler for creating my Lucene index from XML files: ### $ cat data-config.xml ... ### No problems with this configuration - All works fine for full-imports, but... ===> What means 'rootEntity="false"' and 'dataSource="null"'? 2.) The documentation from DataImportHandler describes the index update process for SQL databases only... My scenario: - My application creates, deletes and modifies files from /tmp/files every night. - delta-import / DataImportHandler should "mirror" _all_ this changes to my lucene index (=> create, delete, update documents). ===> Is this possible with delta-import / DataImportHandler? ===> If not: Do you have any suggestions on how to do this? 3.) My scenario: - /tmp/files contains 682 'myDoc_.*\.xml' XML files. - Each XML file contains 12 XML elements (e.g. foo). - DataImportHandler transfer only 5 from this 12 elements to the lucene index. I don't understand the output from 'solr/dataimport' (=> status): ### ... 0 1363 0 2008-10-24 13:19:03 Indexing completed. Added/Updated: 681 documents. Deleted 0 documents. 2008-10-24 13:19:05 2008-10-24 13:19:05 0:0:2.648 ... ===> What is "Total Rows Fetched" rsp. what is a "row" in a XML file? An element? Why 1363? ===> Why shows the "Added/Updated" counter 681 and not 682? 4.) And my last questions about Solr statistics/informations... ===> Is it possible to get informations (number of indexed documents, stored values from documents etc.) from the current lucene index? ===> The admin webinterface shows 'numDocs' and 'maxDoc' in 'statistics/core'. Is 'numDocs' = number of indexed documents? What means 'maxDocs'? Thanks a lot! gisto -- GMX Kostenlose Spiele: Einfach online spielen und Spaß haben mit Pastry Passion! http://games.entertainment.gmx.net/de/entertainment/games/free/puzzle/6169196
Re: delta-import for XML files, Solr statistics
On Fri, Oct 24, 2008 at 5:14 PM, <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I have some questions about DataImportHandler and Solr statistics...
>
>
> 1.)
> I'm using the DataImportHandler for creating my Lucene index from XML files:
>
> ###
> $ cat data-config.xml
>
>
>
> processor="FileListEntityProcessor"
>baseDir="/tmp/files"
>fileName="myDoc_.*\.xml"
>newerThan="'NOW-30DAYS'"
>recursive="false"
>rootEntity="false"
>dataSource="null">
> url="${xmlFile.fileAbsolutePath}"
> processor="XPathEntityProcessor"
> forEach="/myDoc">
> ...
>
> ###
>
> No problems with this configuration - All works fine for full-imports, but...
>
> ===> What means 'rootEntity="false"' and 'dataSource="null"'?
It is a menace caused by 'sensible defaults'
An entity directly under the is a root entity. That means
that for each row emitted by the root entity one document is created
in Solr/Lucene . but as in this case we do not wish to make one
document per file. we wish to make one document per row emitted by the
entity 'myDoc' .Because the entity 'xmlFile' is not has
rootEntity=false the entity directly under it becomes a root entity
automatically and each row emitted by that becomes a document.
In most of the cases there is only one datasource (A JdbcDataSource)
and all entities just use them . So it is an overkill to ask them to
write the datSource. So we have chosen to implicitly assign the
datasource with no name to that entity. But in case of
FileListEntityProcessor a datasource is not necessary . But it won't
hurt even if you do not put dataSource=null . It just means that we
won't create a DataSource instance for that.
>
>
>
> 2.)
> The documentation from DataImportHandler describes the index update process
> for SQL databases only...
>
> My scenario:
> - My application creates, deletes and modifies files from /tmp/files every
> night.
> - delta-import / DataImportHandler should "mirror" _all_ this changes to my
> lucene index (=> create, delete, update documents).
The only Entityprocessor which supports delta is SqlEntityProcessor.
The XPathEntityProcessor has not implemented it , because we do not
know of a consistent way of finding deltas for XML. So ,
unfortunately,no delta support for XML. But that said you can
implement those methods in XPathEntityProcessor . The methods are
explained in EntityProcessor.java. if you have questions specific to
this I can help.Probably we can contribute it back
>
> ===> Is this possible with delta-import / DataImportHandler?
> ===> If not: Do you have any suggestions on how to do this?
>
>
>
> 3.)
> My scenario:
> - /tmp/files contains 682 'myDoc_.*\.xml' XML files.
> - Each XML file contains 12 XML elements (e.g. foo).
> - DataImportHandler transfer only 5 from this 12 elements to the lucene index.
>
>
> I don't understand the output from 'solr/dataimport' (=> status):
>
> ###
>
> ...
>
> 0
> 1363
> 0
> 2008-10-24 13:19:03
>
>Indexing completed. Added/Updated: 681 documents. Deleted 0 documents.
>
> 2008-10-24 13:19:05
> 2008-10-24 13:19:05
> 0:0:2.648
>
> ...
>
>
> ===> What is "Total Rows Fetched" rsp. what is a "row" in a XML file? An
> element? Why 1363?
> ===> Why shows the "Added/Updated" counter 681 and not 682?
rows fethed makes a lot of sense with SqlEntityProcessor. It is the
no:of rows fetched from DB . It is the cumulative no:of rows given out
by all entitiies put together. in your case it will be the total files
+ total rows emitted from the xml
Added updated is the no:of docs . How do you know the number is not accurate?
>
>
>
> 4.)
> And my last questions about Solr statistics/informations...
>
> ===> Is it possible to get informations (number of indexed documents, stored
> values from documents etc.) from the current lucene index?
> ===> The admin webinterface shows 'numDocs' and 'maxDoc' in
> 'statistics/core'. Is 'numDocs' = number of indexed documents? What means
> 'maxDocs'?
>
>
> Thanks a lot!
> gisto
> --
> GMX Kostenlose Spiele: Einfach online spielen und Spaß haben mit Pastry
> Passion!
> http://games.entertainment.gmx.net/de/entertainment/games/free/puzzle/6169196
>
--
--Noble Paul
Deadlock problem on searcher at warm up.
Hi everyone,
I'm implementing a search component inherited from SearchComponent .
This component has to build a data structure from the index. Like in
the SpellChecker, I trigger this building by giving a special argument
at query time (from the process method) and I'm using the searcher I
get like this:
RefCounted search = rb.req.getCore()
.getSearcher();
...
search.decref();
I included this component at the end of the chain in my search handler.
What I'd like to do is to trigger this building for a first time at
solr startup so I don't need to artificially trigger it for a first
time.
I though it'd be ok to trigger this the very first time the process
method is called by doing something like that:
private boolean firstTime= true ;
public void process(ResponseBuilder rb) throws IOException {
if ( firstTime ){
firstTime = false ;
buildMyStuff(rb) ;
}
}
The problem is that my method buildMyStuff hangs when calling
rb.req.getCore().getSearcher() ; ,
and I believe this is happening when the warm up queries are executed.
Furthermore, any regular queries on a solr instance like this would
hang and wait forever.
I there any way I can get around this problem, or is there a better
way to buildMyStuff a first time when solr is started up?
Cheers,
Jerome.
--
Jerome Eteve.
Chat with me live at http://www.eteve.net
[EMAIL PROTECTED]
Re: delta-import for XML files, Solr statistics
Thanks for your very fast response :-) > > 2.) > > The documentation from DataImportHandler describes the index update > process for SQL databases only... > > > > My scenario: > > - My application creates, deletes and modifies files from /tmp/files > every night. > > - delta-import / DataImportHandler should "mirror" _all_ this changes to > my lucene index (=> create, delete, update documents). > The only Entityprocessor which supports delta is SqlEntityProcessor. > The XPathEntityProcessor has not implemented it , because we do not > know of a consistent way of finding deltas for XML. So , > unfortunately,no delta support for XML. But that said you can > implement those methods in XPathEntityProcessor . The methods are > explained in EntityProcessor.java. if you have questions specific to > this I can help.Probably we can contribute it back > > > > ===> Is this possible with delta-import / DataImportHandler? > > ===> If not: Do you have any suggestions on how to do this? Ok so, at the moment I have to do a full-import to update my index. What happens with (user) queries while full-import is running? Does Solr block this queries the import is finished? Which configuration options control this behavior? > > My scenario: > > - /tmp/files contains 682 'myDoc_.*\.xml' XML files. > > - Each XML file contains 12 XML elements (e.g. foo). > > - DataImportHandler transfer only 5 from this 12 elements to the lucene > index. > > > > > > I don't understand the output from 'solr/dataimport' (=> status): > > > > ### > > > > ... > > > > 0 > > 1363 > > 0 > > 2008-10-24 13:19:03 > > > >Indexing completed. Added/Updated: 681 documents. Deleted 0 > documents. > > > > 2008-10-24 13:19:05 > > 2008-10-24 13:19:05 > > 0:0:2.648 > > > > ... > > > > > > ===> Why shows the "Added/Updated" counter 681 and not 682? > > Added updated is the no:of docs . How do you know the number is not > accurate? /tmp/files$ ls myDoc_*.xml | wc -l 682 But "Added/Updated" shows 681. Does this mean that one file has an XML error? But the statistic says "Total Documents Skipped" = 0?! > > 4.) > > And my last questions about Solr statistics/informations... > > > > ===> Is it possible to get informations (number of indexed documents, > stored values from documents etc.) from the current lucene index? > > ===> The admin webinterface shows 'numDocs' and 'maxDoc' in > 'statistics/core'. Is 'numDocs' = number of indexed documents? What means > 'maxDocs'? Do you have answers for this questions too? Bye, Simon -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer
timeouts
Hi We're running solr on a win 2k3 box under tomcat with about 100,000 records. When doing large updates of records via solr sharp, solr completely freezes and doesn't come back until we restart tomcat. This has only started happening since putting mysql on the same box (as a source of the data to update from). Are there any known issues with running solr and mysql on the same box? When it's frozen, the cpu usage is around 1-2% not exactly out of resources! Am i best using something else instead of tomcat? We're still trialling solr (presently, used for our main site search www.shoe-shop.com and search and navigation for our microsites ). It's an excellent search product, but I don't want to fork out on new hardware for it just yet - until i know how more about the performance and which environment i'm best to go for (win/linux). If anyone has any suggestions/needs more info, i'd be extremely grateful. Thanks Simon Simon Collins Systems Analyst Telephone: 01904 606 867 Fax Number: 01904 528 791 shoe-shop.com ltd Catherine House Northminster Business Park Upper Poppleton, YORK YO26 6QU www.shoe-shop.com This message (and any associated files) is intended only for the use of the individual or entity to which it is addressed and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Messages sent to and from us may be monitored. Internet communications cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Therefore, we do not accept responsibility for any errors or omissions that are present in this message, or any attachment, that have arisen as a result of e-mail transmission. If verification is required, please request a hard-copy version. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. (PAVD001) Shoe-shop.com Limited is a company registered in England and Wales with company number 03817232. Vat Registration GB 734 256 241. Registered Office Catherine House, Northminster Business Park, Upper Poppleton, YORK, YO26 6QU. This message has been scanned for malware by SurfControl plc. www.surfcontrol.com
Re: delta-import for XML files, Solr statistics
On Fri, Oct 24, 2008 at 6:07 PM, <[EMAIL PROTECTED]> wrote: > Thanks for your very fast response :-) > > > > > 2.) > > > The documentation from DataImportHandler describes the index update > > process for SQL databases only... > > > > > > My scenario: > > > - My application creates, deletes and modifies files from /tmp/files > > every night. > > > - delta-import / DataImportHandler should "mirror" _all_ this changes > to > > my lucene index (=> create, delete, update documents). > > The only Entityprocessor which supports delta is SqlEntityProcessor. > > The XPathEntityProcessor has not implemented it , because we do not > > know of a consistent way of finding deltas for XML. So , > > unfortunately,no delta support for XML. But that said you can > > implement those methods in XPathEntityProcessor . The methods are > > explained in EntityProcessor.java. if you have questions specific to > > this I can help.Probably we can contribute it back > > > > > > ===> Is this possible with delta-import / DataImportHandler? > > > ===> If not: Do you have any suggestions on how to do this? > > Ok so, at the moment I have to do a full-import to update my index. What > happens with (user) queries while full-import is running? Does Solr block > this queries the import is finished? Which configuration options control > this behavior? No queries to SOLR are not blocked during full import. > > > > > > My scenario: > > > - /tmp/files contains 682 'myDoc_.*\.xml' XML files. > > > - Each XML file contains 12 XML elements (e.g. foo). > > > - DataImportHandler transfer only 5 from this 12 elements to the lucene > > index. > > > > > > > > > I don't understand the output from 'solr/dataimport' (=> status): > > > > > > ### > > > > > > ... > > > > > > 0 > > > 1363 > > > 0 > > > 2008-10-24 13:19:03 > > > > > >Indexing completed. Added/Updated: 681 documents. Deleted 0 > > documents. > > > > > > 2008-10-24 13:19:05 > > > 2008-10-24 13:19:05 > > > 0:0:2.648 > > > > > > ... > > > > > > > > > ===> Why shows the "Added/Updated" counter 681 and not 682? > > > > Added updated is the no:of docs . How do you know the number is not > > accurate? > > > /tmp/files$ ls myDoc_*.xml | wc -l > 682 > > But "Added/Updated" shows 681. Does this mean that one file has an XML > error? But the statistic says "Total Documents Skipped" = 0?! It might be the case that somewhere there is a extra line in one of the XML files, a line like or something. > > > > > > > 4.) > > > And my last questions about Solr statistics/informations... > > > > > > ===> Is it possible to get informations (number of indexed documents, > > stored values from documents etc.) from the current lucene index? > > > ===> The admin webinterface shows 'numDocs' and 'maxDoc' in > > 'statistics/core'. Is 'numDocs' = number of indexed documents? What means > 'maxDocs'? > > Do you have answers for this questions too? > > Bye, > Simon > -- > Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! > Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer > -- Regards, Akshay Ukey.
Re: How to search a DataImportHandler solr index
Hi, below is a simplified copy of my data-config file: I have defined the following fields in schema.xml: Hope that makes it a bit clearer. Thanks. Kind regards, Nick -- View this message in context: http://www.nabble.com/How-to-search-a-DataImportHandler-solr-index-tp20120698p20149960.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: One document inserted but nothing showing up ? SOLR 1.3
Hi there, Are you sure you did a commit after your insertion ? On Fri, Oct 24, 2008 at 8:11 AM, sunnyfr <[EMAIL PROTECTED]> wrote: > > Even that doesn't work, > How can I check properly, I did insert one document but I can't get it back > ??? > > > Feak, Todd wrote: >> >> Unless "q=ALL" is a special query I don't know about, the only reason you >> would get results is if "ALL" showed up in the default field of the single >> document that was inserted/updated. >> >> You could try a query of "*:*" instead. Don't forget to URL encode if you >> are doing this via URL. >> >> -Todd >> >> >> -Original Message- >> From: sunnyfr [mailto:[EMAIL PROTECTED] >> Sent: Thursday, October 23, 2008 9:17 AM >> To: solr-user@lucene.apache.org >> Subject: One document inserted but nothing showing up ? SOLR 1.3 >> >> >> Hi >> >> Can somebody help me ? >> How can I see all my documents, I just did a full import : >> >> Indexing completed. Added/Updated: 1 documents. Deleted 0 documents. >> >> >> and when I do :8180/solr/video/select/?q=ALL, I've no result ? >> >> − >> >> 0 >> 0 >> − >> >> ALL >> >> >> >> >> >> Thanks a lot, >> >> -- >> View this message in context: >> http://www.nabble.com/One-document-inserted-but-nothing-showing-up---SOLR-1.3-tp20134357p20134357.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> >> >> > > -- > View this message in context: > http://www.nabble.com/One-document-inserted-but-nothing-showing-up---SOLR-1.3-tp20134357p20145343.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- Jerome Eteve. Chat with me live at http://www.eteve.net [EMAIL PROTECTED]
Re: Deadlock problem on searcher at warm up.
On Fri, Oct 24, 2008 at 8:21 AM, Jérôme Etévé <[EMAIL PROTECTED]> wrote:
> I though it'd be ok to trigger this the very first time the process
> method is called by doing something like that:
>
> private boolean firstTime= true ;
>
> public void process(ResponseBuilder rb) throws IOException {
>if ( firstTime ){
>firstTime = false ;
>buildMyStuff(rb) ;
>}
> }
>
>
> The problem is that my method buildMyStuff hangs when calling
> rb.req.getCore().getSearcher() ; ,
> and I believe this is happening when the warm up queries are executed.
getSearcher() can wait for a searcher to be registered.
getNewestSearcher() can be used from places like inform(), but if you
are already in process()
then the one you should use is the one bound to the request (the
SolrQueryRequest object) - rb.req.getSearcher()
-Yonik
customizing results in StandardQueryHandler
Hi, In my usecase, I query a set of fields. Then based on the results, I want to output a customized set of fields. Can I do this without using a search component? E:g. I query for fields f1, f2, f3, f4. Now based on some conditions, I want to output just f1, f3, f4 (the list of final fields may vary). How do I rewrite the resultant xml optimally? Any thoughts on this will be helpful Thanks, Kalyan
Re: Deadlock problem on searcher at warm up.
Great, it works now.
Thanks !
J
On Fri, Oct 24, 2008 at 4:45 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> On Fri, Oct 24, 2008 at 8:21 AM, Jérôme Etévé <[EMAIL PROTECTED]> wrote:
>> I though it'd be ok to trigger this the very first time the process
>> method is called by doing something like that:
>>
>> private boolean firstTime= true ;
>>
>> public void process(ResponseBuilder rb) throws IOException {
>>if ( firstTime ){
>>firstTime = false ;
>>buildMyStuff(rb) ;
>>}
>> }
>>
>>
>> The problem is that my method buildMyStuff hangs when calling
>> rb.req.getCore().getSearcher() ; ,
>> and I believe this is happening when the warm up queries are executed.
>
> getSearcher() can wait for a searcher to be registered.
> getNewestSearcher() can be used from places like inform(), but if you
> are already in process()
> then the one you should use is the one bound to the request (the
> SolrQueryRequest object) - rb.req.getSearcher()
>
> -Yonik
>
--
Jerome Eteve.
Chat with me live at http://www.eteve.net
[EMAIL PROTECTED]
Re: customizing results in StandardQueryHandler
isn't this just: fl=f1,f3,f4 etc or am I missing something? On Oct 24, 2008, at 12:26 PM, Manepalli, Kalyan wrote: Hi, In my usecase, I query a set of fields. Then based on the results, I want to output a customized set of fields. Can I do this without using a search component? E:g. I query for fields f1, f2, f3, f4. Now based on some conditions, I want to output just f1, f3, f4 (the list of final fields may vary). How do I rewrite the resultant xml optimally? Any thoughts on this will be helpful Thanks, Kalyan
RE: customizing results in StandardQueryHandler
Ryan, Actually, what I need is: I always query for a set of fields say (f1, f2, f3 .. f6). Now once I get the results, based on some logic, I need to generate the XML which is customized and contains only fields say (f2, f3, and some new data). So the fl will always be (f1 ... f6) Thanks, Kalyan Manepalli -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 1:25 PM To: solr-user@lucene.apache.org Subject: Re: customizing results in StandardQueryHandler isn't this just: fl=f1,f3,f4 etc or am I missing something? On Oct 24, 2008, at 12:26 PM, Manepalli, Kalyan wrote: > Hi, > In my usecase, I query a set of fields. Then based on the > results, I want to output a customized set of fields. Can I do this > without using a search component? > E:g. I query for fields f1, f2, f3, f4. Now based on some > conditions, I > want to output just f1, f3, f4 (the list of final fields may vary). > > How do I rewrite the resultant xml optimally? > Any thoughts on this will be helpful > > Thanks, > Kalyan
RE: customizing results in StandardQueryHandler
Ah! This will let you post-process result sets with an XSL script: http://wiki.apache.org/solr/XsltResponseWriter -Original Message- From: Manepalli, Kalyan [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 11:44 AM To: solr-user@lucene.apache.org Subject: RE: customizing results in StandardQueryHandler Ryan, Actually, what I need is: I always query for a set of fields say (f1, f2, f3 .. f6). Now once I get the results, based on some logic, I need to generate the XML which is customized and contains only fields say (f2, f3, and some new data). So the fl will always be (f1 ... f6) Thanks, Kalyan Manepalli -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 1:25 PM To: solr-user@lucene.apache.org Subject: Re: customizing results in StandardQueryHandler isn't this just: fl=f1,f3,f4 etc or am I missing something? On Oct 24, 2008, at 12:26 PM, Manepalli, Kalyan wrote: > Hi, > In my usecase, I query a set of fields. Then based on the results, I > want to output a customized set of fields. Can I do this without using > a search component? > E:g. I query for fields f1, f2, f3, f4. Now based on some conditions, > I want to output just f1, f3, f4 (the list of final fields may vary). > > How do I rewrite the resultant xml optimally? > Any thoughts on this will be helpful > > Thanks, > Kalyan
Re: customizing results in StandardQueryHandler
I'm still not following... Does the logic depend on the result? That is, are you asking for doc A to have f1,f2 and doc B to have f1,f4? If that is your question, then no -- as is, the ResponseWriter will write the same fields for every document. When you say "(f2, f3, and some new data)" -- where does the "new data" come from? Are you trying to augment the results of a document with something else? If so, take a look at the local solr patch in: https://issues.apache.org/jira/browse/LUCENE-1387 that adds a distance calculation to each matched document. On Oct 24, 2008, at 2:43 PM, Manepalli, Kalyan wrote: Ryan, Actually, what I need is: I always query for a set of fields say (f1, f2, f3 .. f6). Now once I get the results, based on some logic, I need to generate the XML which is customized and contains only fields say (f2, f3, and some new data). So the fl will always be (f1 ... f6) Thanks, Kalyan Manepalli -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 1:25 PM To: solr-user@lucene.apache.org Subject: Re: customizing results in StandardQueryHandler isn't this just: fl=f1,f3,f4 etc or am I missing something? On Oct 24, 2008, at 12:26 PM, Manepalli, Kalyan wrote: Hi, In my usecase, I query a set of fields. Then based on the results, I want to output a customized set of fields. Can I do this without using a search component? E:g. I query for fields f1, f2, f3, f4. Now based on some conditions, I want to output just f1, f3, f4 (the list of final fields may vary). How do I rewrite the resultant xml optimally? Any thoughts on this will be helpful Thanks, Kalyan
Re: timeouts
What is happening at the time of the freeze? Are you indexing? Searching? For either of these options, what are the operations that are happening? Is Tomcat and the Solr admin accessible while this happens? Are there any exceptions in the logs? How much memory are you giving it versus MySQL? I assume you're hitting both, right? Do you have enough memory to do that? Typically DBs like as much mem as they can get, as does Solr (for the most part). That being said, 100K records is next to nothing, so I doubt Solr is taking that much memory. Please describe a bit more what you are doing and seeing. On Oct 24, 2008, at 8:58 AM, Simon Collins wrote: Hi We’re running solr on a win 2k3 box under tomcat with about 100,000 records. When doing large updates of records via solr sharp, solr completely freezes and doesn’t come back until we restart tomcat. This has only started happening since putting mysql on the same box (as a source of the data to update from). Are there any known issues with running solr and mysql on the same box? When it’s frozen, the cpu usage is around 1-2% not exactly out of resources! Am i best using something else instead of tomcat? We’re still trialling solr (presently, used for our main site search www.shoe-shop.com and search and navigation for our microsites ). It’s an excellent search product, but I don’t want to fork out on new hardware for it just yet – until i know how more about the performance and which environment i’m best to go for (win/linux). If anyone has any suggestions/needs more info, i’d be extremely grateful. Thanks Simon Simon Collins Systems Analyst Telephone: 01904 606 867 Fax Number: 01904 528 791 shoe-shop.com ltd Catherine House Northminster Business Park Upper Poppleton, YORK YO26 6QU www.shoe-shop.com This message (and any associated files) is intended only for the use of the individual or entity to which it is addressed and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Messages sent to and from us may be monitored. Internet communications cannot be guaranteed to be secure or error- free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Therefore, we do not accept responsibility for any errors or omissions that are present in this message, or any attachment, that have arisen as a result of e-mail transmission. If verification is required, please request a hard-copy version. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. (PAVD001) Shoe-shop.com Limited is a company registered in England and Wales with company number 03817232. Vat Registration GB 734 256 241. Registered Office Catherine House, Northminster Business Park, Upper Poppleton, YORK, YO26 6QU. This message has been scanned for malware by SurfControl plc. www.surfcontrol.com -- Grant Ingersoll Lucene Boot Camp Training Nov. 3-4, 2008, ApacheCon US New Orleans. http://www.lucenebootcamp.com Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ
Entity extraction?
During a recent sales pitch to my company by FAST, they mentioned entity extraction. I'd never heard of it before, but they described it as basically recognizing people/places/things in documents being indexed and then being able to do faceting on this data at query time. Does anything like this already exist in SOLR? If not, I'm not opposed to developing it myself, but I could use some pointers on where to start. Thanks, - Charlie
RE: customizing results in StandardQueryHandler
Populating the field at the index time is a good option. Is there any other way to do it in search time Thanks, Kalyan Manepalli -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 2:14 PM To: solr-user@lucene.apache.org Subject: Re: customizing results in StandardQueryHandler I'm still not following... Does the logic depend on the result? That is, are you asking for doc A to have f1,f2 and doc B to have f1,f4? If that is your question, then no -- as is, the ResponseWriter will write the same fields for every document. When you say "(f2, f3, and some new data)" -- where does the "new data" come from? Are you trying to augment the results of a document with something else? If so, take a look at the local solr patch in: https://issues.apache.org/jira/browse/LUCENE-1387 that adds a distance calculation to each matched document. On Oct 24, 2008, at 2:43 PM, Manepalli, Kalyan wrote: > Ryan, > Actually, what I need is: I always query for a set of fields say > (f1, f2, f3 .. f6). Now once I get the results, based on some logic, I > need to generate the XML which is customized and contains only fields > say (f2, f3, and some new data). > So the fl will always be (f1 ... f6) > > > > Thanks, > Kalyan Manepalli > > -Original Message- > From: Ryan McKinley [mailto:[EMAIL PROTECTED] > Sent: Friday, October 24, 2008 1:25 PM > To: solr-user@lucene.apache.org > Subject: Re: customizing results in StandardQueryHandler > > isn't this just: fl=f1,f3,f4 etc > > or am I missing something? > > > On Oct 24, 2008, at 12:26 PM, Manepalli, Kalyan wrote: > >> Hi, >> In my usecase, I query a set of fields. Then based on the >> results, I want to output a customized set of fields. Can I do this >> without using a search component? >> E:g. I query for fields f1, f2, f3, f4. Now based on some >> conditions, I >> want to output just f1, f3, f4 (the list of final fields may vary). >> >> How do I rewrite the resultant xml optimally? >> Any thoughts on this will be helpful >> >> Thanks, >> Kalyan >
Solr 1.3 stack overflow when accessing solr/admin page
Hi, I run Solr 1.3 in Weblogic 10.3 Java 6; I have a single core application deployed to the same server as Solr 1.3. I am not using solrj. I do not have any problems with search and update - they work fine. However I am getting stack overflow when I try to access solr/admin page. java.lang.StackOverflowError javax.servlet.ServletException: java.lang.StackOverflowError at weblogic.servlet.internal.RequestDispatcherImpl.forward(RequestDispatche rImpl.java:266) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j ava:273) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java: 42) at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDis patcherImpl.java:503) at ... I am pretty sure it is my configuration, and I would appreciate any pointers at what I should be checking. Thank you very much, -Olga
Re: Solr 1.3 stack overflow when accessing solr/admin page
are you running the packaged .war directly? or something custom? Did it ever work? Is anyone else running successfully on weblogic? On Oct 24, 2008, at 5:10 PM, Dadasheva, Olga wrote: Hi, I run Solr 1.3 in Weblogic 10.3 Java 6; I have a single core application deployed to the same server as Solr 1.3. I am not using solrj. I do not have any problems with search and update - they work fine. However I am getting stack overflow when I try to access solr/admin page. java.lang.StackOverflowError javax.servlet.ServletException: java.lang.StackOverflowError at weblogic .servlet.internal.RequestDispatcherImpl.forward(RequestDispatche rImpl.java:266) at org .apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j ava:273) at weblogic .servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java: 42) at weblogic .servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDis patcherImpl.java:503) at ... I am pretty sure it is my configuration, and I would appreciate any pointers at what I should be checking. Thank you very much, -Olga
Re: Entity extraction?
Solr can do a simple facet seach like FAST, but the entity extraction demands other tecnologies. I do not know how FAST does it but at the company I´m working on (www.cortex-intelligence.com), we use a mix of statistical and language-specific tasks to recognize and categorize entities in the text. Ling Pipe is another tool (free) that does that too. In case you would like to see a simple demo: http://www.cortex-intelligence.com/tech/ Rossini On Fri, Oct 24, 2008 at 6:18 PM, Charlie Jackson <[EMAIL PROTECTED] > wrote: > During a recent sales pitch to my company by FAST, they mentioned entity > extraction. I'd never heard of it before, but they described it as > basically recognizing people/places/things in documents being indexed > and then being able to do faceting on this data at query time. Does > anything like this already exist in SOLR? If not, I'm not opposed to > developing it myself, but I could use some pointers on where to start. > > > > Thanks, > > - Charlie > >
Re: Entity extraction?
You can find more about this topic in this book availabe at amazon: http://www.amazon.com/Building-Search-Applications-Lucene-Lingpipe/dp/0615204252/ 2008/10/24 Rafael Rossini <[EMAIL PROTECTED]> > Solr can do a simple facet seach like FAST, but the entity extraction > demands other tecnologies. I do not know how FAST does it but at the > company > I´m working on (www.cortex-intelligence.com), we use a mix of statistical > and language-specific tasks to recognize and categorize entities in the > text. Ling Pipe is another tool (free) that does that too. In case you > would > like to see a simple demo: http://www.cortex-intelligence.com/tech/ > > Rossini > > > On Fri, Oct 24, 2008 at 6:18 PM, Charlie Jackson < > [EMAIL PROTECTED] > > wrote: > > > During a recent sales pitch to my company by FAST, they mentioned entity > > extraction. I'd never heard of it before, but they described it as > > basically recognizing people/places/things in documents being indexed > > and then being able to do faceting on this data at query time. Does > > anything like this already exist in SOLR? If not, I'm not opposed to > > developing it myself, but I could use some pointers on where to start. > > > > > > > > Thanks, > > > > - Charlie > > > > > -- Regards, Rogério (_rogerio_) [Blog: http://faces.eti.br] [Sandbox: http://bmobile.dyndns.org] [Twitter: http://twitter.com/ararog] "Faça a diferença! Ajude o seu país a crescer, não retenha conhecimento, distribua e aprenda mais." (http://faces.eti.br/2006/10/30/conhecimento-e-amadurecimento)
Re: Entity extraction?
This is not something solr does currently... It sounds like something that should be added to Mahout: http://lucene.apache.org/mahout/ On Oct 24, 2008, at 4:18 PM, Charlie Jackson wrote: During a recent sales pitch to my company by FAST, they mentioned entity extraction. I'd never heard of it before, but they described it as basically recognizing people/places/things in documents being indexed and then being able to do faceting on this data at query time. Does anything like this already exist in SOLR? If not, I'm not opposed to developing it myself, but I could use some pointers on where to start. Thanks, - Charlie
RE: Entity extraction?
Thanks for the replies, guys, that gives me a good place to start looking. - Charlie -Original Message- From: Rogerio Pereira [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 5:14 PM To: solr-user@lucene.apache.org Subject: Re: Entity extraction? You can find more about this topic in this book availabe at amazon: http://www.amazon.com/Building-Search-Applications-Lucene-Lingpipe/dp/0615204252/ 2008/10/24 Rafael Rossini <[EMAIL PROTECTED]> > Solr can do a simple facet seach like FAST, but the entity extraction > demands other tecnologies. I do not know how FAST does it but at the > company > I´m working on (www.cortex-intelligence.com), we use a mix of statistical > and language-specific tasks to recognize and categorize entities in the > text. Ling Pipe is another tool (free) that does that too. In case you > would > like to see a simple demo: http://www.cortex-intelligence.com/tech/ > > Rossini > > > On Fri, Oct 24, 2008 at 6:18 PM, Charlie Jackson < > [EMAIL PROTECTED] > > wrote: > > > During a recent sales pitch to my company by FAST, they mentioned entity > > extraction. I'd never heard of it before, but they described it as > > basically recognizing people/places/things in documents being indexed > > and then being able to do faceting on this data at query time. Does > > anything like this already exist in SOLR? If not, I'm not opposed to > > developing it myself, but I could use some pointers on where to start. > > > > > > > > Thanks, > > > > - Charlie > > > > > -- Regards, Rogério (_rogerio_) [Blog: http://faces.eti.br] [Sandbox: http://bmobile.dyndns.org] [Twitter: http://twitter.com/ararog] "Faça a diferença! Ajude o seu país a crescer, não retenha conhecimento, distribua e aprenda mais." (http://faces.eti.br/2006/10/30/conhecimento-e-amadurecimento)
Re: Entity extraction?
I agree Ryan and I would like see a completly integration between solr, nutch, tika and mahout in the future. 2008/10/24 Ryan McKinley <[EMAIL PROTECTED]> > This is not something solr does currently... > > It sounds like something that should be added to Mahout: > http://lucene.apache.org/mahout/ > > > > On Oct 24, 2008, at 4:18 PM, Charlie Jackson wrote: > > During a recent sales pitch to my company by FAST, they mentioned entity >> extraction. I'd never heard of it before, but they described it as >> basically recognizing people/places/things in documents being indexed >> and then being able to do faceting on this data at query time. Does >> anything like this already exist in SOLR? If not, I'm not opposed to >> developing it myself, but I could use some pointers on where to start. >> >> >> >> Thanks, >> >> - Charlie >> >> > -- Regards, Rogério (_rogerio_) [Blog: http://faces.eti.br] [Sandbox: http://bmobile.dyndns.org] [Twitter: http://twitter.com/ararog] "Faça a diferença! Ajude o seu país a crescer, não retenha conhecimento, distribua e aprenda mais." (http://faces.eti.br/2006/10/30/conhecimento-e-amadurecimento)
Lucene project & subprojects news RSS feed?
On the main lucene web page: http://lucene.apache.org/index.html There is a list of news items spanning all the lucene subprojects. Does anyone know if there is an RSS feed or "announce" mailing list that has this information? ~ David Smiley -- View this message in context: http://www.nabble.com/Lucene-project---subprojects-news-RSS-feed--tp20158991p20158991.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Solr 1.3 stack overflow when accessing solr/admin page
I am running the packaged war. It used to work 1 year ago when I briefly looked into using Solr instead of Lucene - at that time I just built the trunk and it worked from the first plug (Solr 1.3 dev around Sep 9th 2007). -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Friday, October 24, 2008 5:28 PM To: solr-user@lucene.apache.org Subject: Re: Solr 1.3 stack overflow when accessing solr/admin page are you running the packaged .war directly? or something custom? Did it ever work? Is anyone else running successfully on weblogic? On Oct 24, 2008, at 5:10 PM, Dadasheva, Olga wrote: > Hi, > > I run Solr 1.3 in Weblogic 10.3 Java 6; I have a single core > application deployed to the same server as Solr 1.3. > I am not using solrj. > > I do not have any problems with search and update - they work fine. > > However I am getting stack overflow when I try to access solr/admin > page. > > > java.lang.StackOverflowError javax.servlet.ServletException: > java.lang.StackOverflowError at > weblogic > .servlet.internal.RequestDispatcherImpl.forward(RequestDispatche > rImpl.java:266) at > org > .apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j > ava:273) at > weblogic > .servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java: > 42) at > weblogic > .servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDis > patcherImpl.java:503) at > ... > > I am pretty sure it is my configuration, and I would appreciate any > pointers at what I should be checking. > > > Thank you very much, > > -Olga
Index partitioning
Hi, I want to partition my index based on category information. Also, while indexing I want to store particular category data to corresponding index partition. In the same way I need to search for category information on corresponding partition.. I found some information on wiki link http://wiki.apache.org/solr/IndexPartitioning. But it couldn't help much in implementation. Could some one give example with how I can configure, index data into particular partitions and search on particular partition. Would appreciate for any help!! Thanks & Regards, Mahendra
