How to clear the Collection Creation failed errors?

[srinalluri]

I tried to create two collections but failed due to some known reasons. I
want to clear the errors which you see the attached screenshot. Please see
the screenshot. As the collections failed to create, there is no point to
delete the collection.


 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-clear-the-Collection-Creation-failed-errors-tp4310446.html
Sent from the Solr - User mailing list archive at Nabble.com.

full-import failed after 5 hours with Exception: ORA-01555: snapshot too old: rollback segment number with name "" too small ORA-22924: snapshot too old

[srinalluri]

Hello,

I am using Solr 4.3.2 and Oracle DB. The sub entity is using
CachedSqlEntityProcessor. The dataSource is having batchSize="500". The
full-import is failed with 'ORA-01555: snapshot too old: rollback segment
number  with name "" too small ORA-22924: snapshot too old' Exception after
5 hours.

We already increased the undo space 4 times at the database end. Number of
records in the jan_story table is 800,000 only. Tomcat is with 4GB JVM
memory.

Following is the entity (there are other sub-entities, I didn't mention them
here. As the import failed with article_details entity. article_details is
the first sub-entity) 














  
  


The full-import without CachedSqlEntityProcessor is taking 7 days. That is
why I am doing all this.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/full-import-failed-after-5-hours-with-Exception-ORA-01555-snapshot-too-old-rollback-segment-number-wd-tp4073822.html
Sent from the Solr - User mailing list archive at Nabble.com.

full-import takes 4 days(48 hours) to complete where main db table size 700k only

[srinalluri]

Hi,

Environment is Solr 3.6.1. The database is having enough indexes. The box is
having enough memory. The DB is performance is good. Auto commit is enabled
for every 1 minute. 
Please see the following entity. The full-import of this entity is taking
over 48 hours to complete on production environment. The number records in
the main table is around 700,000 only.  I tried materialized view, but that
view is having duplicate records. So I can't go with materialized view for
all these queries. 
Can someone please suggest how to improve the performance for full-import?







\[(pullquote|summary)\]\|\[video
[0-9]+?\]|\[youtube .+?\]', '') as BODY, PUBLISHED_DATE, MODIFIED_DATE,
DATELINE, REPORTER_NAME, TICKER_CODES,ADVERTORIAL_CONTENT from tab_story2
where id = '${oracle-article.VCMID}'">


















http://qa.global.fqstatic.com' || sourcepath) as
IMAGE_URL from ( select mc.sourcepath from tab_rel_content rc, tab_story2
st, gnasmomap mm, dsx_media_common mc where rc.parent_id =
'${oracle-article.VCMID}' and rc.parent_id = st.id and (st.NO_FEATURED_MEDIA
!= 'yes' OR st.NO_FEATURED_MEDIA is null) and rc.ref_id = mm.recordid and
mm.keystring1 = mc.mediaid and rc.rank = 1 union all select mc.sourcepath
from tab_rel_content arm, tab_story2 st, gnasmomap cmm, tab_rel_media crm,
gnasmomap mmm, dsx_media_common mc where arm.parent_id =
'${oracle-article.VCMID}' and arm.parent_id = st.id and
(st.NO_FEATURED_MEDIA !='yes' OR st.NO_FEATURED_MEDIA is null) and
arm.ref_id = cmm.recordid and cmm.keystring1 = crm.parent_id and crm.rank =
1 and crm.ref_id = mmm.recordid and mmm.keystring1 = mc.mediaid and arm.rank
= 1)">



















--
View this message in context: 
http://lucene.472066.n3.nabble.com/full-import-takes-4-days-48-hours-to-complete-where-main-db-table-size-700k-only-tp4058690.html
Sent from the Solr - User mailing list archive at Nabble.com.

java.lang.NullPointerException. I am trying to use CachedSqlEntityProcessor

[srinalluri]

I am in Solr 3.6.1. 
The following entity gives java.lang.NullPointerException. How to debug
this? Here I am usingCachedSqlEntityProcessor. 


  



  
  








  
  

Here is the exception message:


SEVERE: Exception while processing: vig8-article-mon document :
SolrInputDocument[{repository=repository(1.0)={vig8qamon},
native_id=native_id(1.0)={8f2210474fea2310VgnVCM10d1c1a8c0RCRD},
content_type=content_type(1.0)={article}}]:org.apache.solr.handler.dataimport.DataImportHandlerException:
java.lang.NullPointerException
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:64)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:333)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:296)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:683)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:709)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:619)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:327)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:225)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:375)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:445)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:426)
Caused by: java.lang.NullPointerException
at java.util.TreeMap.getEntry(TreeMap.java:342)
at java.util.TreeMap.get(TreeMap.java:273)
at
org.apache.solr.handler.dataimport.SortedMapBackedCache.add(SortedMapBackedCache.java:57)
at
org.apache.solr.handler.dataimport.DIHCacheSupport.populateCache(DIHCacheSupport.java:124)
at
org.apache.solr.handler.dataimport.DIHCacheSupport.getIdCacheData(DIHCacheSupport.java:176)
at
org.apache.solr.handler.dataimport.DIHCacheSupport.getCacheData(DIHCacheSupport.java:145)
at
org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:132)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.pullRow(EntityProcessorWrapper.java:330)




--
View this message in context: 
http://lucene.472066.n3.nabble.com/java-lang-NullPointerException-I-am-trying-to-use-CachedSqlEntityProcessor-tp4059815.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: java.lang.NullPointerException. I am trying to use CachedSqlEntityProcessor

[srinalluri]

Thanks James for your reply.

I have updated to 3.6.2. Now the NullPointerException is gone. But the
entities with CachedSqlEntityProcessor don't add anything to solr.

And entities without CachedSqlEntityProcessor, are working fine.

Why entities with CachedSqlEntityProcessor don't do anything? What is wrong
in my entity?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/java-lang-NullPointerException-I-am-trying-to-use-CachedSqlEntityProcessor-tp4059815p4060043.html
Sent from the Solr - User mailing list archive at Nabble.com.

HTTP Status 503 - Server is shutting down

[srinalluri]

Hi,

I am unable to setup solr4. I am getting this error: HTTP Status 503 -
Server is shutting down. I don't see anything in tomcat logs.

conf/Catalina/localhost\solr4new.xml:



  


/apps/solr1/solr4new/:
|-- bin
|-- collection1
|   |-- conf
|   |   |-- lang
|   |   |-- velocity
|   |   `-- xslt
|   `-- data
`-- logs



--
View this message in context: 
http://lucene.472066.n3.nabble.com/HTTP-Status-503-Server-is-shutting-down-tp4065958.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: HTTP Status 503 - Server is shutting down

[srinalluri]

I am using Solr 4.3.0. I have already copied solr4j*.jar files and log4j far
file to tomcat lib, and restarted the tomcat.

Before copy these jar files, I got 404 error. Now I am getting this 503
error for host:18080/solr4new/

I don't have zookeeper. Is zookeeper must in order to work on solr 4.3.0?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/HTTP-Status-503-Server-is-shutting-down-tp4065958p4065970.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: HTTP Status 503 - Server is shutting down

[srinalluri]

the following files are at /apps/tomcat/solr4new/lib:
./annotations-api.jar
./tomcat-jdbc.jar
./slf4j-log4j12-1.6.6.jar
./tomcat-dbcp.jar
./catalina-tribes.jar
./tomcat-i18n-fr.jar
./catalina.jar
./jasper.jar
./el-api.jar
./tomcat-api.jar
./jsp-api.jar
./slf4j-api-1.6.6.jar
./tomcat-i18n-es.jar
./tomcat-i18n-ja.jar
./catalina-ha.jar
./tomcat-coyote.jar
./servlet-api.jar
./log4j-1.2.16.jar
./catalina-ant.jar
./tomcat-util.jar
./ecj-4.2.1.jar
./jasper-el.jar

logging.properties is at conf folder, not in lib. The following path I added
in that file. But still solr.2013-05-24.log is blank.
5localhost.org.apache.juli.FileHandler.directory = /apps/solr1/solr4new/logs 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/HTTP-Status-503-Server-is-shutting-down-tp4065958p4065992.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: HTTP Status 503 - Server is shutting down

[srinalluri]

Thanks Shawn for this help. After coping the log4j.properties file from
example/resources/ to tomcat lib and changing the log file path, the logs
are getting generated.

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/HTTP-Status-503-Server-is-shutting-down-tp4065958p4068187.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: java.lang.NullPointerException. I am trying to use CachedSqlEntityProcessor

[srinalluri]

I have solved this problem and able work with CachedSqlEntityProcessor
successfully after a very long struggle. 

I tried this on 4.2.

There are still existing bugs it seems:
1. What ever you mention in cacheKey, that field name must in the select
statement explicitly.
2. If I am correct, the field name in cacheKey and in the select statement
are case sensitive.
3. We have ID field in our table, I tried to give cacheKey="ID". But that
got conflicted with the uniqueKey as uniqueKey is also "ID". So I wrote it
as "SELECT ID AS AID,." and cacheKey="AID"

thanks
Srini  




--
View this message in context: 
http://lucene.472066.n3.nabble.com/java-lang-NullPointerException-I-am-trying-to-use-CachedSqlEntityProcessor-tp4059815p4070059.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 3.6.1: query performance is slow when asterisk is in the query

[srinalluri]

Our environment is Solr 3.6.1. I have the following fieldType. There is a
field called 'body' of this fieldType. When I make a query: q=body:*, it is
talking longer than the expected. What are the changes I need to do to this
fieldType for better query performance? Some other fieldTypes in our schema
are performing better.


  





  
  




  


Thanks in advance
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-6-1-query-performance-is-slow-when-asterisk-is-in-the-query-tp4002496.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 3.6.1: query performance is slow when asterisk is in the query

[srinalluri]

Thanks Jack for your reply.

I don't have much documents which have a null field value.

I added ReversedWildcardFilterFactory to test the performance improvement
only, but that didn't help.

What else changes I can do to the fieldType?

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-6-1-query-performance-is-slow-when-asterisk-is-in-the-query-tp4002496p4002522.html
Sent from the Solr - User mailing list archive at Nabble.com.

search is slow for URL fields of type String.

[srinalluri]

This is string fieldType:



These are the filelds using 'string' fieldType:

  
  

And this the sample query:
/select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/
AND image_url:*

Each query like this taking around 400 milli seconds. What are the change I
can do to the fieldType to improve query performance?

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html
Sent from the Solr - User mailing list archive at Nabble.com.

What are the available parameters in field tag in schema.xml, and data-config.xml ?

[srinalluri]

I want to know XSD of schema.xml and data-config.xml.

Basically I want to know available parameters in field tag in schema.xml,
and data-config.xml.

For example, in schema.xml, field tag is having a parameter called
'default', what else are available?
For example, in data-config.xml, field tag is having a parameter called
'splitBy', what else are available?

thanks
Srini 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/What-are-the-available-parameters-in-field-tag-in-schema-xml-and-data-config-xml-tp4003142.html
Sent from the Solr - User mailing list archive at Nabble.com.

How to boost date field while boosting a text field?

[srinalluri]

The following is the working query with more weight to title. I am using
default parser. But the published_date of the results of this query is not
in order. I want date is in order.

?q=content_type:video AND (title:("obama budget")^2 OR
export_headline:("obama budget"))&fl=title,score,export_headline

I want to give weight to date field also. I can't use 'sort' as I used the
'^2'. What are the changes needed to the above query 

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-boost-date-field-while-boosting-a-text-field-tp4009455.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to boost date field while boosting a text field?

[srinalluri]

Thanks Hoss for your reply.


I already seen that Solr wiki, but I am unable to merge that solution with
my query.

The example in the wiki gave to search for one string. I tried this, but the
date is not sorted order, why?
?q={!boost b=recip(ms(NOW,date),3.16e-11,1,1)}ipad&fl=date


How to merge 'title:("ipad")^2'  to the above query?

thanks
Srini




--
View this message in context: 
http://lucene.472066.n3.nabble.com/How-to-boost-date-field-while-boosting-a-text-field-tp4009455p4010189.html
Sent from the Solr - User mailing list archive at Nabble.com.

Dismax parser. Where can I add field:value to the query?

[srinalluri]

If I use dismax parser in the query, I knew I can't give field:value to the
q.

The following query works fine.
/select/?defType=dismax&pf=title^2&q="obama%20budget"&qf=title

But I want these results when content_type:video only. Where can I add
content_type:video in the query, so that I can give video type results only?

thanks
Srini





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Dismax-parser-Where-can-I-add-field-value-to-the-query-tp4011643.html
Sent from the Solr - User mailing list archive at Nabble.com.

404 error for http://host:port/solr1-newmaster/dataimport

[srinalluri]

Hi I am new to Solr 4.0, I am familiar with solr 3.6.

I have set solr 4.0. 

I am getting 'There are no SolrCores running. ' message for
http://host:port/solr1-newmaster/ URL. What does it mean, what should I do?

I am getting 404 error for this  : /solr1-newmaster/dataimport
Does it mean it is not recognizing conf files? My conf files are at 
http://host:port/newmaster/collection1/conf/

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/404-error-for-http-host-port-solr1-newmaster-dataimport-tp4014330.html
Sent from the Solr - User mailing list archive at Nabble.com.

Sorl 4.0: ClassNotFoundException DataImportHandler

[srinalluri]

I have the following line in solrconfig.xml.



And I have all the dist jar files in dist folder. I restarted the tomcat,
why I am still getting this error:

java.lang.ClassNotFoundException:
org.apache.solr.handler.dataimport.DataImportHandler

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Sorl-4-0-ClassNotFoundException-DataImportHandler-tp4014348.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Sorl 4.0: ClassNotFoundException DataImportHandler

[srinalluri]

Thanks for your reply. Actually ClassNotFoundException is in tomcat logs. In
3.6, I don't put these jars in tomcat lib. In 4.0, should I put these jars
in tomcat lib?

thanks
srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Sorl-4-0-ClassNotFoundException-DataImportHandler-tp4014348p4014494.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Sorl 4.0: ClassNotFoundException DataImportHandler

[srinalluri]

Thanks Chris for your reply. I really need some help here.

1) If I put the apache-solr-dataimporthandler-*.jar files in solr/lib
folder, the jar files are loading. I see that in the tomcat logs. But in the
end it says 'ClassNotFoundException DataImportHandler'.

2) So If I remove apache-solr-dataimporthandler-*.jar from solr/lib folder
and placed them in tomcat/lib folder. No more ClassNotFoundException. But
this time it says 'Error Instantiating Request Handler,
org.apache.solr.handler.dataimport.DataImportHandler failed to instantiate
org.apache.solr.request.SolrRequestHandler'.








--
View this message in context: 
http://lucene.472066.n3.nabble.com/Sorl-4-0-ClassNotFoundException-DataImportHandler-tp4014348p4014770.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 4:How to call a updateRequestProcessorChain during the /dataimport?

[srinalluri]

I have a new updateRequestProcessorChain called 'bodychain'.  (Please note
CountFieldValuesUpdateProcessorFactory is new in Solr 4). I want to call
this bodychain during the dataimport.


   
 body
 body_count
   
   
 body_count
   
   
 body_count
 0
   
 

Following is the my dataimport handler, which is already having
'update.chain'.  I think I can't give more than one update.chain in this
handler. When can I add 'bodychain'?



  dedupe  
  data-config.xml  

  

thanks
Srini



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-How-to-call-a-updateRequestProcessorChain-during-the-dataimport-tp4020812.html
Sent from the Solr - User mailing list archive at Nabble.com.