I just upgraded from Solr 3.4 to Solr 3.6; I'm using the same
data-import.xml for both versions. The import functioned properly with
3.4.

I'm using a nested entity to fetch authors associated with each
document, and I'm using CachedSqlEntityProcessor to avoid hitting the
DB an unreasonable number of times. However, when indexing, Solr
indexes very slowly and appears to be fetching all authors in the DB
for each document. The index should be ~500 megs; I aborted the
indexing when it reached ~6gigs. If I comment out the nested author
entity below, Solr will index normally.

Am I missing something obvious or is this a bug?

<document name="documents">
    <entity name="document" dataSource="production"
     transformer="HTMLStripTransformer,TemplateTransformer,RegexTransformer"
     query="select id, ..., from document">
        <field column="id" name="id"/>
        <field column="uid" name="uid" template="DOC${document.id}"/>
        <!-- more fields .. -->
        <entity name="author" dataSource="production"
         query="select
                cast(da.document_id as text) as document_id,
                a.id, a.name, a.signature from document_author da
                left outer join author a on a.id = da.author_id"
         cacheKey="document_id"
         cacheLookup="document.id"
         processor="CachedSqlEntityProcessor">
             <field name="author_id" column="id" />
             <field name="author" column="name" />
             <field name="author_signature" column="signature" />
        </entity>
    </entity>
</document>

Also posted at SO if you prefer to answer there:
http://stackoverflow.com/questions/10482484/nested-cachedsqlentityprocessor-running-for-each-entity-row-with-solr-3-6

Kellen

Reply via email to