Using SqlEntityProcessor with cacheImpl="SortedMapBackedCache" is the same as specifying "CachedSqlEntityProcessor". Because the pluggable caches are only partially committed, I never added details to the wiki, so it still refers to CachedSEP. But its the same thing.
What is new here, though, is that you don't have to use "SortedMapBackedCache" (this is an in-memory cache and can only scale to what fits in heap.) You can use an alternate cache (but none are included in the Solr distribution). Also, you can cache data this doesn't come from SQL. So its more flexible this way rather than the older CachedSEP. Here's the wiki link with an example: http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor James Dyer Ingram Content Group (615) 213-4311 -----Original Message----- From: O. Olson [mailto:olson_...@yahoo.it] Sent: Thursday, May 16, 2013 5:06 PM To: solr-user@lucene.apache.org Subject: RE: Speed up import of Hierarchical Data Thank you James. Are there any examples of SortedMapBackedCache? I am new to Solr and I do not find many tutorials in this regard. I just modified the examples and they worked for me. What is a good way to learn these basics? O. O. Dyer, James-2 wrote > See https://issues.apache.org/jira/browse/SOLR-2943 . You can set up 2 > DIH handlers. The first would query the "CAT_TABLE" and save it to a > disk-backed cache, using DIHCacheWriter. You then would replace your 3 > child entities in the 2nd DIH handler to use DIHCacheProcessor to read > back the cached data. This is a little complicated to do, but it would > let you just cache the data once and because it is disk-backed, will scale > to whatever size the CAT_TABLE is. (For some details, see this thread: > http://lucene.472066.n3.nabble.com/DIH-nested-entities-don-t-work-tt4015514.html) > > A simpler method is simply to specify "cacheImpl=SortedMapBackedCache" on > the 3 child entities. (This is the same as using > CachedSqlEntityProcessor.) It would generate 3 in-memory caches, each > with the same data. If CAT_TABLE is small, this would be adequate. > > In between this would be to create a disk-backed cache Impl (or use the > ones at SOLR-2613 or SOLR-2948) and specify it on "cacheImpl". It would > still create 3 identical caches, but they would be disk-backed and could > scale beyond what in-memory can handle. > > James Dyer > Ingram Content Group > (615) 213-4311 -- View this message in context: http://lucene.472066.n3.nabble.com/Speed-up-import-of-Hierarchical-Data-tp4063924p4064040.html Sent from the Solr - User mailing list archive at Nabble.com.
