: Tuesday, October 25, 2016 12:31 PM
To: solr-user
Subject: Re: CachedSqlEntityProcessor with delta-import
Why not use delete by id rather than query? It'll be more efficient
Probably not a big deal though.
On Tue, Oct 25, 2016 at 1:47 AM, Aniket Khare wrote:
> Hi Sowmya,
>
> I
7;${dataimporter.last_index_time}'"
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Dyer, James [mailto:james.d...@ingramcontent.com]
>> Sent: Friday, October 21, 2016 4:23 PM
>> To: solr-user@lucene.apache.org
>>
> '${dataimporter.last_index_time}'"
>
>
>
>
>
>
> -Original Message-
> From: Dyer, James [mailto:james.d...@ingramcontent.com]
> Sent: Friday, October 21, 2016 4:23 PM
> To: solr-user@lucene.apache.org
> Subject: RE: CachedSqlEntityProcessor wi
ser@lucene.apache.org
Subject: RE: CachedSqlEntityProcessor with delta-import
Sowmya,
My memory is that the cache feature does not work with Delta Imports. In fact,
I believe that nearly all DIH features except straight JDBC imports do not work
with Delta Imports. My advice is to not use the Delta Import f
@lucene.apache.org
Subject: CachedSqlEntityProcessor with delta-import
Good morning,
Can CachedSqlEntityProcessor be used with delta-import? In my setup when
running a delta-import with CachedSqlEntityProcessor, the child entity values
are not correctly updated for the parent record. I am on Solr 4.3
Good morning,
Can CachedSqlEntityProcessor be used with delta-import? In my setup when
running a delta-import with CachedSqlEntityProcessor, the child entity values
are not correctly updated for the parent record. I am on Solr 4.3. Has anyone
experienced this and if so how to resolve it
Hi Gurus,
I am trying to use the Solr DIH CachedSqlEntityProcessor. In my case, I
also need to reference another column from the parent entity in the child
entity. How can that be done?
Observe the product_name column from the parent entity that I am trying to
use in the child entity
The conversation helps me understand Cached processor a lot. I'm working on
DIH cache using MapDB as backed engine instead of default
CachedSqlEntityProcessor
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-do-I-use-CachedSqlEntityProcessor-tp4064919p4198037.html
hey.
are sending the cacheImpl in your request? or where are defining it?
cacheImpl="${cache.impl}"
if i let this string blank, import fails =(
--
View this message in context:
http://lucene.472066.n3.nabble.com/Using-CachedSqlEntityProcessor-with-delta-imports-in-DIH-tp4091620p41
On 17 March 2014 18:13, manju16832003 wrote:
>
> I tried to use *CachedSqlEntityProcessor* in DataImportHandler with
> Sub-entity query. It does not seems to be working.
> Here is my query
>
>processor="CachedSqlEntityProcessor">
>
>
>
> If I
I tried to use *CachedSqlEntityProcessor* in DataImportHandler with
Sub-entity query. It does not seems to be working.
Here is my query
If I remove *processor="CachedSqlEntityProcessor"*, run without
CachedSqlEntityProcessor it works fine. With CachedSqlEntityProcessor my
sub-e
gain,
> O. O.
>
>
>
>
> --
> View this message in context:
http://lucene.472066.n3.nabble.com/Replacing-Deprecated-CachedSqlEntityProcessor-with-SqlEntityProcessor-with-a-cacheImpl-parameter-tp4106018p4106227.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
,
O. O.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Replacing-Deprecated-CachedSqlEntityProcessor-with-SqlEntityProcessor-with-a-cacheImpl-parameter-tp4106018p4106227.html
Sent from the Solr - User mailing list archive at Nabble.com.
ntity Processor (such as
CachedSqlEntityProcessor)*
Also here is an explanation:
http://comments.gmane.org/gmane.comp.jakarta.lucene.solr.user/67782
*"There were 2 major changes to DIH Cache functionality in Solr 3.6, only 1
of which was carried to Solr 4.0:*
*- Solr 3.6 had 2 MAJOR changes
Hi,
I am
looking to replace the Deprecated CachedSqlEntityProcessor with
SqlEntityProcessor
with a cacheImpl parameter but I cannot find documentation.
The Deprecated note at the top of
http://lucene.apache.org/solr/3_6_0/org/apache/solr/handler/dataimport
roche...@gmail.com [mailto:david.r.laroche...@gmail.com] On
Behalf Of David Larochelle
Sent: Monday, September 23, 2013 5:22 PM
To: solr-user
Subject: Using CachedSqlEntityProcessor with delta imports in DIH
I'm trying to use the CachedSqlEntityProcessor on a child entity that also
has a d
I'm trying to use the CachedSqlEntityProcessor on a child entity that also
has a delta query.
Full imports and delta imports of the parent entity work fine however delta
imports for the child entity have no effect. If I remove the
processor="CachedSqlEntityProcessor" attribute
ul 30, 2013 at 11:04 AM, Luis Lebolo wrote:
> Hi All,
>
> I'm trying to use CachedSqlEntityProcessor in one of my sub-entities, but
> the field never gets populated. I'm using Solr 4.4. The field is a
> multi-valued field:
>
> The relevant p
Hi All,
I'm trying to use CachedSqlEntityProcessor in one of my sub-entities, but
the field never gets populated. I'm using Solr 4.4. The field is a
multi-valued field:
The relevant part of my data-config.xml looks like:
...
Let me k
@lucene.apache.org
Betreff: RE: DataImportHandler: Problems with delta-import and
CachedSqlEntityProcessor
Instead of specifying CachedSqlEntityProcessor, you can specify
SqlEntityProcessor with "cacheImpl='SortedMapBackedCache'". If you
parametertize this, to have "SortedMapBack
Instead of specifying CachedSqlEntityProcessor, you can specify
SqlEntityProcessor with "cacheImpl='SortedMapBackedCache'". If you
parametertize this, to have "SortedMapBackedCache" for full updates but blank
for deltas I think it will cache only on the full
]
> Gesendet: Donnerstag, 20. Juni 2013 16:42
> An: solr-user@lucene.apache.org
> Betreff: AW: DataImportHandler: Problems with delta-import and
> CachedSqlEntityProcessor
>
> Hi,
>
> and thanks for the answer. But I'm a little bit confused about what you
> are sugge
-Ursprüngliche Nachricht-
Von: Constantin Wolber [mailto:constantin.wol...@medicalcolumbus.de]
Gesendet: Donnerstag, 20. Juni 2013 16:42
An: solr-user@lucene.apache.org
Betreff: AW: DataImportHandler: Problems with delta-import and
CachedSqlEntityProcessor
Hi,
and thanks for the
ोब्ळ् [mailto:noble.p...@gmail.com]
Gesendet: Donnerstag, 20. Juni 2013 15:42
An: solr-user@lucene.apache.org
Betreff: Re: DataImportHandler: Problems with delta-import and
CachedSqlEntityProcessor
it is possible to create two separate root entities . one for full-import and
another for delta. fo
M article WHERE
> myownid=${dih.delta.myownid}"
> pk="myownid">
>
>
> name="supplier"
> dataSource="ds2"
> query="SELECT * FROM
When switching everything to the CachedSqlEntityProcessor the full import
processed at the speed of 4000 Documents/s
So full import is running quite fine. Now I wanted to use the delta import.
When running the delta import I was expecting the ramp up time to be about the
same as in full import
I have solved this problem and able work with CachedSqlEntityProcessor
successfully after a very long struggle.
I tried this on 4.2.
There are still existing bugs it seems:
1. What ever you mention in cacheKey, that field name must in the select
statement explicitly.
2. If I am correct, the
:
http://lucene.472066.n3.nabble.com/How-do-I-use-CachedSqlEntityProcessor-tp4064919p4065392.html
Sent from the Solr - User mailing list archive at Nabble.com.
, May 22, 2013 10:30 AM
To: solr-user@lucene.apache.org
Subject: Re: How do I use CachedSqlEntityProcessor?
> I am curious why I did not get any errors before.
Because there was no (syntax) error before - the fact that you didn't include a
SKU (but using it as cacheKey) just doesn't m
before. For others, the following worked for me:
>
> query="SELECT CategoryName, SKU from CAT_TABLE WHERE
> CategoryLevel=1" cacheKey="SKU" cacheLookup="Product.SKU"
> processor="CachedSqlEntityProcessor">
>
>
>
> Similarly
ntent Group
> (615) 213-4311
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-do-I-use-CachedSqlEntityProcessor-tp4064919p4065342.html
Sent from the Solr - User mailing list archive at Nabble.com.
esday, May 21, 2013 5:06 PM
To: solr-user@lucene.apache.org
Subject: RE: How do I use CachedSqlEntityProcessor?
Thank you James & bbarani.
This worked in the sense that there was no error or exception in the data
import. Unfortunately, I do not see any of my Category1, Category2 etc. when
.
O. O.
bbarani wrote
> Try this..
> query="SELECT CategoryName,SKU from CAT_TABLE WHERE
> CategoryLevel=1" cacheKey="Cat1.SKU" cacheLookup="Product.SKU"
> processor="CachedSqlEntityProcessor">
>
Try this..
SKU* from
CAT_TABLE WHERE CategoryLevel=1" cacheKey=*"Cat1.SKU"*
cacheLookup="Product.SKU" processor="CachedSqlEntityProcessor">
Also not sure if you are using Alpha / Beta release of SO
ies, then use the
> "cacheKey" and "cacheLookup" parameters to instruct DIH how to do the
> join.
>
> Example:
> name="Cat1"
> cacheKey="SKU"
> cacheLookup="Product.SKU"
> query="SELECT CategoryName from CAT_TABLE where Cat
Try like this...
--
View this message in context:
http://lucene.472066.n3.nabble.com/How-do-I-use-CachedSqlEntityProcessor-tp4064919p4065030.html
Sent from the Solr - User mailing list archive at Nabble.com.
First remove the "where" condition from the child entities, then use the
"cacheKey" and "cacheLookup" parameters to instruct DIH how to do the join.
Example:
See http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor ,
particularly the 3rd con
d to me to use the
CachedSqlEntityProcessor
(http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor).
Hence I modified my db-data-confi
If I remember correctly, 3.6 DIH had bugs related to CachedSqlEntityProcessor
and some were fixed in 3.6.1, 3.6.2, but some were not fixed until 4.0. You
might want to use a 3.5 DIH jar with your 3.6 Solr. Or, post your
data-config.xml and maybe someone can figure something out.
James Dyer
Thanks James for your reply.
I have updated to 3.6.2. Now the NullPointerException is gone. But the
entities with CachedSqlEntityProcessor don't add anything to solr.
And entities without CachedSqlEntityProcessor, are working fine.
Why entities with CachedSqlEntityProcessor don't d
Subject: java.lang.NullPointerException. I am trying to use
CachedSqlEntityProcessor
I am in Solr 3.6.1.
The following entity gives java.lang.NullPointerException. How to debug
this? Here I am usingCachedSqlEntityProcessor
-trying-to-use-CachedSqlEntityProcessor-tp4059815.html
Sent from the Solr - User mailing list archive at Nabble.com.
It's hard to troubleshoot without debug logs. Pls pay attention that
regular configuration for CachedSqlEP is slightly different
http://wiki.apache.org/solr/DataImportHandler#CachedSqlEntityProcessor
see
where="xid=x.id"
On Wed, Jun 27, 2012 at 2:29 AM, ps_sra wrote:
> N
Not sure if this is the right forum to post this question. If not, please
excuse.
I'm trying to use the DataImportHandler with
processor="CachedSqlEntityProcessor" to speed up import from an RDBMS. While
processor="CachedSqlEntityProcessor" is much faster than
processor=
...@ingrambook.com]
Sent: Monday, May 07, 2012 1:47 PM
To: solr-user@lucene.apache.org
Cc: Brent Mills; dye.kel...@gmail.com; keithn...@dswinc.com
Subject: RE: Nested CachedSqlEntityProcessor running for each entity row with
Solr 3.6?
Dear Kellen, Brent & Keith,
There now are fixes available for 2 c
e.kel...@gmail.com]
Sent: Tuesday, May 08, 2012 2:39 AM
To: solr-user@lucene.apache.org
Subject: Re: Nested CachedSqlEntityProcessor running for each entity row with
Solr 3.6?
> In the mean time, I am hoping each of you can test these fixes with your
> installation. The best way to do this
> In the mean time, I am hoping each of you can test these fixes with your
> installation. The best way to do this is to get a fresh SVN checkout of the
> 3.6.1 branch
> (http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_3_6/),
> switch to the "solr" directory, then run "ant dist
.kel...@gmail.com]
Sent: Monday, May 07, 2012 9:43 AM
To: solr-user@lucene.apache.org
Subject: Nested CachedSqlEntityProcessor running for each entity row with Solr
3.6?
I just upgraded from Solr 3.4 to Solr 3.6; I'm using the same
data-import.xml for both versions. The import functioned pr
Your dataconfig.xml is already single threaded. Bug is in DIH 3.6.0 code.
There should be a link to the fixed jar in the comments.
On Mon, May 7, 2012 at 7:15 PM, not interesting wrote:
> > it sounds like
> > https://issues.apache.org/jira/browse/SOLR-3360
> > fix is committed, tests are on going
> it sounds like
> https://issues.apache.org/jira/browse/SOLR-3360
> fix is committed, tests are on going.
Hmm, I'm running solr behind tomcat; where can I configure Solr to use
only a single thread for testing?
Kellen
ed properly with
> 3.4.
>
> I'm using a nested entity to fetch authors associated with each
> document, and I'm using CachedSqlEntityProcessor to avoid hitting the
> DB an unreasonable number of times. However, when indexing, Solr
> indexes very slowly and appears to be fe
I just upgraded from Solr 3.4 to Solr 3.6; I'm using the same
data-import.xml for both versions. The import functioned properly with
3.4.
I'm using a nested entity to fetch authors associated with each
document, and I'm using CachedSqlEntityProcessor to avoid hitting the
DB an unre
syntax is valid and what's not. Here is my entity that worked in 3.1 but
> no longer works in 3.6:
>
> processor="CachedSqlEntityProcessor" where="UserID=Users.UserID">
>
--
Sincerely yours
Mikhail Khludnev
Tech Lead
Grid Dynamics
<http://www.griddynamics.com>
I've read some things in jira on the new functionality that was put into
caching in the DIH but I wouldn't think it should break the old behavior. It
doesn't look as though any errors are being thrown, it's just ignoring the
caching part and opening a ton of connections. Also I cannot find any
FYI my sub-entity looks like the following
On 11/15/11 10:42 AM, Mark wrote:
I am trying to use the CachedSqlEntityProcessor with Solr 1.4.2
however I am not seeing any performance gains. I've read some other
posts that reference cacheKey and cacheLookup however I don't see any
re
I am trying to use the CachedSqlEntityProcessor with Solr 1.4.2 however
I am not seeing any performance gains. I've read some other posts that
reference cacheKey and cacheLookup however I don't see any reference to
them in the wiki
http://wiki.apache.org/solr/DataImp
at, Oct 8, 2011 at 1:47 AM, Mikhail Khludnev
> wrote:
>> Hello,
>>
>> First of all I've got the same feeling for some time.
>> I've checked TestThreaded
>> <http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_4_0/solr/contr
>> ib
it doesn't cover CachedSqlEntityProcessor with fancy where="xid=
x.id" feature.
Pls find the test for it attached:
dih-mt-cached-race-condition-TestThreaded.java.patch (as a patch for
TestThreaded from tags/lucene_solr_3_4_0)
Most times testCached*MultiThread*_FullImport() fails (but really rare it
pass
put up a small and simplified test case together to demo the issue
> and then paste the link for that hosted attachment :)
>
> Sent from my iPhone
>
> On Sep 30, 2011, at 1:28 PM, Maria Vazquez wrote:
>
>> Hi,
>> I¹m using threads with JdbcDataStore and Cac
re and CachedSqlEntityProcessor.
> I noticed that if I make it single threaded CachedSqlEntityProcessor behaves
> as expected (it only queries the db once and caches all the rows). If I make
> it multi threaded it seems to make multiple db queries and when I debug the
> source code it looks like it is not
Hi,
I¹m using threads with JdbcDataStore and CachedSqlEntityProcessor.
I noticed that if I make it single threaded CachedSqlEntityProcessor behaves
as expected (it only queries the db once and caches all the rows). If I make
it multi threaded it seems to make multiple db queries and when I debug
I have come across an issue with the DIH where I get a null exception when
pre-caching entities. I expect my entity to have null values so this is a bit
of a roadblock for me. The issue was described more succinctly in this
discussion:
http://lucene.472066.n3.nabble.com/DataImportHandlerExcepti
select
queries), CachedSqlEntityProcessor works as expected and fires just 3
queries and does the matching after loading the entire data from 3 queries
in memory using the cache key...
When any one of the entity returns zero records, SOLR fires multiple queries
(i.e. it ignores the
cient.
On Wed, Apr 7, 2010 at 1:17 PM, bbarani wrote:
>
> Hi,
>
> I just thought of sharing a suggestion for overcoming OOM issues with
> CachedSQLEntityProcessor.
>
> Consider a scenario as below,
>
> If we have sub entities in DIH,
>
> ---> object
&g
Hi,
I just thought of sharing a suggestion for overcoming OOM issues with
CachedSQLEntityProcessor.
Consider a scenario as below,
If we have sub entities in DIH,
---> object
-->
object properties
cachedSqlEntityprocessor works as below,
•
:09 PM org.apache.solr.update.DirectUpdateHandler2
rollback
INFO: end_rollback
Thanks,
Barani
JavaGuy84 wrote:
Hi,
I am using CachedsqlEntityProcessor in my DIH dataconfig to reduce the
number of queries executed against the database ,
"CachedsqlEntityProcessor"/>
ce)
>>at sun.nio.cs.StreamEncoder.implWrite(Unknown Source)
>>at sun.nio.cs.StreamEncoder.write(Unknown Source)
>>at java.io.OutputStreamWriter.write(Unknown Source)
>>at java.io.BufferedWriter.flushBuffer(Unknown Source)
>>at java.
P
> rintStreamSocketLog.java:169)
>at
> com.metamatrix.common.comm.platform.socket.PrintStreamSocketLog.log(P
> rintStreamSocketLog.java:175)
>at
> com.metamatrix.common.comm.platform.socket.PrintStreamSocketLog.logEr
> ror(PrintStreamSocketLog.java:7
.java:71)
at
com.metamatrix.common.comm.platform.socket.client.SocketServerInstanc
eImpl$1.run(SocketServerInstanceImpl.java:578)
at java.lang.Thread.run(Unknown Source)
Mar 13, 2010 3:52:09 PM org.apache.solr.update.DirectUpdateHandler2 rollback
INFO: end_rollback
Thanks,
Barani
JavaGuy84 wrote:
>
> Hi,
>
> I am using Ca
Hi,
I am using CachedsqlEntityProcessor in my DIH dataconfig to reduce the
number of queries executed against the database ,
I having more than 2 million rows returned for Entity 2 and around 30
rows returned for entity1.
I am have set the heap size to 1 GB but even then I am always
Hi,
I am on the 1.4 Nightly build from September (still need to upgrade).
I am using CachedSqlEntityProcessor for my main queries but was hoping to
use it for my delta-imports as well. Is this possible?
I have a main entity called 'Article' and then
IF you c
Hi,
I am trying to use the CachedSqlEntityProcessor for one of my requirement. I
couldnt make the CachedSqlEntityProcessor accept more than one filter in the
'where' condition
I have something like this
My question is, how to make the CachedSqlEntityProcessor accept multi
StatusId = 1"
>>>
>>> >> query="SELECT LinkedCategoryBC, CmsArticleId as
>>> LinkedCatAricleId
>>> FROM LinkedCategoryBreadCrumb_SolrSearch (nolock)"
>>> processor="CachedSqlEntityProcesso
query="SELECT LinkedCategoryBC, CmsArticleId as
>> LinkedCatAricleId
>> FROM LinkedCategoryBreadCrumb_SolrSearch (nolock)"
>> processor="CachedSqlEntityProcessor"
>> cacheKey="Lin
query="SELECT LinkedCategoryBC, CmsArticleId as
> LinkedCatAricleId
> FROM LinkedCategoryBreadCrumb_SolrSearch (nolock)"
> processor="CachedSqlEntityProcessor"
> cacheKey="LinkedCatArticleId&quo
?
Any help is greatly appreciated!!!
--
View this message in context:
http://old.nabble.com/Help-using-CachedSqlEntityProcessor-tp27337635p27353768.html
Sent from the Solr - User mailing list archive at Nabble.com.
cacheKey and cacheLookup are required attributes .
On Thu, Jan 28, 2010 at 12:51 AM, KirstyS wrote:
>
> Thanks. I am on 1.4..so maybe that is the problem.
> Will try when I get back to work tomorrow.
> Thanks
>
>
> Rolf Johansson-2 wrote:
>>
>> I recently had iss
Thanks. I am on 1.4..so maybe that is the problem.
Will try when I get back to work tomorrow.
Thanks
Rolf Johansson-2 wrote:
>
> I recently had issues with CachedSqlEntityProcessor too, figuring out how
> to
> use the syntax. After a while, I managed to get it working with ca
I recently had issues with CachedSqlEntityProcessor too, figuring out how to
use the syntax. After a while, I managed to get it working with cacheKey and
cacheLookup. I think this is 1.4 specific though.
It seems you have double WHERE clauses, one in the query and one in the
where attribute.
Try
Hi, I have looked on the wiki. Using the CachedSqlEntityProcessor looks like
it was simple. But I am getting no speed benefit and am not sure if I have
even got the syntax correct.
I have a main root entity called 'article'.
And then I have a number of sub entities. One such entity
80 matches
Mail list logo
