Re: Dataimport handler showing idle status with multiple shards

2017-12-05 Thread Sarah Weissman
From: Shawn Heisey Reply-To: "solr-user@lucene.apache.org" Date: Tuesday, December 5, 2017 at 1:31 PM To: "solr-user@lucene.apache.org" Subject: Re: Dataimport handler showing idle status with multiple shards On 12/5/2017 10:47 AM, Sarah Weissman wrote: I’ve rec

Re: Dataimport handler showing idle status with multiple shards

2017-12-05 Thread Shawn Heisey
On 12/5/2017 10:47 AM, Sarah Weissman wrote: I’ve recently been using the dataimport handler to import records from a database into a Solr cloud collection with multiple shards. I have 6 dataimport handlers configured on 6 different paths all running simultaneously against the same DB. I’ve no

RE: DataImport Handler Out of Memory

2017-09-27 Thread Allison, Timothy B.
https://wiki.apache.org/solr/DataImportHandlerFaq#I.27m_using_DataImportHandler_with_a_MySQL_database._My_table_is_huge_and_DataImportHandler_is_going_out_of_memory._Why_does_DataImportHandler_bring_everything_to_memory.3F -Original Message- From: Deeksha Sharma [mailto:dsha...@flexera.co

Re: Dataimport handler Date

2014-03-06 Thread Gora Mohanty
On 7 March 2014 08:50, Pritesh Patel wrote: > I'm using the dataimporthandler to index data from a mysql DB. Been > running it just fine. I've been using full-imports. I'm now trying > implement the delta import functionality. > > To implement the delta query, you need to be reading the last_inde

Re: dataimport handler

2014-01-22 Thread Shalin Shekhar Mangar
I'm guessing that "id" in your schema.xml is also a unique key field. If so, each document must have an id field or Solr will refuse to index them. DataImportHandler will map the id field in your table to Solr schema's id field only if you have not specified a mapping. On Thu, Jan 23, 2014 at 3:0

Re: Dataimport handler exception when migrating from 4.4 to 4.6. Help needed

2013-12-22 Thread William Bell
The best practice for upgrading is take the distribution and expand it. Then take your cores and replace it. Then you are guaranteed to get the jars and not have other WARs/JARs hanging around. On Sun, Dec 22, 2013 at 7:24 PM, Shawn Heisey wrote: > On 12/22/2013 9:51 AM, William Pierce wrote:

Re: Dataimport handler exception when migrating from 4.4 to 4.6. Help needed

2013-12-22 Thread Shawn Heisey
On 12/22/2013 9:51 AM, William Pierce wrote: > My configurations works nicely with solr 4.4. I am encountering a > configuration error when I try to upgrade from 4.4 to 4.6. All I did was the > following: > > a) Replace the 4.4 solr.war file with the 4.6 solr.war in the tomcat/lib > folder. I

Re: DataImport Handler, writing a new EntityProcessor

2013-12-19 Thread Mathias Lux
Hi! Thanks for all the advice! I finally did it, the most annoying error that took me the best of a day to figure out was that the state variable here had to be reset: https://bitbucket.org/dermotte/liresolr/src/d27878a71c63842cb72b84162b599d99c4408965/src/main/java/net/semanticmetadata/lire/solr/

Re: DataImport Handler, writing a new EntityProcessor

2013-12-18 Thread P Williams
Hi Mathias, I'd recommend testing one thing at a time. See if you can get it to work for one image before you try a directory of images. Also try testing using the solr-testframework using your ide (I use Eclipse) to debug rather than your browser/print statements. Hopefully that will give you

Re: DataImport Handler, writing a new EntityProcessor

2013-12-18 Thread Mathias Lux
Unfortunately it is the same in non-debug, just the first document. I also output the params to sout, but it seems only the first one is ever arriving at my custom class. I've the feeling that I'm doing something seriously wrong here, based on a complete misunderstanding :) I basically assume that

RE: DataImport Handler, writing a new EntityProcessor

2013-12-18 Thread Dyer, James
The first thing I would suggest is to try and run it not in debug mode. DIH's debug mode limits the number of documents it will take in, so that might be all that is wrong here. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: mathias@gmail.com [mailto:mathi

Re: dataimport handler

2013-05-10 Thread Shalin Shekhar Mangar
Hmm, I will fix. https://issues.apache.org/jira/browse/SOLR-4788 On Thu, May 9, 2013 at 8:35 PM, William Bell wrote: > It does not work anymore in 4.x. > > ${dih.last_index_time} does work, but the entity version does not. > > Bill > > > > On Tue, May 7, 2013 at 4:19 PM, Shalin Shekhar Mangar

Re: dataimport handler

2013-05-09 Thread William Bell
It does not work anymore in 4.x. ${dih.last_index_time} does work, but the entity version does not. Bill On Tue, May 7, 2013 at 4:19 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > Using ${dih..last_index_time} should work. Make sure you put > it in quotes in your query. > > > On

Re: dataimport handler

2013-05-07 Thread Shalin Shekhar Mangar
Using ${dih..last_index_time} should work. Make sure you put it in quotes in your query. On Tue, May 7, 2013 at 12:07 PM, Eric Myers wrote: > In the data import handler I have multiple entities. Each one > generates a date in the > dataimport.properties i.e. entityname.last_index_time. > > H

Re: Dataimport handler

2013-04-23 Thread William Bell
I also get this. 4.2+ On Fri, Apr 19, 2013 at 10:43 PM, Eric Myers wrote: > I have multiple parallel entities in my document and when I run an import > there are times like > xxx.last_index_time > where xxx is the name of the entity. > > I tried accessing these using dih.xxx.last_index_time but

RE: DataImport Handler : Transformer Function Eval Failed Error

2012-11-05 Thread Mishra, Shikhar
Looks like it will be helpful. I'm going to give it a shot. Thanks, Otis. Shikhar From: Otis Gospodnetic [otis.gospodne...@gmail.com] Sent: Friday, November 02, 2012 4:36 PM To: solr-user@lucene.apache.org Subject: Re: DataImport Handler : Transf

Re: DataImport Handler : Transformer Function Eval Failed Error

2012-11-02 Thread Otis Gospodnetic
> > Thanks, > Shikhar > > -Original Message- > From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] > Sent: Thursday, November 01, 2012 8:13 PM > To: solr-user@lucene.apache.org > Subject: Re: DataImport Handler : Transformer Function Eval Failed Error > > Hi, >

RE: DataImport Handler : Transformer Function Eval Failed Error

2012-11-02 Thread Mishra, Shikhar
-Original Message- From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] Sent: Thursday, November 01, 2012 8:13 PM To: solr-user@lucene.apache.org Subject: Re: DataImport Handler : Transformer Function Eval Failed Error Hi, That looks a little painful... what are you trying to achieve by

Re: DataImport Handler : Transformer Function Eval Failed Error

2012-11-01 Thread Otis Gospodnetic
Hi, That looks a little painful... what are you trying to achieve by storing JSON in there? Maybe there's a simpler way to get there... Otis -- Performance Monitoring - http://sematext.com/spm On Nov 1, 2012 6:16 PM, "Mishra, Shikhar" wrote: > Hi, > > I'm trying to store a list of JSON objects

RE: Dataimport Handler in solr 3.6.1

2012-08-30 Thread Dyer, James
Ingram Content Group (615) 213-4311 -Original Message- From: mechravi25 [mailto:mechrav...@yahoo.co.in] Sent: Tuesday, August 21, 2012 7:47 AM To: solr-user@lucene.apache.org Subject: RE: Dataimport Handler in solr 3.6.1 Hi James, Thanks for the suggestions. Actually it is cacheLookup="ent1

RE: Dataimport Handler in solr 3.6.1

2012-08-21 Thread mechravi25
Hi James, Thanks for the suggestions. Actually it is cacheLookup="ent1.id" . had misspelt it. Also, I will be needing the transformers mentioned as there are other columns as well. Actually tried using the 3.5 DIH jars in 3.6.1 and indexed the same and the indexing was successful. But I wanted

RE: Dataimport Handler in solr 3.6.1

2012-08-14 Thread Dyer, James
One thing I notice in your configuration...the child entity has this: cacheLookup="ent1.uid" but your parent entity doesn't have a "uid" field. Also, you have these 3 transformers: RegexTransformer,DateFormatTransformer,TemplateTransformer but none of your columns seem to make use of these.

Re: dataimport handler (DIH) - notify when it has finished?

2012-05-01 Thread Gora Mohanty
On 1 May 2012 23:12, geeky2 wrote: > Hello all, > > is there a notification / trigger / callback mechanism people use that > allows them to know when a dataimport process has finished? > > we will be doing daily delta-imports and i need some way for an operations > group to know when the DIH has f

Re: dataimport handler with mysql: wrong field mapping

2008-12-15 Thread Luca Molteni
Have you tried using the options in the schema.xml? After the indexing, take a look to the fields DIH has generated. Bye, L.M. 2008/12/15 jokkmokk : > > HI, > > I'm desperately trying to get the dataimport handler to work, however it > seems that it just ignores the field name mapping. > I

Re: dataimport handler with mysql: wrong field mapping

2008-12-15 Thread jokkmokk
sorry, I'm using the 1.3.0 release. I've now worked around that issue by using aliases in the sql statement so that no mapping is needed. This way it works perfectly. best regards Stefan Shalin Shekhar Mangar wrote: > > Which solr version are you using? > -- View this message in context: h

Re: dataimport handler with mysql: wrong field mapping

2008-12-15 Thread Shalin Shekhar Mangar
Which solr version are you using? On Mon, Dec 15, 2008 at 6:04 PM, jokkmokk wrote: > > HI, > > I'm desperately trying to get the dataimport handler to work, however it > seems that it just ignores the field name mapping. > I have the fields "body" and "subject" in the database and those are call

Re: dataimport handler multiple databases

2008-04-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
each entity has an optional attribute called dataSource. If you have multiple dataSources give them a name and use the name is dataSource .So you solrconfig must look like /home/username/data-config.xml datasource-1 com.mysql.jdbc.Driver dat