Re: error with delta import

2008-10-26 Thread Chris Hostetter
: The case in point is DIH. DIH uses the standard DOM parser that comes : w/ JDK. If it reads the xml properly do we need to complain?. I guess : that data-config.xml may not be used for any other purposes. that's a vague statement as well ... there is no such thing as "the standard DOM parser

Re: error with delta import

2008-10-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
The case in point is DIH. DIH uses the standard DOM parser that comes w/ JDK. If it reads the xml properly do we need to complain?. I guess that data-config.xml may not be used for any other purposes. On Wed, Oct 22, 2008 at 10:10 PM, Walter Underwood <[EMAIL PROTECTED]> wrote: > On 10/22/08 8:5

Re: error with delta import

2008-10-22 Thread Walter Underwood
On 10/22/08 8:57 AM, "Steven A Rowe" <[EMAIL PROTECTED]> wrote: > Telling people that it's not a problem (or required!) to write non-well-formed > XML, because a particular XML parser can't accept well-formed XML is kind of > insidious. I'm with you all the way on this. A parser which accepts no

RE: error with delta import

2008-10-22 Thread Steven A Rowe
Hi Shalin, I wasn't talking about the behavior of parsers in the wild, but rather about the XML specification (paraphrasing): 1. An XML document is not well-formed unless it matches the production labeled document. 2. Violations of well-formedness constraints are fatal errors. 3. Once a fatal e

Re: error with delta import

2008-10-22 Thread Shalin Shekhar Mangar
Actually, most XML parsers don't require you to escape such characters in attributes. You are welcome to try this out, just look at the example-DIH :) On Tue, Oct 21, 2008 at 11:11 PM, Steven A Rowe <[EMAIL PROTECTED]> wrote: > Wow, I really should read more closely before I respond - I see now,

RE: error with delta import

2008-10-21 Thread Steven A Rowe
Wow, I really should read more closely before I respond - I see now, Noble, that you were talking about DIH's ability to parse escaped '<'s in attribute values, rather than about whether '<' was an acceptable character in attribute values. I should repurpose my remarks to note to Shalin, though

RE: error with delta import

2008-10-21 Thread Steven A Rowe
On 10/21/2008 at 12:14 AM, Noble Paul നോബിള്‍ नोब्ळ् wrote: > On Tue, Oct 21, 2008 at 12:56 AM, Shalin Shekhar Mangar <[EMAIL PROTECTED]> > wrote: > > Your data-config looks fine except for one thing -- you do not need to > > escape '<' character in an XML attribute. It maybe throwing off the > >

Re: error with delta import

2008-10-21 Thread Florian Aumeier
hello everybody thank you all for your help and ideas it works now. what are we doing wrong? Florian actually, I am not sure what we did wrong. After we started it again from scratch and with the simplified query it all worked as expected. Regards Florian

Re: error with delta import

2008-10-20 Thread Noble Paul നോബിള്‍ नोब्ळ्
you are still doing a delta import . with the modified data-config you must do a command=full-import On Mon, Oct 20, 2008 at 7:31 PM, Florian Aumeier <[EMAIL PROTECTED]> wrote: > sorry to bother you again, but the delta import still does not work for me > :-( > > We tried: > * delta-import by ful

Re: error with delta import

2008-10-20 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Tue, Oct 21, 2008 at 12:56 AM, Shalin Shekhar Mangar <[EMAIL PROTECTED]> wrote: > Your data-config looks fine except for one thing -- you do not need to > escape '<' character in an XML attribute. It maybe throwing off the parsing > code in DataImportHandler. not really '<' is fine in attribute

Re: error with delta import

2008-10-20 Thread Shalin Shekhar Mangar
Your data-config looks fine except for one thing -- you do not need to escape '<' character in an XML attribute. It maybe throwing off the parsing code in DataImportHandler. Another question, does the full-import work fine? On Mon, Oct 20, 2008 at 7:31 PM, Florian Aumeier <[EMAIL PROTECTED]>wrote

Re: error with delta import

2008-10-20 Thread Florian Aumeier
sorry to bother you again, but the delta import still does not work for me :-( We tried: * delta-import by full-import query=""> with entity=articles-delta&clean=false * delta-import by full-import with simplified query * delta-import with simplified query deltaQuery="SELECT * FROM

Re: error with delta import

2008-10-17 Thread Florian Aumeier
Lance Norskog schrieb: If you make a database view with the query, it is easy to examine the data you want to index. Then, your solr import query would just pull the view. The Solr setup file is much simpler this way. I will try and let you know.

Re: error with delta import

2008-10-17 Thread Florian Aumeier
Noble Paul നോബിള്‍ नोब्ळ् schrieb: the last-index_time is available only from second time onwards that is . It expects a full-import to be done first It knows that by the presence of dataimport.properties in the config directory. Did you check if it is present? yes, I did a check and the fil

Re: error with delta import

2008-10-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
the last-index_time is available only from second time onwards that is . It expects a full-import to be done first It knows that by the presence of dataimport.properties in the config directory. Did you check if it is present? On Thu, Oct 16, 2008 at 5:33 PM, Florian Aumeier <[EMAIL PROTECTED]>

RE: error with delta import

2008-10-16 Thread Lance Norskog
: Wednesday, October 15, 2008 2:46 AM To: solr-user@lucene.apache.org Subject: Re: error with delta import The delta implementation is a bit fragile in DIH for complex queries I recommend you do delta-import using a full-import .

Re: error with delta import

2008-10-16 Thread Florian Aumeier
Noble Paul നോബിള്‍ नोब्ळ् schrieb: Well, when doing the way you described below (full-import with the delta query), the '${dataimporter.last_index_time}' timestamp is empty: I guess this was fixed post 1.3 . probably you can take dataimporthandler.jar from a nightly build (you may also need

Re: error with delta import

2008-10-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Thu, Oct 16, 2008 at 2:08 PM, Florian Aumeier <[EMAIL PROTECTED]> wrote: > Noble Paul നോബിള്‍ नोब्ळ् schrieb: >> >> The delta implementation is a bit fragile in DIH for complex queries >> >> > > that's too bad. It's a nice interface and less complex to configure than to > go the XML /update way.

Re: error with delta import

2008-10-16 Thread Florian Aumeier
Noble Paul നോബിള്‍ नोब्ळ् schrieb: The delta implementation is a bit fragile in DIH for complex queries that's too bad. It's a nice interface and less complex to configure than to go the XML /update way. Well, when doing the way you described below (full-import with the delta query), the

Re: error with delta import

2008-10-15 Thread Noble Paul നോബിള്‍ नोब्ळ्
The delta implementation is a bit fragile in DIH for complex queries I recommend you do delta-import using a full-import it can be done as follows define a diffferent entity when you wish to do a full-import pass the request parameter entity=articles-full

Re: error with delta import

2008-10-15 Thread Florian Aumeier
Shalin Shekhar Mangar schrieb: You are missing the "pk" field (primary key). This is used for delta imports. I added the pk field and rebuild the index yesterday. However, when I run the delta-import, I still have this error message in the log: INFO: Starting delta collection. Oct 15, 2008

Re: error with delta import

2008-10-14 Thread Noble Paul നോബിള്‍ नोब्ळ्
the query makes my head spin . joining on an sql does not enable you to populate multivalued fields . Otherwise , it is all fine pk attribute is missing in the entity On Tue, Oct 14, 2008 at 6:16 PM, Florian Aumeier <[EMAIL PROTECTED]> wrote: > Noble Paul നോബിള്‍ नोब्ळ् schrieb: >> >> apparently

Re: error with delta import

2008-10-14 Thread Shalin Shekhar Mangar
You are missing the "pk" field (primary key). This is used for delta imports. On Tue, Oct 14, 2008 at 6:16 PM, Florian Aumeier <[EMAIL PROTECTED]>wrote: > Noble Paul നോബിള്‍ नोब्ळ् schrieb: > >> apparently you have not specified the deltaQuery attribute in the entity. >> Check the delta-import s

Re: error with delta import

2008-10-14 Thread Florian Aumeier
Noble Paul നോബിള്‍ नोब्ळ् schrieb: apparently you have not specified the deltaQuery attribute in the entity. Check the delta-import section in the wiki http://wiki.apache.org/solr/DataImportHandler or you can share your data-config file and we can take a quick look here is my data-config. I

Re: error with delta import

2008-10-14 Thread Noble Paul നോബിള്‍ नोब्ळ्
apparently you have not specified the deltaQuery attribute in the entity. Check the delta-import section in the wiki http://wiki.apache.org/solr/DataImportHandler or you can share your data-config file and we can take a quick look On Tue, Oct 14, 2008 at 5:05 PM, Florian Aumeier <[EMAIL PROTEC

error with delta import

2008-10-14 Thread Florian Aumeier
Hi, I have some problems with delta-import. Here are the infos I have. The result from the web API, apparantly everything is fine: − 0 0 − − db-psql-data-config.xml idle − 0:29:30.615 1 16194 0 0 2008-10-14 11:23:31 2008-10-14 11:23:31 2008-10-14 11:32:16 2008-10-14 11:32:16 16194 − T