Norberto, You said last week:
"why not generate your SQL output directly into your oracle server as a file, upload the file to your SOLR server? Then the data file is local to your SOLR server , you will bypass any WAN and firewall you may be having. (or some variation of it, sql -> SOLR server as file, etc..)" I think this is the best solution that we are going to without changing too much on our setup. Like said we have file name "test.xml" which come from SQL output , we put it locally on the solr server under "/opt/test.xml" So, I need to execute the commands from solr system to add and update this to the solr data/indexes. What commands do I have to use, for example the xml file named" /opt/test.xml" ? Thanks Francis -----Original Message----- From: Norberto Meijome [mailto:numard...@gmail.com] Sent: Sunday, July 05, 2009 3:57 AM To: Francis Yakin Cc: solr-user@lucene.apache.org Subject: Re: Is there any other way to load the index beside using "http" connection? On Thu, 2 Jul 2009 11:02:28 -0700 Francis Yakin <fya...@liquid.com> wrote: > Norberto, Thanks for your input. > > What do you mean with "Have you tried connecting to SOLR over HTTP from > localhost, therefore avoiding any firewall issues and network latency ? it > should work a LOT faster than from a remote site." ? > > > Here are how our servers lay out: > > 1) Database ( Oracle ) is running on separate machine > 2) Solr master is running on separate machine by itself > 3) 6 solr slaves ( these 6 pulll the index from master using rsync) > > We have a SQL(Oracle) script to post the data/index from Oracle Database > machine to Solr Master over http. We wrote those script(Someone in Oracle > Database administrator write it). You said in your other email you are having issues with slow transfers between 1) and 2). Your subject relates to the data transfer between 1) and 2, - 2) and 3) is irrelevant to this part. My question (what you quoted above) relates to the point you made about it being slow ( WHY is it slow?), and issues with opening so many connections through firewall. so, I'll rephrase my question (see below...) [....] > > We can not do localhost since it's solr is not running on Oracle machine. why not generate your SQL output directly into your oracle server as a file, upload the file to your SOLR server? Then the data file is local to your SOLR server , you will bypass any WAN and firewall you may be having. (or some variation of it, sql -> SOLR server as file, etc..) Any speed issues that are rooted in the fact that you are posting via HTTP (vs embedded solr or DIH) aren't going to go away. But it's the simpler approach without changing too much of your current setup. > Another alternative that we think of is to transform XML into CSV and > import/export it. > > How about if LUSQL, some mentioned about this? Is this apps free(open source) > application? Do you have any experience with this apps? Not i, sorry. Have you looked into DIH? It's designed for this kind of work. B _________________________ {Beto|Norberto|Numard} Meijome "Great spirits have often encountered violent opposition from mediocre minds." Albert Einstein I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned.
