On Mon, Jun 16, 2014 at 6:57 AM, Floyd Wu <floyd...@gmail.com> wrote:
> Hi Mikhail, > What is the pros. to disable tlog? > I consumes the heap much providing the benefits (real-time get, recover uncommitted docs on failure) which are not necessary in old-school bulk index scenario. > Each of my xml file contained to doc, one is main content and the other is > acl. > How I can guess how many of them you have? Once again submitting a few (let's say ten) huge files in parallel allows to utilize indexing jvm fully, and yields the best performance. > Currently I'm not using SolrCloud due to my poor understanding of this > architecture and pros/cons. > The main system is developed using .Net C# so using SolrJ won't be a > solution. > anyway, if you submit small requests by C# code via REST, make sure that http keep-alive is enabled, and you don't waste time for establishing TCP connection. I might be wrong but I've thought Lucid guys provide some C# client or just its' scratch for Solr. Don't they? > > Floyd > > > > 2014-06-15 18:14 GMT+08:00 Mikhail Khludnev <mkhlud...@griddynamics.com>: > > > Hello Floyd, > > > > Did you consider to disable tlog? > > Does a file consist of many docs? > > Do you have SolrCloud? Do you use just sh/curl or have a java program? > > DIH is not really performant so far. Submitting roughly ten huge files in > > parallel is a way to perform good. Once again, nuke tlog. > > > > > > On Sun, Jun 15, 2014 at 12:44 PM, Floyd Wu <floyd...@gmail.com> wrote: > > > > > Hi, > > > I have many XML Message file formatted like this > > > https://wiki.apache.org/solr/UpdateXmlMessages > > > > > > These files are generated by my index builder daily. > > > Currently I am sending these file through http post to Solr but > > sometimes I > > > hit OOM exception or pending too many tlog. > > > > > > Do you have better way to "import" these files to Solr to build index? > > > > > > Thanks for the suggestion > > > > > > Floyd > > > > > > > > > > > -- > > Sincerely yours > > Mikhail Khludnev > > Principal Engineer, > > Grid Dynamics > > > > <http://www.griddynamics.com> > > <mkhlud...@griddynamics.com> > > > -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics <http://www.griddynamics.com> <mkhlud...@griddynamics.com>
