Hi Mikhail Thanks for you suggestions. Floyd
2014-06-16 17:28 GMT+08:00 Mikhail Khludnev <mkhlud...@griddynamics.com>: > On Mon, Jun 16, 2014 at 6:57 AM, Floyd Wu <floyd...@gmail.com> wrote: > > > Hi Mikhail, > > What is the pros. to disable tlog? > > > I consumes the heap much providing the benefits (real-time get, recover > uncommitted docs on failure) which are not necessary in old-school bulk > index scenario. > > > > Each of my xml file contained to doc, one is main content and the other > is > > acl. > > > How I can guess how many of them you have? Once again submitting a few > (let's say ten) huge files in parallel allows to utilize indexing jvm > fully, and yields the best performance. > > > > Currently I'm not using SolrCloud due to my poor understanding of this > > architecture and pros/cons. > > The main system is developed using .Net C# so using SolrJ won't be a > > solution. > > > anyway, if you submit small requests by C# code via REST, make sure that > http keep-alive is enabled, and you don't waste time for establishing TCP > connection. I might be wrong but I've thought Lucid guys provide some C# > client or just its' scratch for Solr. Don't they? > > > > > > Floyd > > > > > > > > 2014-06-15 18:14 GMT+08:00 Mikhail Khludnev <mkhlud...@griddynamics.com > >: > > > > > Hello Floyd, > > > > > > Did you consider to disable tlog? > > > Does a file consist of many docs? > > > Do you have SolrCloud? Do you use just sh/curl or have a java program? > > > DIH is not really performant so far. Submitting roughly ten huge files > in > > > parallel is a way to perform good. Once again, nuke tlog. > > > > > > > > > On Sun, Jun 15, 2014 at 12:44 PM, Floyd Wu <floyd...@gmail.com> wrote: > > > > > > > Hi, > > > > I have many XML Message file formatted like this > > > > https://wiki.apache.org/solr/UpdateXmlMessages > > > > > > > > These files are generated by my index builder daily. > > > > Currently I am sending these file through http post to Solr but > > > sometimes I > > > > hit OOM exception or pending too many tlog. > > > > > > > > Do you have better way to "import" these files to Solr to build > index? > > > > > > > > Thanks for the suggestion > > > > > > > > Floyd > > > > > > > > > > > > > > > > -- > > > Sincerely yours > > > Mikhail Khludnev > > > Principal Engineer, > > > Grid Dynamics > > > > > > <http://www.griddynamics.com> > > > <mkhlud...@griddynamics.com> > > > > > > > > > -- > Sincerely yours > Mikhail Khludnev > Principal Engineer, > Grid Dynamics > > <http://www.griddynamics.com> > <mkhlud...@griddynamics.com> >
