Hi! I had the same issue and found that actual problem with the file limit (in spite of the error message) To increase file limit:
On Linux, you can increase the limits by running the following command as root: sysctl -w vm.max_map_count=262144 To set this value permanently, update the vm.max_map_count setting in /etc/sysctl.conf. To verify after rebooting, run sysctl vm.max_map_count. Hope, it'll help -- Vadim > -----Original Message----- > From: Martin Frank Hansen (MHQ) [mailto:m...@kmd.dk] > Sent: Tuesday, February 12, 2019 3:25 PM > To: solr-user@lucene.apache.org > Subject: unable to create new threads: out-of-memory issues > > Hi, > > I am trying to create an index on a small Linux server running Solr-7.5.0, but > keep running into problems. > > When I try to index a file-folder of roughly 18 GB (18000 files) I get the > following error from the server: > > java.lang.OutOfMemoryError: unable to create new native thread. > > From the server I can see the following limits: > > User$ ulimit -a > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 257568 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 1024 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 257568 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited > > I do not see any limits on threads only on open files. > > I have added a autoCommit of a maximum of 1000 documents, but that did > not help. How can I increase the thread limit, or is there another way of > solving this issue? Any help is appreciated. > > Best regards > > Martin > > Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du > KMD’s > Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der fortæller, hvordan > vi behandler oplysninger om dig. > > Protection of your personal data is important to us. Here you can read KMD’s > Privacy Policy<http://www.kmd.net/Privacy-Policy> outlining how we process > your personal data. > > Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information. > Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst informere > afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi dig slette > e- > mailen i dit system uden at videresende eller kopiere den. Selvom e-mailen og > ethvert vedhæftet bilag efter vores overbevisning er fri for virus og andre > fejl, > som kan påvirke computeren eller it-systemet, hvori den modtages og læses, > åbnes den på modtagerens eget ansvar. Vi påtager os ikke noget ansvar for > tab og skade, som er opstået i forbindelse med at modtage og bruge e-mailen. > > Please note that this message may contain confidential information. If you > have received this message by mistake, please inform the sender of the > mistake by sending a reply, then delete the message from your system > without making, distributing or retaining any copies of it. Although we > believe > that the message and any attachments are free from viruses and other errors > that might affect the computer or it-system where it is received and read, the > recipient opens the message at his or her own risk. We assume no > responsibility for any loss or damage arising from the receipt or use of this > message.
