Re: Docker and Solr Indexing

2019-02-12 Thread solrnoobie
Oh ok then that must no be the culprit then. I got this logs from our application server but I'm not sure if this is useful: Caused by: org.apache.solr.client.solrj.SolrServerException: org.apache.http.ParseException: Invalid content type: at org.apache.solr.client.solrj.impl.LBHttpSolrS

Re: Docker and Solr Indexing

2019-02-12 Thread Shawn Heisey
On 2/12/2019 6:56 AM, solrnoobie wrote: I know this is too late of a reply but I found this on our solr.log java.nio.file.NoSuchFileException: USUALLY, this is a harmless annoyance, not an indication of an actual problem. Some people have indicated that it causes problems when using the bac

Re: Docker and Solr Indexing

2019-02-12 Thread solrnoobie
I know this is too late of a reply but I found this on our solr.log java.nio.file.NoSuchFileException: /opt/solr/server/solr/primaryCollectionPERF_shard1_replica9/data/index/segments_78 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base

Re: Docker and Solr Indexing

2019-02-12 Thread solrnoobie
I know this is too late of a reply but I found this on our solr.log java.nio.file.NoSuchFileException: /opt/solr/server/solr/primaryCollectionPERF_shard1_replica9/data/index/segments_78 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) at java.base

Re: Docker and Solr Indexing

2018-09-12 Thread Shawn Heisey
On 9/12/2018 7:43 AM, Dominique Bejean wrote: Are you aware about issues in Java applications in Docker if java version is not 10 ? https://blog.docker.com/2018/04/improved-docker-container-integration-with-java-10/ Solr explicitly sets heap size when it starts, so Java is *NOT* determining th

Re: Docker and Solr Indexing

2018-09-12 Thread Dominique Bejean
Hi, Are you aware about issues in Java applications in Docker if java version is not 10 ? https://blog.docker.com/2018/04/improved-docker-container-integration-with-java-10/ Regards. Dominique Le mer. 12 sept. 2018 à 05:42, Shawn Heisey a écrit : > On 9/11/2018 9:20 PM, solrnoobie wrote: > >

Re: Docker and Solr Indexing

2018-09-11 Thread Shawn Heisey
On 9/11/2018 9:20 PM, solrnoobie wrote: So what we did is we upgraded the instances to 16 gigs and we rarely encounter this now. So what we did was to increase the batch size to 500 instead of 50 and it worked for our test data. But when we tried 1000 batch size, the invalid content type error r

Re: Docker and Solr Indexing

2018-09-11 Thread solrnoobie
Thank you all for the kind and timely reply. So what we did is we upgraded the instances to 16 gigs and we rarely encounter this now. So what we did was to increase the batch size to 500 instead of 50 and it worked for our test data. But when we tried 1000 batch size, the invalid content type err

Re: Docker and Solr Indexing

2018-09-11 Thread Jan Høydahl
You have not shed any light on what the reason for the container restart was, and there is too little information about your setup and Solr usage to guess what goes on. Whether 4Gb is sufficient or not depends on how much data and queries you plan for each shard to handle, how much heap you give

Re: Docker and Solr Indexing

2018-09-11 Thread Walter Underwood
4 Gb is very small for Solr. Solr is not designed for Dockerized, fail-often use. We use a LOT of Docker ECS, but all of our Solr servers are on EC2 instances. That’s about sixty instances in several clusters. We run an 8 Gb heap for all our Solr instances. Instances in our biggest cluster (in t

Docker and Solr Indexing

2018-09-10 Thread solrnoobie
So we have a dockerized aws environment with the solr docker container having only 4 gigs for max ram. Our problem is whenever we index, the container containing the leader shard will restart after around 2 or less minutes of index time (batch is 50 docs per batch with 3 threads in our app thread