Hi, We did further tests to see where the problem exactly is. These are our outcomes:
The content-length is calculated correctly, a quick test with curl showed this. The problem is that the stream with the gzip data is not fully consumed and afterwards not closed. Using the debugger with a breakpoint at org/apache/solr/common/util/Utils.java:575 shows that it won't enter the function readFully((entity.getContent()) most likely due to how the gzip stream content is wrapped and extracted beforehand. On line org/apache/solr/common/util/Utils.java:582 the consumeQuietly(entity) should close the stream but does not because of a silent exception. This seems to be the same as it is described in https://issues.apache.org/jira/browse/SOLR-14457 We saw that the problem happened also with correct GZIP responses from jetty. Not only with non-GZIP as described within the jira issue. Best, Johannes Am Do., 23. Apr. 2020 um 09:55 Uhr schrieb Johannes Siegert < johannes.sieg...@offerista.com>: > Hi, > > we want to use gzip-compression between our application and the solr > server. > > We use a standalone solr server version 8.4.1 and the prepackaged jetty as > application server. > > We have enabled the jetty gzip module by adding these two files: > > {path_to_solr}/server/modules/gzip.mod (see below the question) > {path_to_solr}/server/etc/jetty-gzip.xml (see below the question) > > Within the application we use a HttpSolrServer that is configured with > allowCompression=true. > > After we had released our application we saw that the number of > connections within the TCP-state CLOSE_WAIT rising up until the application > was not able to open new connections. > > > After a long debugging session we think the problem is that the header > "Content-Length" that is returned by the jetty is sometimes wrong when > gzip-compression is enabled. > > The solrj client uses a ContentLengthInputStream, that uses the header > "Content-Lenght" to detect if all data was received. But the InputStream > can not be fully consumed because the value of the header "Content-Lenght" > is higher than the actual content-length. > > Usually the method PoolingHttpClientConnectionManager.releaseConnection is > called after the InputStream was fully consumed. This give the connection > free to be reused or to be closed by the application. > > Due to the incorrect header "Content-Length" the > PoolingHttpClientConnectionManager.releaseConnection method is never called > and the connection stays active. After the connection-timeout of the jetty > is reached, it closes the connection from the server-side and the TCP-state > switches into CLOSE_WAIT. The client never closes the connection and so the > number of connections in use rises up. > > > Currently we try to configure the jetty gzip module to return no > "Content-Length" if gzip-compression was used. We hope that in this case > another InputStream implementation is used that uses the NULL-terminator to > see when the InputStream was fully consumed. > > Do you have any experiences with this problem or any suggestions for us? > > Thanks, > > Johannes > > > gzip.mod > > ----- > > DO NOT EDIT - See: > https://www.eclipse.org/jetty/documentation/current/startup-modules.html > > [description] > Enable GzipHandler for dynamic gzip compression > for the entire server. > > [tags] > handler > > [depend] > server > > [xml] > etc/jetty-gzip.xml > > [ini-template] > ## Minimum content length after which gzip is enabled > jetty.gzip.minGzipSize=2048 > > ## Check whether a file with *.gz extension exists > jetty.gzip.checkGzExists=false > > ## Gzip compression level (-1 for default) > jetty.gzip.compressionLevel=-1 > > ## User agents for which gzip is disabled > jetty.gzip.excludedUserAgent=.*MSIE.6\.0.* > > ----- > > jetty-gzip.xml > > ----- > > <?xml version="1.0"?> > <!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" " > http://www.eclipse.org/jetty/configure_9_3.dtd";> > > <!-- =============================================================== --> > <!-- Mixin the GZIP Handler --> > <!-- This applies the GZIP Handler to the entire server --> > <!-- If a GZIP handler is required for an individual context, then --> > <!-- use a context XML (see test.xml example in distribution) --> > <!-- =============================================================== --> > > <Configure id="Server" class="org.eclipse.jetty.server.Server"> > <Call name="insertHandler"> > <Arg> > <New id="GzipHandler" > class="org.eclipse.jetty.server.handler.gzip.GzipHandler"> > <Set name="minGzipSize"> > <Property name="jetty.gzip.minGzipSize" > deprecated="gzip.minGzipSize" default="2048" /> > </Set> > <Set name="checkGzExists"> > <Property name="jetty.gzip.checkGzExists" > deprecated="gzip.checkGzExists" default="false" /> > </Set> > <Set name="compressionLevel"> > <Property name="jetty.gzip.compressionLevel" > deprecated="gzip.compressionLevel" default="-1" /> > </Set> > <Set name="inflateBufferSize"> > <Property name="jetty.gzip.inflateBufferSize" > default="0" /> > </Set> > <Set name="deflaterPoolCapacity"> > <Property name="jetty.gzip.deflaterPoolCapacity" > default="-1" /> > </Set> > <Set name="syncFlush"> > <Property name="jetty.gzip.syncFlush" default="false" > /> > </Set> > > <Set name="excludedAgentPatterns"> > <Array type="String"> > <Item> > <Property name="jetty.gzip.excludedUserAgent" > deprecated="gzip.excludedUserAgent" default=".*MSIE.6\.0.*" /> > </Item> > </Array> > </Set> > > <Set name="includedMethodList"> > <Property name="jetty.gzip.includedMethodList" > default="GET,POST" /> > </Set> > <Set name="excludedMethodList"> > <Property name="jetty.gzip.excludedMethodList" > default="" /> > </Set> > </New> > </Arg> > </Call> > </Configure> > > ----- >
