hi all, I was wondering if anyone could point me in the right direction.
I am looking into whether enabling Gzip HTTP compression for our Solr clusters (all running Solr 6.6.5) would help performance; my problem is that I can’t figure out how to do that. Our infrastructure setup is like this: our applications are running on a Cloud Foundry PAAS environment, but our Solr clusters run elsewhere. Communication between applications and Solr clusters is secured by firewalls on every Solr machine (we do have a Socks Proxy set up in the CF environment, but unfortunately we can't use that for Solr because of the incompatibility between Zookeeper and Java Nio I/O - much to the chagrin of our sysadmin). We think that HTTP compression might be very interesting for us because of the hight volume of traffic between two separate environments. Here’s what I found out so far: (re. config changes in Solr’s embedded Jetty) - I’m aware that this is mostly a matter of configuring Jetty; - it seems that this should preferably be set in the solr-jetty-context.xml file; - this seems to relate to enabling Jetty's “GzipHandler” (re. gzip ‘module’ activation ..?) - it puzzles me that https://aroratimus.blogspot.com/2017/08/jettyserver-9.html mentions that Jetty’s GzipHandler should be enabled using two files not found in Solr's embedded Jetty: server/etc/jetty-gzip.xml and server/modules/gzip.mod (they are available when installing Jetty separately though); - apparently, Jetty's Gzip module should be activated by adding —add-to-start=gzip to the server startup command. For the embedded Jetty in Solr, it seems that this would require changing the solr startup script (re. changes in Solr client) - the calling application should add the HTTP Accept-Encoding: gzip, deflate ( according to https://menelic.com/2015/12/04/deploying-solrcloud-across-multiple-data-centers-dc-performance/ ) I wonder, has anyone ever got this working? In particular: - is that gzip ‘module’ activation necessary? That would seem a bit far-fetched, because it involves files not found in the Solr installation and possibly hacking the Solr startup script; - what did you add to solr-jetty-context.xml in order to enable gzip compression? I suppose that situations with high volumes of external network traffic between Solr and Client must be quite rare. Otherwise I’d think that a feature that potentially offers such obvious benefits (one of the pages above mentions a drop of 75% of network traffic and a 60% faster response time) would have been turned into an “enable http compression yes/no” setting by now :) Anyhow, we’re stuck with it … I hope I can get it working. Thank in advance for any advice! Luthien Api developer Europeana.eu -- Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
