DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG· RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT <http://issues.apache.org/bugzilla/show_bug.cgi?id=42546>. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND· INSERTED IN THE BUG DATABASE.
http://issues.apache.org/bugzilla/show_bug.cgi?id=42546 Summary: Cannot send a Content-Length header larger than 2GB when connecting to Tomcat with Mod_JK Product: Tomcat 5 Version: 5.5.23 Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: P2 Component: Connector:AJP AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] Summary ------- I am having some trouble using Mod_Jk to serve dynamically generated content that is larger than 2GB. The file is sent successfully, but the Content-Length header is not set properly, forcing chunked encoding. The client downloading this file is then not given an indication of how long the transfer will take to complete. If I use Mod_Proxy to serve the file, or connect directly to Tomcat's HTTP1.1 connector, the Content-Length is set properly and the client can follow the download progress to completion. In my own testing, it seems the Content-Length header is wrapping beyond 2^31, and when Tomcat's AJP connector constructs the AJP packets, it sees a negative Content-Length header and drops this, setting the Content-Length to zero and forcing chunked encoding. I don't have enough Java knowledge to understand the code in question, so hopefully some test cases and a sample webapp will provide enough of a starting point to resolve the issue. Software -------- This test was conducted on 64bit AMD boxes on both a regular RedHat Enterprise Linux 4 update 4 machine and a a Xen instance of the same software. ~ # uname -a Linux sandbox 2.6.12.6-xenU #1 SMP Thu Jul 6 23:24:02 GMT 2006 x86_64 x86_64 x86_64 GNU/Linux ~ # uname -a Linux testbox 2.6.9-42.0.3.ELsmp #1 SMP Mon Sep 25 17:24:31 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux Tomcat is the Core binary distribution of 5.5.23 running on the 64bit version of Java 1.5.0_10 (also tested with 32bit). ~ # java -version java version "1.5.0_10" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_10-b03, mixed mode) To test this through Apache, I've used RedHat's official version of Apache 2.0.52-28.ent. The MPM has been overridden in /etc/sysconfig/httpd to use the worker MPM (although this issue occurs with the default prefork MPM). ~ # httpd -V Server version: Apache/2.0.52 Server built: Aug 2 2006 05:21:18 Server's Module Magic Number: 20020903:9 Architecture: 64-bit Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D HTTPD_ROOT="/etc/httpd" -D SUEXEC_BIN="/usr/sbin/suexec" -D DEFAULT_PIDLOG="logs/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" RedHat's package includes Mod_Proxy. I built Mod_JK from source using the 1.2.23 release. Mod_JK and Tomcat are not using JNI. Apache loads Mod_JK's DSO through the /etc/httpd/conf.d/mod_jk.conf file, which itself is sourced by Apache's httpd.conf. # Load the module LoadModule jk_module modules/mod_jk.so # Location of the workers.properties file JkWorkersFile conf/jk-workers.properties # Where to put jk shared memory JkShmFile logs/jk-runtime-status # Log file location JkLogFile logs/mod_jk.log # Log level: debug, error, info JkLogLevel debug # Log file timestamp format for JkLogFile #JkLogStampFormat "[%a %b %d %H:%M:%S %Y] " # Forwarding options JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories # Environment variables passed into Tomcat from Apache #JkEnvVar SSL_CLIENT_V_START # Request log format JkRequestLogFormat "%w %V %T" The jk-workers.properties file contains an example configuration for a load balanced pair of Tomcat servers, but in this case only the first Tomcat server is added to the worker list. # This worker will be used by Apache worker.list=loadbalancer # Worker type is loadbalanced worker.loadbalancer.type=lb # Comma separated list of loadbalanced workers # By default, we only use the first worker worker.loadbalancer.balanced_workers=tomcat1 # First Tomcat server worker worker.tomcat1.port=8009 worker.tomcat1.host=localhost.localdomain worker.tomcat1.type=ajp13 worker.tomcat1.lbfactor=1 worker.tomcat1.cachesize=10 worker.tomcat1.route=localhost # Second Tomcat server worker # To enable, add "tomcat2" to the balanced_workers directive above worker.tomcat2.port=8009 worker.tomcat2.host=otherserver.localdomain worker.tomcat2.type=ajp13 worker.tomcat2.lbfactor=1 worker.tomcat2.cachesize=10 worker.tomcat2.route=otherserver # Path separator ps=/ Apache is configured to listen on port 81 and port 82 for these specific tests. Port 81 will be served via Mod_Proxy to Tomcat's HTTP1.1 connector, and port 82 will be served via Mod_JK to Tomcat's AJP connector. These configuration directives were added to /etc/httpd/conf/httpd.conf: # Test cases # Port 81 will be served via mod_proxy Listen 81 NameVirtualHost *:81 <VirtualHost *:81> ProxyRequests off ProxyPreserveHost On ProxyPass / http://localhost:8080/ ProxyPassReverse / http://localhost:8080/ </Virtualhost> # Port 82 will be served via mod_jk Listen 82 NameVirtualHost *:82 <VirtualHost *:82> JkMount / loadbalancer JkMount /* loadbalancer </VirtualHost> # Note that we'll also test with 8080 # Which will be served directly by Tomcat Test Cases ========== Test cases are done using a small webapp that generates a 3GB file filled with "A" characters. I will attach this webapp with source code. I believe a similar test can be run with Tomcat serving a static 3GB file from it's document root. I didn't test this as I will be serving files dynamically generated by a webapp (in a similar way that this webapp generates the 3GB file). Download (and header) tests were performed via curl on a remote machine (Ubuntu Feisty Fawn x86_64). Packet captures were done using Ethereal/Wireshark. ~ $ curl -V curl 7.15.5 (x86_64-pc-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5 Protocols: tftp ftp telnet dict ldap http file https ftps Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz First, I verified the Content-Length is being properly when I connected directly to Tomcat's HTTP1.1 connector in port 8080. Note that the content itself is truncated in these examples. ~ $ curl -D headers -o bigfile http://sandbox:8080/WebLoad/webload This shows the proper header length, and I'm given a time to completion: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 3192M 0 10.5M 0 0 10.8M 0 0:04:53 --:--:-- 0:04:53 11.0M Ethereal shows the following connection (by selecting Follow TCP Stream): GET /WebLoad/webload HTTP/1.1 User-Agent: curl/7.15.5 (x86_64-pc-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5 Host: sandbox:8080 Accept: */* HTTP/1.1 200 OK Server: Apache-Coyote/1.1 Content-Type: application/octet-stream Content-Length: 3347054592 Date: Tue, 29 May 2007 18:56:29 GMT AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Next, I tested by using Mod_Proxy on port 81. ~ $ curl -D headers -o bigfile http://sandbox:81/WebLoad/webload This returns the Content-Length and the progress of the download is properly reported. Ethereal shows: GET /WebLoad/webload HTTP/1.1 User-Agent: curl/7.15.5 (x86_64-pc-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5 Host: sandbox:81 Accept: */* HTTP/1.1 200 OK Date: Wed, 30 May 2007 16:43:20 GMT Server: Apache-Coyote/1.1 Content-Type: application/octet-stream Content-Length: 3347054592 Connection: close AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Finally, I test the download through the AJP connector/Mod_JK. ~ $ curl -D headers -o bigfile http://sandbox:82/WebLoad/webload GET /WebLoad/webload HTTP/1.1 User-Agent: curl/7.15.5 (x86_64-pc-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5 Host: sandbox:82 Accept: */* HTTP/1.1 200 OK Date: Tue, 29 May 2007 19:06:35 GMT Server: Apache/2.0.52 (Red Hat) Connection: close Transfer-Encoding: chunked Content-Type: application/octet-stream 1ff8 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Note that there is no Content-Length header, and the connection is forced to use chunked encoding (with a chunk size prefixing the data). Apache is not getting a Content-Length header, as show in the entries from the Mod_JK debug log. [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_get_endpoint::jk_ajp_common.c (2343): acquired connection pool slot=0 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_marshal_into_msgb::jk_ajp_common.c (548): ajp marshaling done [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_service::jk_ajp_common.c (1796): processing tomcat1 with 2 retries [Wed May 30 17:03:05 2007] [17422:6496] [debug] jk_open_socket::jk_connect.c (335): socket TCP_NODELAY set to On [Wed May 30 17:03:05 2007] [17422:6496] [debug] jk_open_socket::jk_connect.c (433): trying to connect socket 14 to 127.0.0.1:8009 [Wed May 30 17:03:05 2007] [17422:6496] [debug] jk_open_socket::jk_connect.c (459): socket 14 connected to 127.0.0.1:8009 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connect_to_endpoint::jk_ajp_common.c (850): Connected socket 14 to (127.0.0.1:8009) [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): sending to ajp13 pos=4 len=201 max=8192 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0000 12 34 00 C5 02 02 00 08 48 54 54 50 2F 31 2E 31 - .4......HTTP/1.1 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0010 00 00 10 2F 57 65 62 4C 6F 61 64 2F 77 65 62 6C - .../WebLoad/webl [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0020 6F 61 64 00 00 0A 31 30 2E 38 2E 36 2E 31 33 36 - oad...10.8.6.136 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0030 00 FF FF 00 07 73 61 6E 64 62 6F 78 00 00 52 00 - .....sandbox..R. [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0040 00 04 A0 0E 00 57 63 75 72 6C 2F 37 2E 31 35 2E - .....Wcurl/7.15. [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0050 35 20 28 78 38 36 5F 36 34 2D 70 63 2D 6C 69 6E - 5.(x86_64-pc-lin [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0060 75 78 2D 67 6E 75 29 20 6C 69 62 63 75 72 6C 2F - ux-gnu).libcurl/ [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0070 37 2E 31 35 2E 35 20 4F 70 65 6E 53 53 4C 2F 30 - 7.15.5.OpenSSL/0 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0080 2E 39 2E 38 63 20 7A 6C 69 62 2F 31 2E 32 2E 33 - .9.8c.zlib/1.2.3 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 0090 20 6C 69 62 69 64 6E 2F 30 2E 36 2E 35 00 A0 0B - .libidn/0.6.5... [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 00a0 00 0A 73 61 6E 64 62 6F 78 3A 38 32 00 A0 01 00 - ..sandbox:82.... [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 00b0 03 2A 2F 2A 00 A0 08 00 01 30 00 06 00 09 6C 6F - .*/*.....0....lo [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (896): 00c0 63 61 6C 68 6F 73 74 00 FF 00 00 00 00 00 00 00 - calhost......... [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_send_request::jk_ajp_common.c (1287): (tomcat1) request body to send 0 - request body to resend 0 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): received from ajp13 pos=0 len=52 max=8192 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0000 04 00 C8 00 02 4F 4B 00 00 01 00 0C 43 6F 6E 74 - .....OK.....Cont [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0010 65 6E 74 2D 54 79 70 65 00 00 18 61 70 70 6C 69 - ent-Type...appli [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0020 63 61 74 69 6F 6E 2F 6F 63 74 65 74 2D 73 74 72 - cation/octet-str [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0030 65 61 6D 00 00 00 00 00 00 00 00 00 00 00 00 00 - eam............. [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_unmarshal_response::jk_ajp_common.c (603): status = 200 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_unmarshal_response::jk_ajp_common.c (610): Number of headers is = 1 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_unmarshal_response::jk_ajp_common.c (666): Header[0] [Content-Type] = [application/octet-stream] [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): received from ajp13 pos=0 len=8188 max=8192 [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0000 03 1F F8 41 41 41 41 41 41 41 41 41 41 41 41 41 - ...AAAAAAAAAAAAA [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0010 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 - AAAAAAAAAAAAAAAA [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0020 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 - AAAAAAAAAAAAAAAA [Wed May 30 17:03:05 2007] [17422:6496] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1043): 0030 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 - AAAAAAAAAAAAAAAA If I run Ethereal on the Tomcat server itself and capture the AJP packets, I see the same information. Here is the AJP packet from Apache: 4......HTTP/1.1.../WebLoad/webload.. 10.8.6.136.....sandbox..R......Wcurl/7.15.5 (x86_64-pc-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8c zlib/1.2.3 libidn/0.6.5.... sandbox:82.....*/*.....0....localhost.. And from Tomcat: AB.4.....OK.....Content-Type...application/octet-stream.AB.....AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Again, no Content-Length is being set. Notes ===== Mod_Proxy will not work for the way I am using Tomcat to serve data, as Mod_Proxy strips the end user's client IP when connecting to Tomcat. In this example, all end users downloading these files would appear to be connecting from 127.0.0.1. Mod_JK solves this issue by including the end user's IP in the headers, so webapps can access this and use it for managing security access, etc. I'm filing this bug under Platform:PC and OS:Linux because that's what I've tested, but I believe this bug may affect other platforms as well. -- Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
