Erk, HTML email, thats not even done right! *smacks your client* :)
Uhm, start by looking at the header logging feature (mime_${SOMETHING} in
squid.conf) to log
request and reply headers.
I'd then use those client requests to create test cases to feed to Squid.
Adrian
On Fri, Mar 14, 2008, Saul Waizer wrote:
> <HTML>
> <P>Adrian,</P>
> <P>QUERY ACL has been removed for over a week, can you give me some pointers
> as far as looking at the MISSes</P>
> <P>Thanks!<br>
>
> <br>
>
> <br>
>
> <br>
>
> <B>On Fri , Adrian Chadd <[EMAIL PROTECTED]>sent:<br>
>
> </P></B>
> <BLOCKQUOTE style="PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
> BORDER-LEFT: #f5f5f5 2px solid; MARGIN-RIGHT: 0px">Caching dynamic content
> doesn't work "like that".<br>
>
> <br>
>
> Firstly, removing the QUERY ACL gives you the ability to cache dynamic<br>
>
> content that returned explicit lifetime.<br>
>
> <br>
>
> You need to look at all of those MISSes and see why Squid isn't caching<br>
>
> them. Its hard to tell from where I'm sitting.<br>
>
> <br>
>
> <br>
>
> <br>
>
> <br>
>
> Adrian<br>
>
> <br>
>
> <br>
>
> On Fri, Mar 14, 2008, Saul Waizer wrote:<br>
>
> <SPAN style="COLOR: #ff0000">> -----BEGIN PGP SIGNED
> MESSAGE-----</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Hash: SHA1</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Amos,</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> I've implemented the example you sent on
> Dynamic Content but so far i</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> regret to say that no improvement has been
> made on the hit ratio</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> I added the following to my
> squid.conf</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> refresh_pattern (/cgi-bin/|\?) 0 0%
> 0</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> refresh_pattern . 0 20% 4320</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> acl mydomain dstdomain
> .mydomain.com</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> cache allow mydomain</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> my stats look something like
> this:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> 67.5103% TCP_MISS/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> 6.07349% TCP_HIT/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> 4.55681% TCP_MEM_HIT/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> 1.59761% TCP_IMS_HIT/304</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Any help is appreciated.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Thanks</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Amos Jeffries wrote:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Adrian Chadd wrote:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >> G'day,</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >> Just remove the QUERY ACL and the
> cache ACL line using "QUERY" in it.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >> Then turn on header logging
> (log_mime_hdrs on) and see if the replies</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >> to the dynamically generated
> content is actually giving caching info.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >> Adrian</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > <A
> href="http://mail.hoodiny.com/parse.php?redirect=http%3A%2F%2Fwiki.squid-cache.org%2FConfigExamples%2FDynamicContent";
> target=_blank><SPAN style="COLOR:
> #ff0000">http://wiki.squid-cache.org/ConfigExamples/DynamicContent</SPAN></A></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Amos</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >> On Fri, Feb 29, 2008, Saul Waizer
> wrote:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Hello List,</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > I am having problems trying to cache
> images*/content that comes from a</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > URL containing a question mark on it
> ('?')</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Background:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > I am running squid Version
> 2.6.STABLE17 on FreeBSD 6.2 as a reverse</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > proxy to accelerate content hosted in
> America served in Europe.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > The content comes from an application
> that uses TOMCAT so a URL</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > requesting dynamic content would look
> similar to this:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > <A
> href="http://mail.hoodiny.com/parse.php?redirect=http%3A%2F%2Fdomain.com%2Fstorage%2Fstorage%3FfileName%3D%2F.domain.com-1%2Fusr%2F14348%2Fimage%2Fthumbnail%2Fth_8837728e67eb9cce6fa074df7619cd0d193_1_.jpg";
> target=_blank><SPAN style="COLOR:
> #ff0000">http://domain.com/storage/storage?fileName=/.domain.com-1/usr/14348/image/thumbnail/th_8837728e67eb9cce6fa074df7619cd0d193_1_.jpg</SPAN></A></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > The result of such request always
> results on a MISS with a log similar</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > to this:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > TCP_MISS/200 8728 GET <A
> href="http://mail.hoodiny.com/parse.php?redirect=http%3A%2F%2Fdomain.com%2Fstorage%2Fstorage%3F";
> target=_blank><SPAN style="COLOR:
> #ff0000">http://domain.com/storage/storage?</SPAN></A> -</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > FIRST_UP_PARENT/server_1
> image/jpg</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > I've added this to my config: acl
> QUERY urlpath_regex cgi-bin as you can</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > see bellow but it makes no difference
> and I tried adding this:</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl QUERY urlpath_regex cgi-bin \? and
> for some reason ALL requests</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > result in a MISS.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Any help is greatly
> appreciated.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > My squid config looks like this:
> (obviously real ip's were changed)</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > ################# STANDARD ACL'S
> ###############################</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl all src 0.0.0.0/0.0.0.0</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl manager proto
> cache_object</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl localhost src
> 127.0.0.1/255.255.255.255</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl to_localhost dst
> 127.0.0.0/8</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > ################# REVERSE CONFIG FOR
> SITE #####</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > http_port 80 accel vhost</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_peer 1.1.1.1 parent 80 0
> no-query originserver name=server_1</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl sites_server_1 dstdomain
> domain.com</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > ################# REVERSE ACL'S FOR
> OUR DOMAINS ##############</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl ourdomain0 dstdomain <A
> href="http://mail.hoodiny.com/parse.php?redirect=http://www.domain.com";
> target=_blank>www.domain.com</A></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl ourdomain1 dstdomain
> domain.com</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > http_access allow ourdomain0</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > http_access allow ourdomain1</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > http_access deny all</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > icp_access allow all</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > ################ HEADER CONTROL
> ###############</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > visible_hostname
> cacheA.domain.com</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_effective_user nobody</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > forwarded_for on</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > follow_x_forwarded_for allow
> all</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > header_access All allow all</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > ############### SNMP CONTROL
> ###############</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > snmp_port 161</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl snmppublic snmp_community
> public1</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > snmp_access allow all</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > ############## CACHE CONTROL
> ################</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > access_log
> /usr/local/squid/var/logs/access.log squid</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > acl QUERY urlpath_regex
> cgi-bin</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_mem 1280 MB</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_swap_low 95</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_swap_high 98</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > maximum_object_size 6144 KB</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > minimum_object_size 1 KB</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > maximum_object_size_in_memory 4096
> KB</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_dir ufs /storage/ram_dir1 128 16
> 256</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_dir ufs /storage/cache_dir1 5120
> 16 256</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_dir ufs /storage/cache_dir2 5120
> 16 256</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > cache_dir ufs /storage/cache_dir3 5120
> 16 256</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Also here is the result of a custom
> script I made to parse the</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > access.log that will sort and display
> the top 22 responses so I can</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > compare them with cacti, I am trying
> to increase the Hit ratio but so</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > far is extremely low.</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 1 571121 69.3643%
> TCP_MISS/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 2 98432 11.9549% TCP_HIT/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 3 51590 6.26576%
> TCP_MEM_HIT/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 4 47009 5.70938%
> TCP_MISS/304</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 5 17757 2.15664%
> TCP_IMS_HIT/304</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 6 11982 1.45525%
> TCP_REFRESH_HIT/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 7 11801 1.43327%
> TCP_MISS/404</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 8 6810 0.827095%
> TCP_MISS/500</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 9 2508 0.304604%
> TCP_MISS/000</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 10 1323 0.160682%
> TCP_MISS/301</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 11 1151 0.139792%
> TCP_MISS/403</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 12 1051 0.127647%
> TCP_REFRESH_HIT/304</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 13 430 0.0522248%
> TCP_REFRESH_MISS/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 14 127 0.0154245%
> TCP_CLIENT_REFRESH_MISS/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 15 83 0.0100806%
> TCP_MISS/401</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 16 81 0.00983769%
> TCP_CLIENT_REFRESH_MISS/304</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 17 35 0.00425085%
> TCP_MISS/503</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 18 20 0.00242906%
> TCP_DENIED/400</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 19 19 0.00230761%
> TCP_HIT/000</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 20 19 0.00230761%
> TCP_DENIED/403</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 21 14 0.00170034%
> TCP_SWAPFAIL_MISS/200</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > 22 1 0.000121453%
> TCP_SWAPFAIL_MISS/30</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > Thanks!</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> > </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> >></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> -----BEGIN PGP SIGNATURE-----</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Version: GnuPG v1.4.6 (GNU/Linux)</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> Comment: Using GnuPG with Mozilla - <A
> href="http://mail.hoodiny.com/parse.php?redirect=http%3A%2F%2Fenigmail.mozdev.org";
> target=_blank><SPAN style="COLOR:
> #ff0000">http://enigmail.mozdev.org</SPAN></A></SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> </SPAN><br>
>
> <SPAN style="COLOR: #ff0000">>
> iD8DBQFH2ugEAcr37anguZsRAixSAJ9GioRmL42D1bOSVveMKYcpi21fygCfd4VP</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> UMqi1CB3vQ5IeaTblK/vYQM=</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> =6XUg</SPAN><br>
>
> <SPAN style="COLOR: #ff0000">> -----END PGP SIGNATURE-----</SPAN><br>
>
> <br>
>
> -- <br>
>
> - Xenion - <A href="parse.php?redirect=http%3A%2F%2Fwww.xenion.com.au%2F"
> target=_blank><SPAN style="COLOR:
> #ff0000">http://www.xenion.com.au/</SPAN></A> - VPS Hosting - Commercial
> Squid Support -<br>
>
> - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -<br>
>
> <br>
>
> </BLOCKQUOTE><BR></HTML>
--
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
![http://mail.hoodiny.com/parse.php?redirect=http%3A%2F%2Fdomain.com%2Fstorage%2Fstorage%3FfileName%3D%2F.domain.com-1%2Fusr%2F14348%2Fimage%2Fthumbnail%2Fth_8837728e67eb9cce6fa074df7619cd0d193_1_.jpg"](http://mail.hoodiny.com/parse.php?redirect=http%3A%2F%2Fdomain.com%2Fstorage%2Fstorage%3FfileName%3D%2F.domain.com-1%2Fusr%2F14348%2Fimage%2Fthumbnail%2Fth_8837728e67eb9cce6fa074df7619cd0d193_1_.jpg"){kind=link}
