Greetings,
This e-mail shows you how to modify the squid source to support your own custom
http header lines in 3 easy steps. Feel free to ignore if you don't care =)
I recently modified my local version of the squid 3.1.9 source so it would play
nice with some of our proprietary tools. Basically, I had to modify the URL
that was getting included in the MD5 used as the key for cache storage/lookup.
(see my e-mail with subject "Caching Content from Dynamic URLs (very hacky
solution)" for code details)
Because I modified the URL, I ran into some collisions, where the "relative
url" for two different files was the same, meaning the cache "key" was the
same. Essentially, the cache would think a requested file was already cached,
but the cached file did not match the requested one.
So I needed a way to make the MD5 used for each file more unique without using
the full url (since the full url in our case is super dynamic and ends up being
TOO unique). I decided on a custom header line in our tool that sends the http
request (since our tool knows the expected last modified time of the file it's
requesting). Here's what the http request header looks like:
GET http://*****/foo.txt HTTP/1.0
Connection: Keep-Alive
Expected-DateTime: 1292611588
As you can see, the custom header line is "Expected-DateTime" followed by an
int.
Then, I made the following changes to the squid source:
1. In HttpHeader.h, add a new value to the http_hdr_type enum.
The one I added was called HDR_EXPECTED_DATETIME
2. In HttpHeader.cc, I made two changes.
The first was in the HeadersAttrs array. I had add the name of the
header, the enum, and the data type, as follows:
{"Expected-DateTime", HDR_EXPECTED_DATETIME, ftStr}
I left it a string, but it could have been an int as well.
The second change was to add the enum to the RequestHeadersArr array.
static http_hdr_type RequestHeadersArr[] = {
HDR_AUTHORIZATION, HDR_FROM, HDR_HOST,
HDR_IF_MATCH, HDR_IF_MODIFIED_SINCE, HDR_IF_NONE_MATCH,
HDR_IF_RANGE, HDR_MAX_FORWARDS, HDR_PROXY_CONNECTION,
HDR_PROXY_AUTHORIZATION, HDR_RANGE, HDR_REFERER,
HDR_REQUEST_RANGE,
HDR_USER_AGENT, HDR_X_FORWARDED_FOR, HDR_SURROGATE_CAPABILITY,
HDR_EXPECTED_DATETIME
};
Note that it goes in this array because I'm supporting the new header
line in http requests only.
If you want it supported in replies as well, you should probably put it
in the GeneralHeadersArr array.
3. In store_key_md5.cc, we need to make sure the new header line gets used when
calculating the cache key.
In storeKeyPublicByRequestMethod, after "SquidMD5Update(&M, (unsigned
char *) relativeUrl, strlen(relativeUrl));", I added the following:
// Add the last modified time from the header into the cache
key
// This is to avoid 2 files with the same relativeUrl from
stomping eachother
if (request->header.has(HDR_EXPECTED_DATETIME))
{
const char *expectedDateTime =
request->header.getStr(HDR_EXPECTED_DATETIME);
SquidMD5Update(&M, (unsigned char *) expectedDateTime,
strlen(expectedDateTime));
}
As you can see, I'm checking if the header has an entry of type
"HDR_EXPECTED_DATETIME". If it does, I get the value, and I update the MD5 used
as the cache key.
That's all there is to it!
Let me know if you have any questions.
Thanks for your time,
-Adam
