Hi Thanks Dustin...
> Adding further metadata to the cache and changing the protocol to > return it isn't really an option at this point, but it's easy to add > to the data in your application. > Yeah, unless someone really needs it. It seems like this feature could be added to the current protocol easily enough just by adding an "expired" flag to the return data, and perhaps a command line switch to enable memcached to return expired data with a timestamp. > Why would you disable caching just because something's writing? > There's always a last write. Ok, first I better mention what I was referring to with "guaranteed" current data. Database programmers are fanatical about making sure the data is consistent to any reader during a whole transaction. They get into all sorts permutations with things like read committed, serialized, mvcc processing etc to handle the issue. The problem is a truly thorny one, which explains to me why databases can run so darn slowly! >From my research into concurrent transactions, things just get too hairy to try to keep track of all the possible concurrent issues when caching anything that's being updated in transactions. If it has been done, is there anyone interested in sharing how they did it, since the locking/transaction/caching issues can be hard to solve. Speaking of this, it seems that a better solution would have the database interface module include caching. It could create a huge performance boost by integrating memcached functionality into the a database interface module. Rather than having all database functionality run on a core set of machines, you could have a database pre-processor with memcached functionality running on the client machines. The pre-processor could handle all of the sql parsing, memcached server, and communication to the core database machines. If done right, the database interface wouldn't change a bit, but reads from the database could occur so much faster. To do this, the database programmers would have to be convinced that they could provide "guaranteed" accurate cached data. I've tried talking with the postgresql programmers, and they are stone cold to the idea... Perhaps Sun could get a few guys working on it for Mysql since they seem to now have an interest in both memcached and mysql, and the technique could do so much better than the current caching implemented in mysql!!! > Most of this is just tired ramblings of someone guessing at > requirements, though. Once there are particular constraints for an > application, the mechanism to ensure correctness becomes more clear. So, just in case my ramblings were not especially clear, has anyone dealt with the constraints of caching data in the middle of multiple on-going transactions in a way that is demonstrateably effective? Is there any reason why Memcached couldn't be baked into a database interface module? Thanks Daniel
