Let's fix this by monkeypatching Zlib compression into Marshal.
@@load_uc = method :load
@@dump_uc = method :dump
def self.load(v) @@load_uc[Zlib::Inflate.inflate(v)] end
def self.dump(v) Zlib::Deflate.deflate(@@dump_uc[v]) end
And there we go! Four lines of patching Marshal.load for a better memory footprint.
To dissect each phrase of that sentence:
- "Four lines": I wanted just to try how well zlib would work on my Marshalled ActiveRecord objects and html fragments, and it did so handily, almost 3:1. Indeed, the only reason I poked around at the source code is because one of my largest but still highly-compressible HTML fragments was 1.2MB, over the size limit. I've since gone back to storing large HTML fragments on disk (uncompressed), having found many more values to store in Memcached.
- "patching Marshal.load": monkeypatching Marshal is not as bad as String. Chances are, you use the Marshal format as a blob, and you keep your Marshal files to yourself (and leave external serialization to friendlier fare like JSON). So, all in all, it's much easier to change the Marshal format than mucking through the memcache-client code.
- "better memory footprint": instead of Zlib, try LZMA, with slightly smaller compressed sizes than BZIP and faster decompression times, good properties for cache compression. But Zlib is already in the standard library, so it's a good first approximation.