Manual CPSing in the database

I'm currently implementing text notes for NoDictionaries. You enter a really insightful comment, you need to log in to post your comment, so you log in and your comment is there.

A pretty convenient way of going about is to have a hash table of closures, and store the continuation of the commit, continuation-passing style.

However, it'd be better to persist data in the database. (What happens if you restart the server?)

Here's how committing a text note works in NoDictionaries: when you POST /add-note, you basically make the note , except the associated text word's id is the negative of the correct id, and the user id is whatever user id you have, whether a real database row or just the time in seconds of your first page view. Then you're redirected to GET /commit-note, which makes sure you're logged in (and takes you on a round-trip to the login/signup if necessary), verifies the id of the note, the id of your user id, and the negativity of the text word id. You can spoof it if you know the database id of an uncommited note, the database id of the word it's associated with, and the permanent or temporary user id that the person submitted it has, so you'd basically have to snoop on their wifi for that split second when they're filling out the login/signup form.

It still feels like a kludge.

DB indices

I think it'd be pretty nifty to get a daily summary of which indices would have been much handier to have: for example, snapshot my DB at 4am, record the day's GETs/POSTs/PUTs/DELETEs, index the hell out of my tables (creating 2^n indices for each combination of columns), give me statistics based on speed of completing a day's work, the IO and RAM required, and let me tweak the parameters, and then upgrade my box or add indices or whatever.

(This is poorly thought out, but interesting nonetheless.)

Ruby's memcache-client 1.7.2 does NOT support compression; patch it in with 7 lines of code

The Ruby memcache-client 1.7.2 code seems like the most popular memcached client. Alas, memcache-client 1.7.2 does not recognize :compress => true, all the demos out there notwithstanding.

Let's fix this by monkeypatching Zlib compression into Marshal.

require 'zlib'

module Marshal
  @@load_uc = method :load
  @@dump_uc = method :dump
  def self.load(v) @@load_uc[Zlib::Inflate.inflate(v)] end
  def self.dump(v) Zlib::Deflate.deflate(@@dump_uc[v]) end
end

And there we go!  Four lines of patching Marshal.load for a better memory footprint.

To dissect each phrase of that sentence:

1. "Four lines": I wanted just to try how well zlib would work on my Marshalled ActiveRecord objects and html fragments, and it did so handily, almost 3:1.  Indeed, the only reason I poked around at the source code is because one of my largest but still highly-compressible HTML fragments was 1.2MB, over the size limit.  I've since gone back to storing large HTML fragments on disk (uncompressed), having found many more values to store in Memcached.
2. "patching Marshal.load": monkeypatching Marshal is not as bad as String.  Chances are, you use the Marshal format as a blob, and you keep your Marshal files to yourself (and leave external serialization to friendlier fare like JSON).  So, all in all, it's much easier to change the Marshal format than mucking through the memcache-client code.
3. "better memory footprint": instead of Zlib, try LZMA, with slightly smaller compressed sizes than BZIP and faster decompression times, good properties for cache compression.  But Zlib is already in the standard library, so it's a good first approximation.
   

The ersatz alias_method_chaining feels kludgy, as does Ruby's distinction between methods and lambdae.  Ah well.

Thoughts?

Reality

When art critics get together they talk about Form and Structure and Meaning. When artists get together they talk about where you can buy cheap turpentine. -- Picasso

A Full Web Service with HTTP caching in 7 lines

Two lovely gems, sinatra and rack-cache. Sinatra is pretty easy web-service-creation, and Rack::Cache is pretty easy http caching. Together? Jubilation.

require 'rubygems'
require 'sinatra'
require 'rack/cache'

use Rack::Cache

get('/quadruple/:n') {
  sleep 1
  response.headers['Cache-Control'] = 'max-age=1000000'
  (params[:n].to_i * 4).to_s
}

and then

ruby sinatra-add.rb -e production

and you're done.

There are, of course, many other fiddly bits to configure Rack::Cache with, like use Rack::Cache, :entitystore => 'file:/tmp/' if you don't want to keep it all in a hash in memory, and :verbose => false if you don't want that in your logs, but that's basically it.

It's pretty amazing: that's a real live web service, with HTTP caching, in 7 gentle lines. (I don't count the closing brace, nor the sleep 1, which is purely for effect.)

Anyone know an easier way in any other language? Either in number of lines, or in directness of code?

Also, how bad would it be to authbind that to port 80, and just let it go open to the world?



Update: It's interesting to me for the same reason PHP is interesting, both as a social commentary on the bits of plumbing we've agreed upon as useful, and as an aid to wrapping an HTTP interface around some Ada code with its own homegrown database.

Top five contradictory positions of Ron Paul Graham.

1. Supports secure borders, but writes macros for variable capture.
2. Rejects conspiracy theories of 9/11, but funded a social news website full of 9/11 conspiracy theories.
3. Will launch an ad blimp in two weeks, but remains cynical about submarine-style PR campaigns.
4. Says he doesn't organize his supporters on the internet, but provides a Q&A internet forum.
5. Opposes inflation, but supports accumulator generators.

Leave more in the comments...

Anti-Features?, or, Never attribute to malice what can be explained by math.

Apparently, if you buy a cheap camera, and you can't save an uncompressed picture, that's treacherous computing, that's an anti-feature.

It's not. The camera won't stop working if you correctly patch in a different chip set.

And assuming the worst, assuming that the chips are exactly the same in an expensive camera, it still doesn't matter. The native resolution is still small on the cheap camera--that's mostly what makes it cheap--so you'll be better off buying the high-resolution expensive camera.

Of course, if you really want a camera that you can program, you might like an XO that drives a Roomba.