Tuesday, January 11, 2011

CouchDB in the middle?

I have been discussing JavaScript applications with my friend Bryan and naturally we ended up agreeing that Node.js is a technology that simply can't be ignored. I think we are both going to end up playing around with it and deciding where it fits in with how we want to write web applications.

Node.js (Express.js is a simple web framework for it) seems like a nice replacement for say... Python or Java for your middle layer. So:

That would mean you still need some random implementation of a b-tree in a file so your application can remember things. JavaScript on the front end, JavaScript in the middle, why not JavaScript in the backend. This seems perfectly reasonable:

Ah CouchDB, the JavaScript friendly database. TO THE GOOGLE!

I came across this quite nice presentation from Js Conf discussing node.js and couchdb. It has a title you can't refuse: node.js + couchdb = Crazy Delicious. A formula only rivaled by Red Vines and Mr. Pibb. Anyway, this video hit me with this little number:

That's right node.js as your backend and your database as your middle. *starts twitching* Now it seems couchdb is sort of designed for this scenario with a create concurrent connection story, the ability to render html templates, and some kind of built in user security/session model. Also couchdb only speaks HTTP through a REST api. Quite the nice fit for interfacing with web clients it seems, but there are some bumps in this road. Couchdb is a bit of a sandbox.

It seems this model works by node.js listening on the _changes api from couchdb so that it can perform things that require sending e-mails, integrating with external services, manipulating multiple documents; things like that. How intriquing and unconventional.

This _changes api seems to allow for polling for updates since a previous one, long polling to get an single update when it happens, or a continuous streaming of updates as they happen. You can also implement filters for these so you get what you want. Pretty sexy sounding actually.

Node.js as a backend must mean a couple of things (I think). First thing that comes to mind is you could design a system that triggers events off of it's logging. So you log that the user wants to send an e-mail, then node.js picks that up, sends it, updates the document as sent. I'm not sure if that is a good design. It's interesting to think you log what you want to do, then it happens. The client just does a poll for that update and notifies the user it's complete.

Second thing that comes to mind is that you have to design your system around being asynchronous. The client code would write/update the documents, node.js would pick it up, perform the action, update the database. To my knowledge you couldn't do that synchronously. This is probably an advantage that you are forced into this. 

Anyone think of some strange scenarios to show disadvantages of this scenario? In the "enterprise" world we seem to be moving towards message passing and message queuing... basically an asynchronous world.

The simple idea of CouchDB with a Node.js backend hit me pretty good. I want to explore this idea some more and hope to write a little project. Maybe a custom single social posting app thing. Post it once in my database then node.js picks it up and posts it to my Facebook, Twitter, maybe something else. Then have node.js pick up replies/comments and post them back into the database for a single social global view.


  1. I figured something about relationships and relational joins would be in someone's mind. Well I found this blog post that does a nice job going through the options including the more idiomatic couch way of using complex keys:


  2. You're coming up with cool stuff faster than we can explore it all in the dojo. We may need to run parallel projects.

  3. Well perhaps my technology enthusiasm is becoming destructive.

    I still like the idea of using Ruby and RSpec to explore TDD though, and Rails would probably be a good fit for a lot of things.

    Oh another interesting thing about couchdb is the idea of couchapps. Where couchdb can host everything and replicate down locally so the user has their data in an offline scenario. They even have couchdb for android and working on one for iOS.

  4. Just came across this Http client library that the author uses to connect to CouchDB, et al: http://devlicio.us/blogs/hadi_hariri/archive/2011/01/16/easyhttp.aspx

    Looks pretty bad ass for setting up a CouchDB ReSTful service, connecting via .NET or JavaScript or whatever, etc. I definitely need to check this out.

  5. Looks pretty cool. At some point I should read the original REST paper.

    Reading that 'exposing rest to my wife' makes me feel like web services as I know them are not needed and not very web like.

  6. I felt the same way after reading it as well. We already know that WebForms were a lie. Was SOAP a lie as well? I remember when I first worked with web services in .NET and thought, "Cool, as long as it exposes this WSDL than anything anywhere can use it. Sounds pretty open." But lately I've been wanting to pull away from that stack even more. I've spent the past week working on ideas for a "unified service theory" at work, generally leading myself towards the idea of using ReST/JSON at the core and putting a .NET layer over it for the .NET stuff. But it looks like ServiceStack did it for me.