06 Sep 2010

On _designs undocumented

With the aim of understanding CouchDB as an application platform (and not just a database), I've been exploring some of the more recent features, as well as a few older ones that I'd previously neglected. I've found experimenting with these at turns both exciting and frustrating. CouchDB continues to lack thorough documentation, and although the reasons behind most API choices are blindingly obvious once you understand the context of their implementation, getting there is a steepening learning curve.

In the interest of sharing my findings and hopefully saving someone else (or perhaps my future-self) some time digging through source code and mailing-lists, I'm posting a few notes here for reference.

CouchDB JavaScript reference

The following functions are available in the scope of JavaScript executed by CouchDB. Some of them only make sense in certain contexts (emit is only used in map functions), and some are not available in specific circumstances (you cannot used require in views).

emit
Pushes a key-value pair onto the map function's results array.
sum
Adds together the properties of an object and returns the result.
log
Outputs an object to the couchdb logs (useful when running couchdb directly in your terminal).
toJSON
Calls JSON.stringify.
JSON
The JSON object provided by json2.js with two methods: JSON.stringify and JSON.parse.
provides
A global helper function for responding with the appropriate mime-type, depending on the request's format parameter or accept headers. See this section in Formatting with show and list on the CouchDB wiki for more information. Alternatively, see the test for example usage.
registerType
Registers a new type for use with the provides function. See this section in Formatting with show and list on the CouchDB wiki for more information.
start
Sets the response headers to send from a list function. These are sent on the first call to getRow. Therefore you cannot call start after getRow , even if you only send data after getting all rows.
send
Sets a chunk of response data to be sent on the next call to getRow , or at the end of the list functions execution.
getRow
Returns the next row from the view or null if there are no more rows. On the first call to get row, the headers set by start() are sent, on subsequent calls the data chunks set by send() are sent.
require
Returns an evaluated CommonJS module. Not available in view functions. See the section on CommonJS modules below for more information.

CommonJS Modules

It is possible to require a CommonJS style module stored on a design document using the require function. The path to the function uses '/' instead of '.' to reference properties. For example, the following path would reference a module stored on the design document at lib.mymodule:

var mymodule = require('lib/mymodule');

You can use require inside CommonJS modules as well as validate, show, list and update functions, but not in view functions (this is so that changes in view functions can be detected, and the view rebuilt). Require paths are relative to the current module, but not relative if called inside a list, show, validate or update function. In these cases the path to the module should be relative to the design document.

Rewrites

Firstly, rewrites cannot be absolute:

{from: '/_session', to: '/_session'}  // incorrect!
{from: '/_session', to: '../../../_session'}  // correct

By default, rewrites are also restricted to the database level (no more than two '..'), unless you set secure rewrites to false in your default.ini. This is more about fullfilling an admin's expectations of same-origin policy, that databases on different domains will not affect each other, than providing any user-level security. Currently (v1.0.1), this means that if you want special handlers (e.g. sessions) available to couchapps using rewrites, you have to turn off this setting and proxy requests as in the above example.

This will also stop you from rewriting URLs to other databases (useful if you have one database per-user for example). If you're aware of the risks and the database only hosts one app (comprising of multiple databases) you can probably switch safe rewrites off. Note, that you can access `_session` from the root level of the current URL regardless of rewrites, so it will work normally at `/_session` when not behind a virtual host, but will also be available at `/_session` on the virtual host too. This happens automatically.

You can use parameters and 'splats' in rewrite urls and queries, but not inside query strings. For example, the following rewrites will work:

{from: '/:name', to: '_show/test/:name'}   // OK!
{from: '/test/:val', to: '_view/test', query: {key: ':val'}}   // OK!

However, key's must be valid JSON strings, meaning you must use URLs in the form: test\"mykey\" - which looks wrong. Unfortunately, you cannot fix this by embedding the :val part in quotes:

{from: '/test/:val', to: '_view/test', query: {key: '\":val\"'}} // FAIL!

In this case, :val is not replaced with the matching part of the URL and the key parameter will literally equal \":val\". In order to get around this, you can use an array as the key for your view:

{from: '/test/:val', to: '_view/test', query: {
    startkey: [':val'], endkey: [':val', {}]
}}

In this case, the :val part will be replaced with the value from the URL. See: http://www.mail-archive.com/dev@couchdb.apache.org/msg06923.html.

Closing comments

A few of these points are documented on the CouchDB wiki or on blog posts such as this, but finding out what I don't know has been difficult in itself. I believe much of this stems from the reliance on global functions, which enter your code via a poorly documented sandbox. Where normally you would require the functions from a module, or have them passed to your function, or used as the functions execution context ('this' in the case of JavaScript), in CouchDB they are just there.

Despite my concerns regarding an increasingly steep learning curve, I'm really excited about the capabilities of CouchDB, and couchapps in particular. Unfortunately, I don't think couchapps have seen as much interest and activity as I would have expected, and I intend to spend some time exploring the possibilities and evangelising about them soon!