Designing a document databaseRemote API & Public API
One of the greatest challenges that we face when we try to design API is balancing power, flexibility, complexity, extensibility and version tolerance. There is a set of tradeoffs that we have to make in order to make sure that we design an API, and selecting the right tradeoffs impacts the way that users will work with the software.
One of the decisions that I made about the docs db is that it would be actually have two published API. The first would be the remote API, accessible for clients, and the second would be a public API, accessible for people who want to extend the db. I consider extending the db to be a very common operation, for doing things like adding more functions for the views, supplying the sharding function, etc. I am assuming that most people who would want to do this extension would be .Net devs, so I am just going to use my usual style here and expose some extension points so it will be possible to mix & match things.
For the remote API, we need a way to add / remove documents, get a document and query a view. I would like that part of the API to be accessible from any language, even if I don’t foresee it getting used much in other languages. The idea of being able to access it from JavaScript, and making the entire application hosted through the DB is actually a very interesting idea, see CouchApp for more details. Making it accessible for everyone is a pretty good idea, and I think that adopting’s a REST API similar to the one that CouchDB is using would be a good choice. Something that I would be extremely interested on, as a matter of fact, would be to make use of Astoria’s REST API, instead of having to write my own implementation. I think it bears some investigating, since on the minus side there is the problem that Astoria expects some sort of schema from the backend implementation, but it would be an interesting avenue for investigating.
Currently I am thinking about an API similar to:
- POST or PUT to /docs[/id] – to create/update a document
- POST to /bulk_docs – to create/update a batch of documents
- POST or PUT to /views – to create / update a view definition
- GET from /docs/id to get a document
- GET from /views/name – to get all view docs of particular view
- GET from /views/name?key=foo – to get all view docs with matching key
- GET from /views/name?start=foo&end=bar – to get all view docs within range
This seems to be a pretty simple proposition, and it seems pretty complete.
Thoughts?
More posts in "Designing a document database" series:
- (17 Mar 2009) What next?
- (16 Mar 2009) Remote API & Public API
- (16 Mar 2009) Looking at views
- (15 Mar 2009) View syntax
- (14 Mar 2009) Aggregation Recalculating
- (13 Mar 2009) Aggregation
- (12 Mar 2009) Views
- (11 Mar 2009) Replication
- (11 Mar 2009) Attachments
- (10 Mar 2009) Authorization
- (10 Mar 2009) Concurrency
- (10 Mar 2009) Scale
- (10 Mar 2009) Storage
Comments
I see.. some REST like API. Do you care if you get the correlated ID back from a POST to /bulk_docs? If so, then do you assume a reply in the same order, or allows the client to specify the ID, or return the whole post content with ID's added?
Josh,
Probably just same order response of the ids.
How about delete? Would you use an HTTP DELETE or a POST to a delete url?
Configurator,
Yes, delete should work
Hmm, where is the representational state transfer in that api ? :)
Bunter,
documents are resources
uniquely addressable by their id
There are cachable, stateless and layered
Ayende, I'm sure you have read it: roy.gbiv.com/.../rest-apis-must-be-hypertext-dr...
I know, so much semantic buzz is around rest it makes people unrest, but I think the main concept (i.e. browsable API with single entry point) of representational state transfer missies many.
Addressing schemes to consider:
JsonQuery: [ www.sitepen.com/.../jsonquery-data-querying-bey... ]
Astoria: [ msdn.microsoft.com/en-us/library/cc668766.aspx ]
Google APIs (all of them more or less revolving around the same format): [ code.google.com/p/pyrfeed/wiki/GoogleReaderAPI ]
Implementations of document dbs:
SimpleDB
NSimpleDB
CoachDB
Persevere
Miscelanea :-)
spot a mistake regarding Astoria in Dare's article: [ www.25hoursaday.com/.../...InARESTfulProtocol.aspx ]
yours truly is dreaming about exposing NH repository via universal web service: [ n2.codeplex.com/Thread/View.aspx?ThreadId=28810 ]
Probably just same order response of the ids.
Or you could simply use OpenRasta...
That said, there is no such thing as an addressing scheme or predefined URIs in ReST. If you doucment what APIs to document with, that's a ReST smell. :)
Comment preview