FOREST is one of the layers making up The Object Network.
Part of FOREST is the definition of a would-be application/forest+json
Media Type. This page describes that.
Functional Observer
Functional Observer describes a linked object graph structure - the Object Network. The Objects are essentially JSON structures: lists, maps, strings, etc. Here's an example in JSON:
{ "b": ["c", "x", 123, "y"], "d": true }Single elements and single-element lists are not distinguished, and string numbers and booleans aren't distinguished from native numbers and booleans. Also, null elements disappear.
All Object Network objects have UID object references which will usually contain a GUID, a UUID or some other random-looking string after a "
uid-
" prefix. Links are just UID or URL strings as values after a property tag and can appear anywhere in an object. Links are transparent: so any object linked to can be merged in-line instead:{ "b": "c", "f": "uid-222" }where uid-222 is:
{ "a": "x", "b": [ "e", 123 ] }is the same as:
{ "b": "c", "f": { "a": "x", "b": [ "e", 123 ] } }Objects (and sub-objects) can link to "further details" using
More
:
{ "b": "c", "More": "uid-222" }where uid-222 is:
{ "a": "x", "b": [ "e", 123 ] }is the same as:
{ "a": "x", "b": [ "c", "e", 123 ] }The elements at 'More:' are merged in set-wise: duplicates are removed.
Links to list objects are also seen as the same as inline lists. For example:
{ .. "email": "uid-111", .. }where uid-111 is:
{ "is": "list", "list": [ "a@b.com", "c@d.com" ] }is the same as:
{ .. "email": [ "a@b.com", "c@d.com" ] .. }This allows lists to be grown outside of their containing object, or reduced back in again. An object can switch at any time between these forms: single-to-list-to-single, link-to-inline-to-link.
There are no nulls or empty lists in the Object Network, or rather, null and empty list are equivalent to non-existent. So:
{ "a": null, "b": [ null, "c" ] }is the same as:
{ "b": "c" }These objects interact by mutual observation. The Functional Observer interaction model is simply:
Set an object's state as a Function of its current state plus the state of other objects it Observes through links.
This observation occurs through either pull or push of linked object state.
Such a programming model is declarative in nature, and thus very expressive, as well as being naturally concurrent.
FOREST
FOREST is a distributed object architecture. FOREST is an acronym for Functional Observer REST: it maps directly to RESTful distribution over HTTP, using GET for pull and POST for push of object state, in both directions between interacting peer servers. POST is effectively an idempotent GET response or cache update. Even though HTTP is client-server and doesn't have notification, FOREST implements a symmetric peer-to-peer protocol over HTTP's GET and POST. Each server can act as client; GET is used to pull data, POST to notify updates to that data. PUT, DELETE, etc aren't used. HEAD may be used.
Objects are published into a global interacting object network by turning their UIDs into URLs. A link can be a full URL starting "
http://
", or just a UID starting "uid-
". UIDs can be converted to full URLs by wrapping them appropriately with prefix and postfix - e.g.uid-0fe6-7d5d
can be wrapped to give:http://object.network/events/uid-0fe6-7d5d.json
. Object Network URLs are considered opaque to clients. The use of the.json
extension is optional but advised for human readability, editing and saving of files.The only response codes in FOREST are 200 for successful GET or POST, 204 for successful POST without response or long-poll heartbeat, 303 for redirect, 304 for Etag match, 401 for auth challenge, 403 for unsubscribing from updates, 404 object not found on that URL, 405 if POST not allowed, 50X server failures.
The only headers used to manage the cache are
Etag
andmax-age
.Etag
is an incrementing Version number. ThusIf-None-Match
is used for conditional GET.Here's a simple FOREST GET example:
GET /events/uid-f330a3-4011e.json HTTP/1.1 Host: object.network If-None-Match: "32" HTTP/1.1 200 OK Cache-Control: max-age=1800 ETag: "33" Content-Type: application/forest+json { "is": [ "attendable", "event" ], "title": "The Object Network Conference", "content": "This year is expected to be the best yet!", "start": "30 Sep 2016 09:00:00 GMT", "end": " 1 Oct 2016 23:00:00 GMT", "location": "http://bletchleypark.org/address.json", "attendees": "uid-0fe6-7d5d" }POST
In FOREST, POST is defined as a "pushed GET response". It therefore is idempotent - you can re-POST the same thing as often as you like.
The POSTed content body, being equivalent to a cache-filling GET response, can have HTTP headers set for
Content-Location
,Etag
andCache-Control: max-age
.Cache-Control
is somewhat ill-specified in this case, so may be replaced withExpires
. If there is any reason not to send these headers in the POST, they can be carried in the body, as the"URL"
,"Version"
and"Max-Age"
properties.POST normally returns
204 No Content
. It is also an "open channel" for return notifications of new state, which otherwise would have been POSTed back. In this case, the response can either be200 OK
with the content body and itsContent-Location
if it differs from the original POST target, or303
with aLocation
, or303
with content body and bothLocation
andContent-Location
.The POST response, and GET responses, can return a
Cache-Notify
header giving the URL of "the cache". This can be used to POST to it directly, allowing the state update to be propagated to all interested parties, to save an object being POSTed directly to multiple targets on the same host. Similarly, if theCache-Notify
header is set in a request header, then updates to the target will be POSTed back there.The
Cache-Notify
URL can also be used in an asymmetric long-polling setup, where a response is returned on a GET on that serverCache-Notify
URL whenever any update of interest occurs for the clientCache-Notify
given on that GET's headers. A POST can be used in the same way to long-poll, combining notifications in both directions. If there was nothing to POST in the first instance, an empty POST body can be sent, to set up the long poll channel.It is possible for a server or client that isn't visible and cannot publish its own URLs to work asymmetrically by POSTing the basic UIDs of its own data instead of their wrapped URL.
Here's a simple FOREST POST example:
POST /events/uid-f330a3-4011e.json HTTP/1.1 Host: object.network Content-Location: http://fred.info/o/uid-f330a3-4011e.json Cache-Control: max-age=1800 ETag: "1" Content-Type: application/forest+json { "is": "rsvp", "event": "http://object.network/events/uid-f330a3-4011e.json", "person": "http://..", "attending": "yes", } HTTP/1.1 200 OK Cache-Control: max-age=1800 ETag: "34" Content-Type: application/forest+json { "is": [ "attendable", "event" ], "title": "The Object Network Conference", "content": "This year is expected to be the best yet!", "start": "30 Sep 2016 09:00:00 GMT", "end": " 1 Oct 2016 23:00:00 GMT", "location": "http://bletchleypark.org/address.json", "attendees": [ "uid-0fe6-7d5d", "http://fred.info/o/uid-f330a3-4011e.json" ] }The RSVP request is a spontaneous notification - a "pushed observation" - to the target Event object. The Event will then link to this notified object if it is interested in it. It can then poll for changes, or wait for further POST updates. If not interested, or stops being interested, it would return 403.
The POST response is optional, and is sent on the assumption that the source, the RSVP, is interested in the resulting update - that the RSVP object is observing this Event, which it probably is. This could be made explicit through the use of
Cache-Notify
.
Coffee Shop Example
There is a tradition in the distributed systems or integration community to use the example of ordering coffee to demonstrate your asynchronous system integration or distribution model.
It started with Gregor Hohpe's article about asynchronous messaging, then was continued by Jim Webber as "RESTbucks" to discuss his REST interpretation in an open forum and subsequently used to describe a refined version of the approach in the book by Jim and Ian Robinson.
Here's the FOREST version. It's based on the Foreign Exchange ordering example I used in my FOREST paper, which in turn was actually a coffee ordering scenario in its first drafts.
GET /baristas HTTP/1.1 Host: coffee-shop.com HTTP/1.1 200 OK Etag: "313" Cache-Control: max-age=10 Content-Type: application/json { "is": [ "coffee", "shop", "vendor" ], "products": [ { "title": "Espresso", "More": "/prod/esp1" }, { "title": "Cappuccino", "More": "/prod/cap2" } ], "trackers": [ "http://coffee-shop.com/track110", "http://coffee-shop.com/track109" ] }Here we have an entry point URL on the
coffee-shop.com
server which returns a "vendor" object listing products and currently outstanding orders being tracked. Let's have a look at that Espresso:GET /prod/esp1 HTTP/1.1 Host: coffee-shop.com HTTP/1.1 200 OK Etag: "12" Cache-Control: max-age=3600 Content-Type: application/json { "is": [ "coffee", "product" ], "title": "Espresso", "description": "A shot of pure coffee", "options": { "shots": { "input": "textfield", "label": "Shots", "range": [ 1, "..", 5 ] } "ristretto": { "input": "checkbox", "label": "Ristretto" } } "price-text": "£1.50 single, £2.50 double" }So now we know all about it, and it allows certain options - "shots" and "ristretto" - that we can present on a user interface. Lets give our order, now:
POST /baristas HTTP/1.1 Host: coffee-shop.com Content-Type: application/json { "URL": "http://my-orders.com/ordr321", "Version": 1, "Max-Age": 10, "is": [ "coffee", "order" ], "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 1, "ristretto": false } ], "vendor": "http://coffee-shop.com/baristas" }This is hosted on
my-orders.com
. We're proactively notifying the vendor, who doesn't know about this object yet, that here's something it may be interested in. If so, it'll add the order's URL to itself and take up observing it. In this case, it's not interested in the order, but creates a delegate - a tracker object - that is observing the order. Since the vendor can guess that the order is observing it, it could return itself showing the link to the new tracker:HTTP/1.1 200 OK Etag: "314" Cache-Control: max-age=10 Content-Type: application/json { "is": [ "coffee", "shop", "vendor" ], "products": [ { "title": "Espresso", "More": "/prod/esp1" }, { "title": "Cappuccino", "More": "/prod/cap2" } ], "trackers": [ "http://coffee-shop.com/track111", "http://coffee-shop.com/track110", "http://coffee-shop.com/track109" ] }But that's pretty useless to the order. Better to be redirected to the new tracker:
HTTP/1.1 303 See Other Location: http://coffee-shop.com/track111Or better still to be returned the whole tracker - it notifies itself, again pro-actively, to the order:
HTTP/1.1 200 OK Content-Location: http://coffee-shop.com/track111 Etag: "1" Cache-Control: max-age=10 Content-Type: application/json { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 1, "ristretto": false } ], "price": "£1.50", "status": "waiting" }Or do both:
HTTP/1.1 303 See Other Location: http://coffee-shop.com/track111 Content-Location: http://coffee-shop.com/track111 Etag: "1" Cache-Control: max-age=10 Content-Type: application/json { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 1, "ristretto": false } ], "price": "£1.50", "status": "waiting" }Or return nothing (204 No Content), but POST the tracker to the order when it's good and ready:
HTTP/1.1 204 No Content : POST /ordr321 HTTP/1.1 Host: my-orders.com Content-Location: http://coffee-shop.com/track111 Etag: "1" Cache-Control: max-age=10 Content-Type: application/json { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 1, "ristretto": false } ], "price": "£1.50", "status": "waiting" } HTTP/1.1 204 No ContentOr the same but with those HTTP headers in the body:
HTTP/1.1 204 No Content : POST /ordr321 HTTP/1.1 Host: my-orders.com Content-Type: application/json { "URL": "http://coffee-shop.com/track111", "Version": 1, "Max-Age": 10, "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 1, "ristretto": false } ], "price": "£1.50", "status": "waiting" } HTTP/1.1 204 No ContentThe Cache-Control and Content-Type headers will not be shown from now on, for brevity. We'll also use the in-body version of URL, Version and Max-Age.
Now the updates to the order are notified at the tracker directly - let's add a shot:
POST /track111 HTTP/1.1 Host: coffee-shop.com { "URL": "http://my-orders.com/ordr321", "Version": 2, "Max-Age": 10, "is": [ "coffee", "order" ], "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "vendor": "http://coffee-shop.com/baristas", "tracker": "http://coffee-shop.com/track111" } HTTP/1.1 200 OK Content-Location: http://coffee-shop.com/track111 Etag: "2" { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "price": "£2.50", "status": "waiting" }The
Content-Location
is actually not necessary here. Let's check if we missed our order being fulfilled:GET /track111 HTTP/1.1 Host: coffee-shop.com If-None-Match: "2" HTTP/1.1 304 Not Modified Etag: "2"Nope. Time passes. The order gets completed:
POST /ordr321 HTTP/1.1 Host: my-orders.com { "URL": "http://coffee-shop.com/track111", "Version": 3, "Max-Age": 10, "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "price": "£2.50", "status": "filled" } HTTP/1.1 204 No ContentBut we have a race condition - the order has been updated, but it's too late to change, now:
POST /track111 HTTP/1.1 Host: coffee-shop.com { "URL": "http://my-orders.com/ordr321", "Version": 3, "Max-Age": 10, "is": [ "coffee", "order" ], "products": [ { "product": "/prod/cap2", "quantity": 1, "shots": 2 } ], "vendor": "http://coffee-shop.com/baristas", "tracker": "http://coffee-shop.com/track111" } HTTP/1.1 200 OK Content-Location: http://coffee-shop.com/track111 Etag: "4" { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "price": "£2.50", "status": [ "filled" , "not-as-ordered" ] }We have a tantrum:
POST /track111 HTTP/1.1 Host: coffee-shop.com { "URL": "http://my-orders.com/ordr321", "Version": 4, "Max-Age": 10, "is": [ "coffee", "order" ], "products": "cancelled", "vendor": "http://coffee-shop.com/baristas", "tracker": "http://coffee-shop.com/track111" } HTTP/1.1 200 OK Content-Location: http://coffee-shop.com/track111 Etag: "5" { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "price": "£0.00", "status": "cancelled" }Then, feeling thirsty, decide to take it anyway:
POST /track111 HTTP/1.1 Host: coffee-shop.com { "URL": "http://my-orders.com/ordr321", "Version": 5, "Max-Age": 10, "is": [ "coffee", "order" ], "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "vendor": "http://coffee-shop.com/baristas", "tracker": "http://coffee-shop.com/track111" } HTTP/1.1 200 OK Content-Location: http://coffee-shop.com/track111 Etag: "6" { "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "price": "£2.50", "status": "filled" }Now it's time to pay up, so create a payment object and add a link to it on the order:
POST /track111 HTTP/1.1 Host: coffee-shop.com { "URL": "http://my-orders.com/ordr321", "Version": 6, "Max-Age": 10, "is": [ "coffee", "order" ], "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "vendor": "http://coffee-shop.com/baristas", "tracker": "http://coffee-shop.com/track111", "payment": "http://my-orders.com/paym432" } HTTP/1.1 204 No ContentThe tracker observes the payment:
GET /paym432 HTTP/1.1 Host: my-orders.com HTTP/1.1 200 OK Etag: "1" { "is": "payment", "invoice": "http://coffee-shop.com/track111", "order": "http://my-orders.com/ordr321", "amount": "£2.50", "account": { .. } }And adds it, takes the payment and sets the status to "paid":
POST /ordr321 HTTP/1.1 Host: my-orders.com { "URL": "http://coffee-shop.com/track111", "Version": 7, "Max-Age": 10, "is": [ "coffee", "order-tracker" ], "order": "http://my-orders.com/ordr321", "products": [ { "product": "/prod/esp1", "quantity": 1, "shots": 2, "ristretto": false } ], "price": "£2.50", "status": "paid", "payment": "http://my-orders.com/paym432" } HTTP/1.1 204 No Content
Contact me and/or subscribe to my blog and/or follow me on Twitter.
[E225]