The Object Network

A Decentralised, Declarative Object Notification Network

The Object Network is a network of chunks of updating data called "NetObjects".

NetObjects can be linked up into a global mesh through their UIDs, which are like URLs.

A NetObject can observe another NetObject that it links to, and be notified when that one changes.

It can then update its own state, which may in turn notify that state on to any observing peers.

NetObjects can be hosted on many peer devices and exchange updates over the LAN or WAN.

Quick intro to NetObjects for techies

The NetObject programming model can be used throughout your system. You can create a couple of NetObjects like this:

  net_object* purchase_obj=net_object_new(  |  net_object* total_obj=net_object_new(
    "{ UID: 123-123                         |    "{ UID: 789-789
       is: purchase                         |       is: totaller
       price: 1.25                          |       purchase: 123-123
       quantity: 1                          |     }", eval_total);
     }", eval_purchase);                    |

The second object links to the first object through the 'purchase' property. The 'link' is just the purchase object's UID, not a direct in-memory link.

There are callbacks for each type of object. This callback is called at various points in the lifecycle to allow the object to set its state. It's called on creation, for example. Inside the callback, you can access the properties of the object:

  void eval_total(net_object* total_obj){
    char* purchase_uid=net_object_get(total_obj, "purchase");  // "123-123"
  }

Further - and here is the core concept of NetObjects - you can equally well directly access the properties of a linked object:

  void eval_total(net_object* total_obj){
    double price   =net_object_get_d(total_obj, "purchase:price");        // 1.25
    int    quantity=net_object_get_i(total_obj, "purchase:quantity");     // 1
    ;               net_object_set_d(total_obj, "total", price*quantity); // 1.25
  }

Here, you are "looking right through" the purchase UID link into the target object in a single step.

Doing this also sets up an observation - you get a callback when the target is changed:

  :
    net_object_set_i(purchase_obj, "quantity", 3);
  :

This change triggers the callbacks for this and any other observers, allowing them to update in turn. Hence, the total object will always maintain its dependency on the purchase object:

  void eval_total(net_object* total_obj){
    double price   =net_object_get_d(total_obj, "purchase:price");        // 1.25
    int    quantity=net_object_get_i(total_obj, "purchase:quantity");     // 3
    ;               net_object_set_d(total_obj, "total", price*quantity); // 3.75
  }

Here's what the update results in:

   { UID: 123-123        { UID: 789-789
     is: purchase          is: totaller
     price: 1.25    =>     purchase: 123-123
     quantity: 3           total: 3.75
   }                     }

That's basically it - NetObjects link to each other and observe each other's state. And they can even do this across machines.

You can program all the static and dynamic aspects of your system this way.

Why that's such a powerful programming model

This simple mechanism, when used as your base programming model, has many powerful consequences.

Easy to program: all interactions, whether local or remote, can be programmed the same way, in the style of a spreadsheet. This is a high-level programming model which allows you to forget about network interactions, cacheing and cache updates, proxying, data persistence, threading, locking, etc., all of which are all handled under the NetObject interface. Instead, you focus on writing pure application logic, user interfaces, data models, adaptors to other systems and other non-net I/O. You construct the inter-object links and write simple local data dependencies that 'internally animate' the NetObjects. The system's global behaviour then emerges without central control. There are no application boundaries in this approach - just small, self-animated NetObjects that can be mashed up and re-mashed.

Great visibility, interoperability, mashability: if you use NetObjects everywhere, you end up with a 'data fabric' of linked up NetObjects. A single path string can describe a journey through this data fabric, jumping links and servers as it goes. If you view them in a GUI, you can then just jump around your data estate from server to server, peer to peer, exploring how everything interlinks. You'd be able to watch those NetObjects live, as they update. Such a client would render the NetObject mesh onwards from one NetObject as nested sub-blocks, which may be opened or expanded to see more, then jumped in to, etc. Such interlinked and interdependent data will encourage the evolution to stable or common data types. This will then permit many applications to be mashed up out of collections of linked-up data rendered by a generic client that is programmed to understand and allow interaction with those common types. And if a type still isn't recognised by this client, it's just a NetObject dependency mapping away from a type that is.

Fast: NetObjects are naturally distributed amongst peer hosts to form a fine-grained mesh. Such an architecture makes it easy to scale up through distributed and parallel processing and fine-grained cacheing and network optimisation. Each NetObject is inherently self-animated and wrap all the threading and locking code, making massively parallel execution as simple as adding more hosts or processors. This also maps nicely to a microservice approach. And small chunks of data distributed to caches and having their changes pushed can make more efficient use of the network than larger chunks being sent repeatedly in hand-coded messages or being polled for changes. You can also benefit from the classic efficiency characteristics of P2P architectures by taking cache copies from nearer neighbours. Startup time is reduced if a NetObject can get going with the last known cached state of its dependencies. This fine-grained tuning of small NetObjects with different cache lifetimes allows you to build more efficient web pages with a mix and mash of static and dynamic elements.

Robust: NetObjects are highly tolerant of their peers unavailability - small patches of the data fabric can drop out through network or host failure without affecting the rest; everything is done on the basis of the best knowledge of the current peer state - accessed through the latest local cache. NetObjects are autonomous and interact directly, peer-to-peer, so a single individual NetObject can fail while all its peers carry on, or be upgraded without stopping a whole application, since there are no application boundaries.

Notes on the NetObject programming model

Four Elements

The Object Network technology is built from four elements:

The ONF layer offers up an interface - the NetObject interface - to the layer formed by the ONR and ONT elements. The ONP layer sits under the ONF.

Functional Observer

The ONF layer's NetObject interface implements a programming model called Functional Observer.

An object can link to a peer object through a string containing its UID. Objects - or their animating programs - can only see and react to peer objects that they can reach through these links. Further, an object can never update another object directly.

These two constraints underly the basic Object Network programming model or interaction and animation model - which is called "Functional Observer". This model says:

"An object's new state is a function of its current state including the state of peer objects observed through links".

The observed object may be more than one hop of links away.

To non-programmers, the Object Network probably all seems pretty obvious. To programmers, it's radical and perhaps heretical - particularly in its appreciation of evolving state. Dynamic state is the basis of the Object Network, unlike in the Object-Oriented and Functional Programming models, which hide or deprecate it.

In the Object Network, we make all state public, but read only. An object is never writable by others, only by the object itself on a single thread with a degree of transactionality that allows the state to be read while it's being updated in the background. This should go some way to easing the objections to updating state that OO and FP people may hold.

ONF and ONP

The Object Network is designed to be a global web of structured data items linked by UIDs and interacting with each other directly over the internet via their evolving public state.

Each object can be rendered as a ONF object and each is given a UID to identify and locate it. Objects are exchanged between peers over the network through ONP, which has cache update built in. They are addressed by IPv6 and interact in a peer-to-peer model using UDP.

Prototype and C implementation

There's currently an Android app and Java server prototype of all this, called NetMash. This code includes an implementation of the old approach to formats (i.e., including JSON alongside a cleaner notation) and protocol (i.e., based only on HTTP) described in the FOREST ROA material.

However the current production implementation - Onex - is written in C, which is better for portability, for limited performance IoT devices, for rule language speed and for 3D performance. It will only use the cleaner ONF notation and simpler ONP protocol.

It'll be coded first for Android, Linux and Cortex M-class devices such as the BBC Micro:Bit. All of the NetObject interface implementation stack - ONF and ONP - and the Onex programming language - ONR - will be implemented to run on such limited devices from the first line of code cut, with the ultimate goal of becoming an operating system for all devices.

Read more

NetObjects compared with your favourite trendy tech

IoT: you can build a physical fabric of interactive and interacting NetObjects representing Things.

AR/VR: NetObjects create a global landscape of linked up interactive data, including users, which is actually an immersive cyberspace.

P2P: NetObjects themselves work P2P so their hosts do too; there are no clients or servers in a NetObject architecture.

Functional: you write your dependency-satisfying callback as a pure transition function from old state to new state.

Reactive: objects are interdependent, with automatic dependency handling.

Information-Centric Networking: NetObjects have the same goal of a higher level abstraction for the low-level substrate of data for all applications.

Microservices: NetObjects naturally partition into collections that can be hosted together, and, being autonomous, are easy to re-partition.

CQRS: you create write-side 'intentions' through NetObjects that map to CQRS's commands. The read-side NetObjects then depend on them.

REST: NetObjects have global UIDs to fetch their current state. They add a 'GET' that can continue to return updates to clients and their caches.

Dynamic web pages: compose static and dynamic NetObjects into sections of a page.

Interactive web pages: there's a NetObject representing the user and their actions/forms which have a two-way inter-dependency with 'server' state.

Declarative web pages: many applications can be composed from a mesh of NetObjects of common, stable types, driving a generic UI.

Mobile: a generic NetObject UI can render NetObjects like a browser renders HTML, etc, but allows an experience that's closer to native.

Duncan Cragg, 2016

Contact me and/or subscribe to my blog and/or follow me on Twitter.