Global Shared Constructs

When dealing with a distributed system, anything that must be centrally controlled and mutually agreed upon as "correct" (or more aptly, consistent) is an facet of the system design that is necessarily complex.

The following describes what shared constructs must be maintained across the O1 system, and why:
  • Property Type - centralizing property type allows a small surrogate key to be chosen for storage (2 bytes). Defining property types is a design time activity and expected to be low volume. A shared nothing alternative is to hash property types which will result in at least a 16 byte representation, which will significantly increase the size of cells and required RAM.
  • Node List - all active nodes must be under centralized control in some manner. Resolution of RAM Trunk assignment can be done via a global map. Consistent hashing techniques do not apply here as the trunk to node assignment is absolute, and individual "keys" are not intended to be evenly distributed.
  • Node Heartbeat - all active nodes must indicate reach-ability in the graph cluster. This requires some designation of a "head node", and consideration for head node resiliency.
  • Trunk ID - all RAM Trunks are identified by a 2 byte value. These assignments must be globally resolvable and the trunk ID counter state must be globally consistent when adding a new RAM Trunk. This is expected to be an infrequent event that requires absolute consistency for a small window of time.
  • Trunk Local ID - all RAM trunks must maintain a local counter that is a 4 byte non-negative integer value. This counter is strictly scoped to the RAM Trunk and is not global across the whole system, but must be consistent within a given RAM trunk.
  • Global Indexes - to resolve cells based on externally identifiable properties, the system must provide a centralized index capability. This capability influences the need for high consistency between indexed data and graph data. For example, if a property is removed or modified, the corresponding index must be updated accordingly.
  • Transaction Counter - the transaction counter that sets the next transaction ID must be centralized and durable, or at least guaranteed to increase over time.
  • Last Commit Watermark - the last committed transaction watermark must be known across an entire O1 cluster.

raw design notes
Storage
CAP
Cell Format
Query Processing

Last edited Sep 28, 2014 at 10:45 PM by bkjuice, version 10