B-Tree, A Native of the Cluster

Background

Decades ago, when we implemented an application that needed persistence, we embedded B-Tree library. Then RDBMS was invented. Now, we code SQL scripts to do data management, submitted & run through the RDBMS.

Then ORM (Object Relational Mappers) got invented. We are now able to use SQL & ORM API/middleware to do data persistence.

In B-Tree embedded, you have an application that can manage data on a file. In RDBMS, you have a server endpoint and a client API like ODBC to manage data. It is client/server setup. In ORM, you have a somewhat "Objects" technology. You code/model data in Objects then ORM mw maps it to SQL that gets submitted to the backend RDBMS/DBMS/NoSql for execution.

But the RDBMS/DBMS/NoSql is still an endpoint. That is, you have one monolith Database Server App serving your data management.

In Scaleable Objects Persistence(SOP), we turn the Application or Microservice cluster into the database server itself. We broke down the monolith into a "micro service" friendly form, a code library that turns your Application or Microservice into the database server itself. Hot spot free, horizontally scaleable cluster.

Illustration

Now that all of that is out of the way, I would like to show you a sample code, to show how easy it is to manage data in SOP.

t1, _ := in_red_ck.NewTransaction(true, -1)
t1.Begin()
b3, _ := in_red_ck.NewBtree[int, string](ctx, "twophase", 8, false, true, true, "", t1)

// *** Add a single item then commit so we persist "root node".
b3.Add(ctx, 500, "I am the value with 500 key.")
t1.Commit(ctx)
// ***

eg, ctx2 := errgroup.WithContext(ctx)

f1 := func() error {
	t1, _ := in_red_ck.NewTransaction(true, -1)
	t1.Begin()
	b3, _ := in_red_ck.OpenBtree[int, string](ctx2, "twophase", t1)
	b3.Add(ctx2, 5000, "I am the value with 5000 key.")
	b3.Add(ctx2, 5001, "I am the value with 5001 key.")
	b3.Add(ctx2, 5002, "I am the value with 5002 key.")
	return t1.Commit(ctx2)
}

f2 := func() error {
	t2, _ := in_red_ck.NewTransaction(true, -1)
	t2.Begin()
	b32, _ := in_red_ck.OpenBtree[int, string](ctx2, "twophase", t2)
	b32.Add(ctx2, 5500, "I am the value with 5500 key.")
	b32.Add(ctx2, 5501, "I am the value with 5501 key.")
	b32.Add(ctx2, 5502, "I am the value with 5502 key.")
	return t2.Commit(ctx2)
}

eg.Go(f1)
eg.Go(f2)

if err := eg.Wait(); err != nil {
	t.Error(err)
	return
}

The above code spins off two go-routines that each manage data in a transaction. Code has no resource locking, 'just plain straight usage of the SOP library's API to create a B-Tree store and to add items to it, then commit to finalize & submit the changes.

Simple, just like most APIs you've seen before. Nothing fancy, but wait a minute! That code right there spins off two go-routines to add items in two different transaction sessions and their added items are automatically reconciled and merged by SOP.

The outcome is a combination of the records added across different transactions. Imagine, there is no resource locking, just plain vanilla transaction, mgmt action then commit.

Imagine, you have many of that kind of code that you are running across your hosts in Kubernetes, or in Amazon EC2.

Imagine, all you have to bother is to define your Objects structure (key & value) and use the simple, basic API of SOP. That is it. And each of these BTree "store" is an index. That is, you can do high speed searching and "sorted" range queries/updates. And you can have "a lot" of them without slowing down your Cassandra & Redis cluster.

Plain and simple. :)

It is here, right now. SOP V2 that supports that kind of "Objects based easy coding" but produces highly scaleable & ACID "treated" data flows, follow this link for more details: https://github.com/SharedCode/sop

History

• 31^st January, 2024: Initial version