Join our free session on how to model your data in JSON on April 29th
When starting to learn about document databases, it’s nice to have a grip on JSON data modeling best practices. This will help you navigate the space and topics easier having an expanded vocabulary on the subject. Experience with a traditional relational database is also a good start. Knowledge about structuring data, querying, and indexes will give you a headstart. Some concepts will transfer over, especially when learning Couchbase which utilizes a SQL variant (N1QL) for querying. Many of the terms you already know from SQL will be applicable, but differ in that they apply to key-value document data storage and nested documents instead of tables and rows.
Why JSON for Data
Our first set of resources will be helpful in brushing up on your understanding of JavaScript Object Notation.
A Brief History of JSON
With the popularity of AJAX in the early 2000s, the term JSON was coined by Douglas Crockford. The first two links talk about a period of time before JSON when it was being used but didn’t have a name, up to the time period of Single Page Applications. The second link is a video from the last Silicon Valley Code Camp where I had the opportunity to attend Doug’s talk in 2019. His presentation on “The History of JSON” is a great way for us to begin our journey learning more about JSON data modeling.
With a better understanding of its history and vocabulary, we can move on to actually working with JSON. The MDN link below will answer many questions on structure, obtaining JSON with XMLHttpRequest, and testing your own skills. You can test your understanding of the article at the end with a challenge which I think is really fun. When you get done, you will be much more prepared to use JSON.
With JavaScript Object Notation basics out of the way, we are prepared to start our journey into document modeling.
JSON Data Modeling
In this category, we first talk about JSON Data Modeling in general. In the upcoming sections, we will discuss strategies to migrate from relational databases.
Our first resource above is a video by Matthew Groves, outlined below are highlights from each section of his talk which covers why switching to NoSQL (in our case a document database) can be beneficial. It covers strategies for modeling and accessing this data, and also touches on migration tips. Let’s explore some of the topics in this video.
Why NoSQL
Some databases like Couchbase store their data as plain JSON, in this section Matt highlights the many reasons why to use a distributed document database. He also addresses the current NoSQL landscape, gives a brief understanding of NoSQL and its scalability, flexibility, availability, performance, and use cases.
Couchbase customer derived use cases for NoSQL
JSON Data Modeling
In this section, you are given an exercise on modeling data in JSON and provides JSON data modeling examples. This section is complete with properties of real-world data, a brief section on modeling in the relational world as a comparison, rules of thumb on strategies for modeling your data. Also discussed is third party modeling tools like Hackolade, Erwin, and Idera. These are tools geared toward NoSQL databases.
Side by side comparison of table vs JSON data
Accessing Data
This part dives into the strategies and rules of thumb for modeling and accessing your data in a NoSQL database like reading, updating, and deleting records by key. Recommendations for keys are given along with some good examples. We wrap up this section with approaches subdocuments and sub-collections, the N1QL query language found in Couchbase.
Understanding data structure and relationships is key for data access
Migrating Data
Here we get into the moving of data from one database to another with tools that will help you along the way. Tools like Informatica, Kafka, Talend, and Apache Nifi. Couchbase also has a tool called cbimport to help you import data into Couchbase.
Before investing in tools, keep it simple, use the database tools to help export, import, and transform
Couchbase Blogs on Data Modeling
Next, we have some articles from our Couchbase Blog that may pique your interest on JSON data modeling techniques:
I also like the idea of learning as I build real-world projects. My next link can help you get started understanding how to tackle registration, login, and authentication in NoSQL.
Want a walkthrough on planning a simple REST API. The resource below implements ideas from the article above using Node and Express JS. I personally just finished updating this article to utilize the latest Couchbase Node SDK version 3 and provided a GitHub repository of the source code, postman collections, and environment variables to aid in easily getting the project up and running.
And we can’t mention the two articles above without mentioning the frontend client built specifically for the REST API from the repository above using Angular 11. Also, an article which I have recently updated and provided a GitHub repository of the finished code.
Couchbase Training on Data Modeling
Let’s introduce you to our Couchbase Data Modeling resources from Couchbase Academy. Two great courses, one for beginners and another more advanced which gets your hands dirty with Couchbase taking into consideration data modeling and tuning queries for use with the N1QL query language.
The Couchbase documentation has a few really good pages that deal with data modeling and entity relationships in document data. The links below were part of the Couchbase 5x versions but contain valuable information still.
Embedded versus referenced documents
In the current version 6x of Couchbase Server, we also have a page that is evolved and updated with the most recent versions of our database in mind:
Data Modeling in Couchbase Server 7
We have entered a new era here at Couchbase as we start to move toward utilizing Scopes and Collections in Couchbase Server 7. It’s due for release in the Summer of 2021. Below are some recent Couchbase blog articles to help aid in understanding data modeling in a Scopes and Collection world and building real-world applications with them.
When upgrading from Couchbase version six to seven, documents can be assigned to a default scope and collection for immediate compatibility
The table shows the mapping of the familiar relational model constructs to Couchbase
Join our free session on how to model your data in JSON on April 29th
Migrating Data from RDBMS to NoSQL
When you are migrating data from a relational database there are many normalization constraints and schema requirements. These are required when working with tables or relating other tables through foreign keys. This type of linking using unique ids is also possible in a document database. There are other options too like nesting that data, even if it appears redundant. In this first category of resources, we will take links to courses, videos, and articles that will aid in moving from relational databases to a JSON document database.
This first resource is a video from a member of our own team Matthew Groves. He is not only well versed in Couchbase (a document datastore) but also comes from a relational database background. I like this video because somehow in less than twenty minutes he is able to take you through the basics and get you started on your journey moving from tables to documents.
We start with exporting data from a relational database, using a CSV format, use a tool called cbimport to import the data into Couchbase. Then he instructs how to transform and structure that data on the document database side and finalize our data using the N1QL query language. WIth N1QL, we can move data from a staging point to a final data bucket ready for use. This video is chock full of real code examples allowing you to follow along and try yourself.
Matt has also written an article on the same topic if you prefer scanning a blog article over taking the time to watch the entire video.
You can also check out Model Your Relational Database Data as NoSQL Document Data again with another video that has many good points and walkthroughs. It will help you think about how to model existing relational data to document data.
Another resource to help you understand the differences from relational data as well as understanding how Couchbase works come from a well-respected member of the Couchbase team David Segleau. He’s a thirty-plus year veteran of the database community.
Our Blogs here on Couchbase are another great resource so I have handpicked several articles that deal specifically with moving from relational systems over to Couchbase, this information is also highly applicable to moving from any relational database like SQL Server to a JSON document database like Couchbase.
External Data Modeling Resources
Couchbase is one of the best places on the web to learn about data modeling for NoSQL, document databases, or key-value datastores. But it’s good to venture outside because there are many more great resources for you out there. There is a wonderful course on Pluralsight that walks you through creating data models in JSON and addresses the difference between modeling relational vs document data.
Pluralsight Course: Create Data Models in JSON
In this slide from the Pluralsight video, Kishan Iyer talks about one of the many different database technologies he covers in his course
Pluralsight is an amazing resource for application developers and also has many Couchbase Server related courses as well.
This article from the highly scalable blog about NoSQL Data Modeling Techniques is informative and funny and is loaded with more external resources.
Migrating from MongoDB to Couchbase
The most common document database we experience our customers switching from is Mongo. Some come for the scalability, others for the query language. If you find yourself needing to get data from Mongo into Couchbase, these resources are for you!
A few blog posts in our Couchbase Blog do a great job of covering this topic. The first two links are articles that are new and contain up to date information on making the switch from MongoDB to Couchbase.
We hope that the resources provided in this article go a long way in helping you hit the ground running on your path to learning about document databases and Couchbase. We have tried to build a learning resource that you can bookmark and revisit. If you feel we have missed any important resources, comment and mention those resources as we are always looking to add to this guide and ensure everyone benefits from having a single point of access for JSON data modeling resources.
Join our free session on how to model your data in JSON on April 29th
