So today, I decided to have a look at NoSQL. It’s not exactly new and actually I’m a bit late to jump on the NoSQL train, but so far I had no need for it (and actually I still don’t, but I had some time to spare and a blog to write). Since NoSQL can be quite complicated, as it imposes a new way of thinking about storing data, and I can’t possibly discuss everything there is to discuss, I’ll add some additional reading at the end of the article.
An Overview of NoSQL
First things first, what is NoSQL? As the name implies, it’s not SQL (Structured Query Language), a standard for databases to support the relational database model. As SQL has been the standard for about twenty to thirty years, I’m not going to discuss it, you probably know it. A common misunderstanding with NoSQL is that it stands for “no SQL”, while it actually means “Not Only SQL”, which implies there is at least some SQL-y goodness to be had in NoSQL as well. Whatever that SQL-y goodness may be, it’s not the relational model. And this is where NoSQL is fundamentally different from SQL, expect de-normalized and duplicated data. This ‘feature’ makes it possible to make schemas flexible though. In NoSQL, it’s generally easy to add fields to your database. Where in a SQL database, you would possibly lock a table for minutes if it contains a bit of data, in NoSQL you can add fields on the fly (during production!). Querying data can also go faster than your typical SQL database, because of the de-normalization you reduce or even eliminate expensive joins. A downside to this method of storing data is that it is harder to get consistency in your data. Where in SQL, consistency is more or less guaranteed if you have normalized your database, NoSQL offers consistency or eventual consistency. How NoSQL databases provide this (eventual) consistency differs per vendor, but it doesn’t come as natural as in SQL databases. Also, because of the way in which data is stored and queried, NoSQL databases tend to scale better across machines than SQL databases.
Other than that, no uniform definition can be given for NoSQL because there is no standard. Still, NoSQL can be roughly divided into four database models (some would say more, let’s not get into such details): Document, Graph, Key-value and Wide Column. So let’s get a quick overview of those and try one out!
The Document Model
First, there’s the Document model. When thinking of a document, don’t think of a Word or Excel document, think of an object like you would have in an object-oriënted language such as Java or C#. Each document has fields containing a value such as a string, a date, another document or an array of values. The schema of a document is dynamic and as such, it’s a breeze to add new fields. Documents can be queried on any field.
Because a value can be another document or array of documents, data access is simplified and it reduces or even eliminates the use for joins, like you would need in a relational database. It also means you will need to de-normalize and store redundant data though!
Document model databases can be used in a variety of applications. The model is flexible and documents have rich query capabilities. Additionally, the document structure closely resembles objects in modern programming languages.
Some examples of Document databases are MongoDB and CouchDB.
The Graph Model
Next there’s the Graph model. This model, like its name implies, stores data in graphs, with nodes, edges and properties to represent the data. A graph is a mathematical structure and I won’t won’t go into it any further. Graph databases model data as networks of relationships between entities. Sounds difficult? I think so too. Anyway, when your application is based on various relationships, such as social networks, the graph database is the way to go.
Some examples of Graph databases are HyperGraphDB and Neo4j.
The Key-value Model
Key-value databases are the simplest of the NoSQL databases. They basically provide a key and a value, where the value can be anything. Data can be queried by key only. Each key can have a different (type of) value. Because of this simplicity, these databases tend to be highly performant and scalable, however, because of this simplicity, they’re also not applicable to many applications.
Some examples of Key-value databases are Redis and Riak.
The Wide Column Model
Last is the Wide Column model. Like the Key-value model, the Wide Column model consists of a key on which data can be queried, can be highly performant and isn’t for each application. Each key holds a ‘single’ value that can have a variable number of columns. Each column can nest other columns. Columns can be grouped into a family and each column can be part of multiple column families. Like the Object model, the schema of a Wide Column store is flexible. Phew, and I thought the Graph model was complicated!
Some examples of Wide Column databases are Cassandra and HBase.
Getting Started with MongoDB
So anyway, there you have it. I must admit I haven’t actually used any of them, but I’m certainly planning to get into them a bit deeper. And actually, as promised, I’m going to try one out right now! I’ve picked MongoDB, one of the fastest growing databases of the moment. It’s a Document store and so has a wider applicability than the other types. You can download the free version at www.mongodb.org. There’s also a lot of documentation on there, so I recommend you look around a bit later. Installation is pretty straightforward. Just click next a few times and install. If you change any settings, I won’t be held responsible if it doesn’t work or if you can’t follow the rest of this post. So go ahead, I’ll wait.
Ready? Once you have installed MongoDB, you’ll need to run it. I was a bit surprised it doesn’t run as a service (like, for example, SQL Server) by default.
So how do you start MongoDB? Open up a command window (yes, really). First, you need to create the data directory where MongoDB stores its files. The default is \data\db, to create it, type md \data\db in your command window. Next, you need to navigate to the folder where you’ve installed MongoDB. For me, this was C:\Program Files\MongoDB 2.6 Standard\bin. Then start mongod.exe. If, like me, you’ve never had to work with a command window, here’s what you need to type in your command window:
cd C:\
md data\db
cd C:\Program Files\MongoDB 2.6 Standard\bin
mongod.exe
If you still encounter problems or you’re not running Windows, you can check this Install MongoDB tutorial. It also explains how to run MongoDB as a service, so recommended reading material there!
You might be wondering if MongoDB has a Management System where we can query and edit data without the need of a programming language. You can use the command window to issue JavaScript commands to your MongoDB database. To do this, you’ll need to start mongo.exe through a command window. The Getting Started with MongoDB page explains this in greater detail. However, I would HIGHLY RECOMMEND that you download MongoVUE instead. It’s an easy to use, graphical, management system for MongoDB. Do yourself a favour and install it before you read any further. You can check out the data we’ll be inserting and editing in the next paragraphs.
One more thing before we continue. Mongo stores its documents as BSON, which stands for Binary JSON. It’s not really relevant right now, but it’s good to know. We’ll see some classes named Bson*, now you know where it comes from. MongoVUE lets you see your stored documents in JSON format.
The C# Side of MongoDB
So now that we are running MongoDB, start up a new C# Console project in Visual Studio. Make sure you have saved your project (just call it MongoDBTest or something). Now open up the Package Manager Console, which can be found in the menu under Tools -> Library Package Manager -> Package Manager Console. Getting MongoDB to work in your project is as simple as entering the following command: PM> Install-Package mongocsharpdriver. The MongoDB drivers will be installed and added to your project automatically. Make sure you import the following namespaces to your file:
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
using MongoDB.Driver.Builders;
using MongoDB.Driver.Linq;
using System;
using System.Linq;
So, are you ready to write some code? First, we’ll need something we want to store in our database, let’s say a Person. I’ve created the following class to work with when we start.
public class Person
{
public ObjectId Id { get; set; }
public string Name { get; set; }
}
Classes don’t come easier. Notice I’ve used the ObjectId for the Id field. Using this type for an ID field makes Mongo generate an ID for you. You can use any type as an ID field, but you’ll need to set it to a unique value yourself (or you’ll overwrite the record that already has that ID). Another gotcha is that you need to call your ID field Id (case-sensitive) or annotate it with the BsonIdAttribute. And since we’re talking about Attributes, here’s another one that’ll come in handy soon, the BsonIgnoreAttribute. Properties with that Attribute won’t be persisted to the store.
public class Person
{
[BsonId()]
public ObjectId MyID { get; set; }
public string Name { get; set; }
[BsonIgnore()]
public string NotPersisted { get; set; }
}
For now, we’ll work with the default Id field. So now let’s make a connection to our instance and create a database. This is actually rather easy as you’ll see. Mongo creates a database automatically whenever you put some data in it. After we got a connection to our database, we’ll want to put some data in that database. More specific, we want to create a Person and store it. To do this, we’ll first ask for a collection of Persons with a specific name (a table name, if you like). You can store multiple collections of Persons if you use different names for the collections, so beware of typos! After we got a collection from the database, we’ll create a Person and save it to the database. That’s a lot of stuff all at once, but actually the code is so simple, you’ll get it anyway!
string connectionString = "mongodb://localhost";
MongoClient client = new MongoClient(connectionString);
MongoServer server = client.GetServer();
MongoDatabase database = server.GetDatabase("testdb");
MongoCollection persons = database.GetCollection("person");
Person p1 = new Person() { Name = "Sander" };
persons.Save(p1);
Console.WriteLine(p1.Id.ToString());
Console.ReadKey();
Wow, that was pretty easy, wasn’t it!? Mongo generated an ID for you, as you can see. Next, we’re going to get this Person back from our database. There’s a few ways to do this. We can work using the Mongo API or we can use LINQ. Both present multiple methods of querying for one or multiple records. I suggest you read the documentation and experiment a bit. I’ll already show you a couple of methods to get our Person back from the database.
ObjectId id = p1.Id;
Person sanderById = persons.FindOneById(id);
Person sanderByName = persons.FindOne(Query.EQ(p => p.Name, "Sander"));
var sandersByLinq = from p in persons.AsQueryable()
where p.Name == "Sander"
select p;
Person sander = sandersByLinq.SingleOrDefault();
You’ll notice the Query.EQ. EQ stands for equal and builds a query that tests if a field is equal to a specific value. There are other query types like GT (Greater Than), LT (Less Than), In, Exists, etc.
But wait, I’m not happy with this code at all! What Person really needs are LastName and Age fields. Now here comes this flexible schema I’ve been telling you about. Simply add the properties to your class. If you’ll fetch a Person that doesn’t have these fields specified, they’ll be set to a default value. In case of Age, you might want to use an int? rather than an int, or your already existing Persons will have an age of 0 rather than null.
Person incompleteSander = persons.FindOne(Query.EQ(p => p.Name, "Sander"));
Console.WriteLine(String.Format("{0}'s last name is {1} and {0}'s age is {2}",
incompleteSander.Name, incompleteSander.LastName, incompleteSander.Age.ToString()));
incompleteSander.LastName = "Rossel";
incompleteSander.Age = 27;
persons.Save(incompleteSander);
Console.ReadKey();
Person completeSander = persons.FindOne(Query.EQ(p => p.Name, "Sander"));
Console.WriteLine(String.Format("{0}'s last name is {1} and {0}'s age is {2}",
completeSander.Name, completeSander.LastName, completeSander.Age.ToString()));
Console.ReadKey();
Now let’s also add an address to Person. Address will be a new class and Person will hold a reference to an Address. Now you can just model this like you always would.
public class Person
{
public ObjectId Id { get; set; }
public string Name { get; set; }
public string LastName { get; set; }
public int? Age { get; set; }
public Address Address { get; set; }
}
public class Address
{
public string AddressLine { get; set; }
public string PostalCode { get; set; }
}
Notice that Address doesn’t need an Id field? That’s because it’s a sub-document of Person, it doesn’t exist without a Person and as such doesn’t need an Id to make it unique. Now fetch your already existing Person from the database, check that its address is empty, create an address, save it and fetch it again.
Person addresslessSander = persons.FindOne(Query.EQ(p => p.Name, "Sander"));
if (addresslessSander.Address != null)
{
Console.WriteLine(String.Format("Sander lives at {0} on postal code {1}",
addresslessSander.Address.AddressLine, addresslessSander.Address.PostalCode));
}
else
{
Console.WriteLine("Sander lives nowhere...");
}
addresslessSander.Address = new Address() { AddressLine = "Somewhere", PostalCode = "1234 AB" };
persons.Save(addresslessSander);
Person addressSander = persons.FindOne(Query.EQ(p => p.Name, "Sander"));
if (addressSander.Address != null)
{
Console.WriteLine(String.Format("Sander lives at {0} on postal code {1}",
addressSander.Address.AddressLine, addressSander.Address.PostalCode));
}
else
{
Console.WriteLine("Sander lives nowhere...");
}
Console.ReadKey();
Make sure you check out the JSON in MongoVUE. Also try experimenting with Lists of classes. Try adding more Addresses, for example. We haven’t deleted or updated any records either, we’ve only overwritten entire entries. Experiment and read the documentation.
We’ve now scratched the surface of NoSQL and MongoDB in particular. Of course, MongoDB has a lot more to offer, but I hope this post has helped get your feet wet in NoSQL and MongoDB. Perhaps it has given you that little push you needed to get started. It has for me. Expect more NoSQL blogs in the future!
Additional Reading
As promised, here’s some additional reading:
Comments are welcome. Happy coding!
The post A first look at NoSQL and MongoDB in particular appeared first on Sander's bits.
