MongoDB (from huMONGOus) is a popular choice in corporations, because it’s built by an actual corporation that supports it, instead of being a “pure” open source project. Their website also looks a lot like it’s targeting large corporations.
In the triangle of the CAP theorem, MongoDB sits in the corner of “consistency + partition tolerance”. It has a single master node that you have to talk to to ensure consistency. But if that master node goes down, your availability is of course gone.
MongoDB has a document-based data model that looks like JSON. This means you can put data of almost any structure in a database, for example blog posts. MongoDB does not enforce a schema, so you can have different fields in every document if you want (although it’s maybe not a good idea to do so). Unlike in Cassandra, you don’t need a single unique key identifier. Instead, you can create indices on any field or combination of fields.
MongoDB uses a slightly different terminology to highlight the fact that it’s different from relational database.
Databases: Basically the same as a database in the relational sense.
Collections: A collection is the equivalent to a table in a relational database.
Documents: Since the JSON model allows you to store very unstructured data, the word “row” might not make sense here. Instead of tables consisting of rows, in MongoDB we talk of collections consisting of single documents.
MongoDB has extensions that are substitutes for some of Hadoop’s core tools. For example, MongoDB has its own file system, GridFS, and it has built-in aggregation capabilities, so that in some cases you don’t even have to use Hadoop’s MapReduce anymore. But, you can still integrate MongoDB with other tools like Spark.
The MongoDB shell
There is one problem though: The user_id is not yet declared as an index. So when you search for the user ID 100, MongoDB can’t do this efficiently, but instead does a full table scan. If you want to speed up these queries, you have to define the user_ID as an index:
The ‘1’ here just means to sort in ascending order. The difference between having an index and not can be visualized by having MongoDB explain what it will do when you execute a query. You can make it show the “winning plan” to get the results for a query like this:
This outputs a JSON document that shows you how it would proceed in finding the user with the user_ID 100. Without an index, you would find the keyword COLLSCAN (for collective scan) in the response, and with an index, it would do the more efficient IXSCAN (an index scan).
Aggregating data in the Mongo shell
As mentioned before, MongoDB has built-in aggregation functions. The syntax is slightly odd, but this is how you would group the user data by occupation, and get the average age in each group:
Integrating MongoDB with Spark
The following Python script uses Spark to import the user table from the ml-100k data set into MongoDB, and then reads out all users under 20 years of age. This is the same task, and almost the same script, as in the Cassandra case. Compare the differences, they are only a few lines.
Run it by issuing:
You’ll again see a lot of info messages, and the following table:
This list contains "mother" posts for larger topics, each spanning multiple blog posts.