Playing with MongoDB (Reactive Mongo)

Note: Examples of this blog uses javascript and CSS. WordPress does not allow Javascript on sites hosted on WordPress.com. In the HTML code, the places where you see ‘script’ and ‘/script’ and ‘link’ in the code, enclose it with the HTML angle brackets <> and </>

You’ll see

script src="@routes.Assets.at("javascripts/jquery-1.9.0.min.js")" type="text/javascript" /script

replace with

<s ....type="text/javascript"> </s..>

Replace

link rel="stylesheet" type="text/css" href="@routes.Assets.at("stylesheets/bootstrap.min.css")"

with

 <link.. >

In this tutorial, I’ll cover using MongoDB with Play. I’ll use ReactiveMongo driver which provides non-blocking and asynchronous I/O operations. I’ll also briefly touch upon how to use mongoDB’s using its inbuilt shell (you could use applications like RoboMongo if you prefer). I’ll create a CRUD application which will create, read, update and delete user information. The example is tested on Chrome browser.

MongoDB Premier

This section covers basics of MongoDB. You may skip this section if you already are familiary with MongoDB.First, you’ll need to install MongoDB (obviously!). There are plenty of good tutorials on the internet about installing First, you’ll need to install MongoDB (obviously!). There are plenty of good tutorials on the internet about installing mongoDB. Assuming that you have installed mongoDB, first start the mongoDB daemon and connect to mongoDB either using shell or Robomongo Start daemon

• open command shell
• Go to the installation path/bin directory
• type mongod
• Start mongodb shell
• open command shell
• Go to the installation path/bin directory
• type mongo

 

A database in mongoDB is called ‘database’ (like in SQL). However, in mongoDB, the term collection refers to traditional tables in SQL. ‘row’ in SQL is called ‘document’ in mongoDB, and ‘column’ in SQL is called ‘field’. I’ll use the mongoDB terminology in rest of this tutorial. Our example will consist of a single collection (‘users’) in database (‘web-business’). Let us create the ‘users’ collection in ‘web-business’ database. Run following commands Let us start by trying out few mongoDB commands in the mongo shell

Creating a database

In mongoDB, we do not explicitly create a database i.e. there is no command to create a database. A database in created when a collection is created. ‘use’ command is used to select a database. If the database already existing, mongoDB will use it. If it doesn’t exist, mongoDB will not create it and will wait until a collection created in the database

> use web-business
switched to db web-business
>

Assuming that ‘web-business’ does not exist yet, the above commands will not create it. We can verify it by using ‘show dbs’ command which shows all existing databases

> show dbs
admin 0.000GB
local 0.000GB

To create ‘users’ collection, use createCollection command. We can check the result by using ‘show dbs’ and ‘db.getCollectionNames()’

> db.createCollection("users")
{ "ok" : 1 }
> show dbs
admin 0.000GB
local 0.000GB
web-business 0.000GB
> db.getCollectionNames()
[ "users" ]

The above step to create the ‘users’ collection is not necessary. Mongodb would have automatically created a collection at the time a document is created. Thus, following command would have created ‘web-business’ database and ‘users’ collection and would have created a document in the collection. Collections are schema-less i.e. each document can have its own set of fields. Thus, we do not need to specify a schema before creating a document. The following document contains user information like name, address, email etc. Note that in insert command, I specified an ‘id’ field and the result from find() contains id and _id. These are two completely different things. ‘id’ is my own generated thing (which I intend to use in the application), _id is a unique number mongoDB generates to uniquely reference each document.

Basic commands

MongoDB allows all CRUD operations on the database. Following commands are used to Create, Read, Update and Delete documents.

Note: Many commands in MongoDB accepts parameters. Parameters are passed withing curly braces {}. If a command takes multiple parameters, the parameters are separated with a comma (,).

• ‘insert’ is used to create a document. The syntax is insert({document}). The document is passed as a parameter within curly braces. As a document has JSON structure, it can itself contain curly braces to represent its internal structure. In the following example, insert accepts a document with fields id, user etc. The user field contains firstname, lastname, email and other fields within it.

db.users.insert({id: 1,user: {firstname:"John",lastname:"Cena",email:["jc@wwe.com","jc1@wwe.com"],password:"YouCantSeeMe",address:{street:"34 some street", country:"USA"}}})
 WriteResult({ "nInserted" : 1 })

Just to show that mongoDB is schema-less, we can add a completely different document to the same collection

> db.users.insert({somethingDifferent:'f'})
WriteResult({ "nInserted" : 1 })
> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e723ac6b76ebe43c4df9e3"), "somethingDifferent" : "f" }

A collection is indexed on ‘_id’ field by default. You can verify this using getIndexes command.

> db.users.getIndexes()
[
{
“v” : 2,
“key” : {
“_id” : 1
},
“name” : “_id_”,
“ns” : “web-business.users”
}
]
>

• ‘find’ is used to read a document. The following command would find all the documents in a collection. Later sections would show how to use selectors to find specific documents.

> db.users.find()
{ “_id” : ObjectId(“58e7213c708ac93c9c8f89e3”), “id” : 1, “user” : { “firstname” : “John”, “lastname” : “Cena”, “email” : [ “jc@wwe.com”, “jc1@wwe.com” ], “password” : “YouCantSeeMe”, “address” : { “street” : “34 some street”, “country” : “USA” } } }

• ‘remove’ is used to delete a document. To remove a document, use ‘remove’ and pass a selector to it. A mondoDB selector is used to specify selection criteria (similar to ‘where’ clause in SQL). The selection criteria is specified as a JSON object ({field:value}) and would match any document whose ‘field’ is equal to ‘value’. For example, following command will remove the document which has field matching ‘somethingDifferent:’f”

> db.users.remove({somethingDifferent:'f'})
WriteResult({ "nRemoved" : 1 })
> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }

• ‘update’ is used to update or replace documents. It is discussed in later sections

MongoDB selectors

Like in any database, knowledge of mongoDB selectors would be essential to develop a database application. As mentioned previously, selectors are specified using JSON notation ({field:value}). We can specify multiple selection criteria and can also embed them. First, let us add following two documents to ‘users’ collection before we play around with selectors

> db.users.insert({id: 2,user: {firstname:"William",lastname:"Goldberg",email:["wg@wwe.com","wg1@wwe.com"],password:"YouAreNext",address:{street:"44 some other street", country:"USA"}}})
WriteResult({ "nInserted" : 1 })
> db.users.insert({id: 3,user: {firstname:"Dwayne",lastname:"Johnson",email:["dj@wwe.com","dj1@wwe.com"],password:"WhatRockIsCooking",address:{street:"44 new street", country:"Canada"}}})
WriteResult({ "nInserted" : 1 })
> db.users.insert({id: 4,user: {firstname:"Finn",lastname:"Balor",email:["fb@wwe.com","fb1@wwe.com"],password:"whateverHisTaglineIs",address:{street:"64 victoria street", country:"UK"}}})
WriteResult({ "nInserted" : 1 })

Selection based on single or multiple fields

To make a selection based on single field value, simply use {field:value} format.

• Select document with field id equal to 2

> db.users.find({id:2})
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "USA" } } }

• Select document with field country equal to UK

> db.users.find({"user.address.country":"UK"})
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "id" : 4, "user" : { "firstname" : "Finn", "lastname" : "Balor", "email" : [ "fb@wwe.com", "fb1@wwe.com" ], "password" : "whateverHisTaglineIs", "address" : { "street" : "64 victoria street", "country" : "UK" } } }

Notice the use of user.address.country structure. As the fields contain embedded fields (user contains address, address contains country, we access them using dot notation. We also had to use double quotes

using operators with selectors (Query selectors)

mongoDB provides several operators which could be used with selectors to create complex queries. The format to use operators with selectors is {field : {operator:value}}

• AND operator:. Comma (,) is used as an AND operator. It selects document based on two or more fields. The format is {field1:value1, field2:value2,…}. This selection represents ‘and’ condition i.e. field1=value1 AND field2=value2 AND ….

db.users.find({"user.address.country":"USA" , "user.firstname":"John"})
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }

• OR operator: $or is used as OR operator. The format is $or:[{field1:value1},{field2:value2}]. It would select documents which have field1 as value1 or field2 as value2. The example below will select documents with country value equal to USA or UK. In this example, instead of OR-ing on two different fields, the document is selected based on whether the same field (country) has value either ‘USA’ or ‘UK’ ($or: [{field1:value1}, {field1,value2}])

> db.users.find({$or: [{"user.address.country":"USA"},{"user.address.country":"UK"}]})
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "id" : 4, "user" : { "firstname" : "Finn", "lastname" : "Balor", "email" : [ "fb@wwe.com", "fb1@wwe.com" ], "password" : "whateverHisTaglineIs", "address" : { "street" : "64 victoria street", "country" : "UK" } } }
>

• applying operators on values instead of fields: AND and OR were used to select documents whose fields were equal to a specific value (Eg country should be UK or USA). Certain operators can be used to specify range of values instead of a single value. These are $lt (less than), $lte (less than or equal), $gt (greater than), $gte (greater than or equal), $ne (not equal). Their syntax is field: {operator:value}. The following example would find documents with id less than 3.

> db.users.find({id:{$lt:3}})
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "USA" } } }
>

• $exists – As MongoDB is schema-less, two different documents in same collection could have entirely different fields. $exists operator is used to check whether a field exists or not. Its syntax is fieldname: {$exists: true or false}. It will check whether ‘fieldname’ exists or not depending on whether the value specified for $exists is ‘true’ or ‘false’. Let us add a new document which contains an extra field ‘boss’

db.users.insert({"id" : 5, "user" : { "firstname" : "Vince", "lastname" : "McMohan", "email" : [ "wm@wwe.com", "wm1@wwe.com" ], "password" : "YouAreFired", "address" : { "street" : "64 some street", "country" : "USA" } },"boss":"President" })

Now query the database based to check if ‘boss’ field exists in a document or not.

> db.users.find({'boss':{$exists:true}})
{ "_id" : ObjectId("58ec6704591221577c0e7ab9"), "id" : 5, "user" : { "firstname" : "Vince", "lastname" : "McMohan", "email" : [ "wm@wwe.com", "wm1@wwe.com" ], "password" : "YouAreFired", "address" : { "street" : "64 some street", "country" : "USA" } }, "boss" : "President" }
>
> db.users.find({'boss':{$exists:false}})
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "id" : 4, "user" : { "firstname" : "Finn", "lastname" : "Balor", "email" : [ "fb@wwe.com", "fb1@wwe.com" ], "password" : "whateverHisTaglineIs", "address" : { "street" : "64 victoria street", "country" : "UK" } } }

• selecting from an Array: In our example, field ’email’ is an array. In MongoDB, an array is treated as a first class object. What it means to us is that we can use an array like any other regular field. In the following example, a selection is based on email field. However, we write the query the same was as if it was a regular field

> db.users.find({"user.email":"jc1@wwe.com"})
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }

Though the example above used ‘find’ command, the selectors discussed above could be used with other commands like remove and count.

> db.users.count()
5
> db.users.count({id:{$lt:3}})
2
> db.users.remove({boss:{$exists:true}})
WriteResult({ "nRemoved" : 1 })
> db.users.count()
4

update command

The ‘update’ command has two purposes. It can be used to replace an entire document or update one or more fields of a document.

• use update to replace a document: To replace an entire document, use the syntax db.collectionName.update({existing document selector},{new document}). For example, following command would replace document with id equal to 4 ({id:4}) with an entirely new document (passed as the second argument). To demonstrate that the new document could be entirely different (because mongoDB is schemaless), first ‘find’ is run to show current documents, then update is used to replace an existing document and then find is used again to show the new document added to the collection.

> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "id" : 4, "user" : { "firstname" : "Finn", "lastname" : "Balor", "email" : [ "fb@wwe.com", "fb1@wwe.com" ], "password" : "whateverHisTaglineIs", "address" : { "street" : "64 victoria street", "country" : "UK" } } }

> db.users.update({id:4},{"newdocument":true, "number":4, "description":"completely random and different than other documents"})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
>
>
> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "newdocument" : true, "number" : 4, "description" : "completely random and different than other documents" }
>

• use update to change fields of a document: The more obvious use of update is to modify an existing document by replacing one or more fields of that document. $set is used in the update command to modify a document. The syntax is document.collectionName.update({existing document selector},{$set: {new fields and/or new values of existing fields}}). The following example selects document with lastname ‘Goldberg’, modifies the country from USA to UK and adds a new field ‘boss’.

> db.users.update({"user.lastname":"Goldberg"},{$set:{"user.address.country":"UK","boss":true}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })> db.users.find({"user.lastname":"Goldberg"})
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true }

Generally, update’ works on an existing document which is passed as first argument to update. If the document doesn’t exist, update would fail. We can pass an additional argument, ‘upsert’ to update. If this argument is set to true, update would insert the document if it is not found. In the following example, update looks for a document with user.lastname equal to McMohan. As such a document does not exist, the update command fails. Next, the update command is used with upsert set to true. In this case, the document is inserted. The new document inserted contains all the fields passed in first and second argument of update.

> db.users.update({"user.lastname":"McMohan"},{$set:{"user.address.country":"UK","boss":true}})
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })

> db.users.update({"user.lastname":"McMohan"},{$set:{"user.address.country":"UK","boss":true}},{upsert:true})
WriteResult({
 "nMatched" : 0,
 "nUpserted" : 1,
 "nModified" : 0,
 "_id" : ObjectId("58ec7f1a06aae294df20dd32")
})
> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "newdocument" : true, "number" : 4, "description" : "completely random and different than other documents" }
{ "_id" : ObjectId("58ec7f1a06aae294df20dd32"), "user" : { "lastname" : "McMohan", "address" : { "country" : "UK" } }, "boss" : true }
>

• updating multiple fields

By default, update would change only 1 field even if multiple fields match the selection criteria. To update multiple fields, use {multi:true} argument. In the following example, empty {} passed as first argument to update would select all documents. Without the ‘multi’ argument, only 1 document gets updated. With the ‘multi’ argument, all the rows get updated.

> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } } }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "newdocument" : true, "number" : 4, "description" : "completely random and different than other documents" }
{ "_id" : ObjectId("58ec7f1a06aae294df20dd32"), "user" : { "lastname" : "McMohan", "address" : { "country" : "UK" } }, "boss" : true }
>
> db.users.update({},{$set:({newfield:true})})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } }, "newfield" : true }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } } }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "newdocument" : true, "number" : 4, "description" : "completely random and different than other documents" }
{ "_id" : ObjectId("58ec7f1a06aae294df20dd32"), "user" : { "lastname" : "McMohan", "address" : { "country" : "UK" } }, "boss" : true }

> db.users.update({},{$set:({newfield:true})},{multi:true})
WriteResult({ "nMatched" : 5, "nUpserted" : 0, "nModified" : 4 })
> db.users.find()
{ "_id" : ObjectId("58e7213c708ac93c9c8f89e3"), "id" : 1, "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } }, "newfield" : true }
{ "_id" : ObjectId("58e72afb6b76ebe43c4df9e4"), "id" : 2, "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true, "newfield" : true }
{ "_id" : ObjectId("58e72b8f6b76ebe43c4df9e5"), "id" : 3, "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } }, "newfield" : true }
{ "_id" : ObjectId("58e72e486b76ebe43c4df9e6"), "newdocument" : true, "number" : 4, "description" : "completely random and different than other documents", "newfield" : true }
{ "_id" : ObjectId("58ec7f1a06aae294df20dd32"), "user" : { "lastname" : "McMohan", "address" : { "country" : "UK" } }, "boss" : true, "newfield" : true }
>

Projections

In ‘find’ command, by default, mongoDB returns all the fields of a document. To limit the number of fields returned, use {fieldname:1 or 0} argument to include or exclude a field. The following example returns only ‘id’ field. The _id field (generated by mongoDB) is always returned unless it is explicitly set to 0.

db.users.find({},{id:1})
{ “_id” : ObjectId(“58e7213c708ac93c9c8f89e3”), “id” : 1 }
{ “_id” : ObjectId(“58e72afb6b76ebe43c4df9e4”), “id” : 2 }
{ “_id” : ObjectId(“58e72b8f6b76ebe43c4df9e5”), “id” : 3 }
{ “_id” : ObjectId(“58e72e486b76ebe43c4df9e6”) }
{ “_id” : ObjectId(“58ec7f1a06aae294df20dd32”) }

To exclude _id, set it to 0

> db.users.find({},{id:1,_id:0})
{ "id" : 1 }
{ "id" : 2 }
{ "id" : 3 }
{ }
{ }

The above example returns documents which contain id field but also all other documents which do not contain id field at all. To get only ‘id’ field, we can combine be more specific in the selection in ‘find’ method by specifying that find returns only those documents which contain ‘id’ field.

db.users.find({id:{$exists:true}},{id:1})
{ “_id” : ObjectId(“58e7213c708ac93c9c8f89e3”), “id” : 1 }
{ “_id” : ObjectId(“58e72afb6b76ebe43c4df9e4”), “id” : 2 }
{ “_id” : ObjectId(“58e72b8f6b76ebe43c4df9e5”), “id” : 3 }

db.users.find({id:{$exists:true}},{id:1, _id:0})
{ “id” : 1 }
{ “id” : 2 }
{ “id” : 3 }

Setting ‘id’ and ‘_id’ field to 0 would exclude them in the projection

> db.users.find({id:{$exists:true}},{id:0, _id:0})
{ "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } }, "newfield" : true }
{ "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true, "newfield" : true }
{ "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } }, "newfield" : true }
>

sorting

sort() method is used to arrange results in ascending or descending order. The syntax is sort({fieldname1: 1 or -1}, fieldname2: 1 or -1}). ‘fieldnname1’ and ‘fieldname2’ denote the fields by which we want to the results. 1 or -1 denote ascending or descending order. The following example arranges results using firstname.

> db.users.find({id:{$exists:true}},{id:0, _id:0}).sort({"user.firstname":1})
{ "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } }, "newfield" : true }
{ "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } }, "newfield" : true }
{ "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true, "newfield" : true }

> db.users.find({id:{$exists:true}},{id:0, _id:0}).sort({"user.firstname":-1})
{ "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true, "newfield" : true }
{ "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } }, "newfield" : true }
{ "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } }, "newfield" : true }
>

pagination

limit() is used to restrict the number of documents returned from a search. skip() is used to ignore (skip) documents from beginning of the result.

> db.users.find({id:{$exists:true}},{id:0, _id:0}).sort({"user.firstname":-1}).limit(1)
{ "user" : { "firstname" : "William", "lastname" : "Goldberg", "email" : [ "wg@wwe.com", "wg1@wwe.com" ], "password" : "YouAreNext", "address" : { "street" : "44 some other street", "country" : "UK" } }, "boss" : true, "newfield" : true }
>
> db.users.find({id:{$exists:true}},{id:0, _id:0}).sort({"user.firstname":-1}).skip(1)
{ "user" : { "firstname" : "John", "lastname" : "Cena", "email" : [ "jc@wwe.com", "jc1@wwe.com" ], "password" : "YouCantSeeMe", "address" : { "street" : "34 some street", "country" : "USA" } }, "newfield" : true }
{ "user" : { "firstname" : "Dwayne", "lastname" : "Johnson", "email" : [ "dj@wwe.com", "dj1@wwe.com" ], "password" : "WhatRockIsCooking", "address" : { "street" : "44 new street", "country" : "Canada" } }, "newfield" : true }
>

count

count() returns number of documents

> db.users.find({id:{$exists:true}},{id:0, _id:0}).count()
3
>

Dropping a database

To drop/delete a database,  switch to the database and use db.dropDatabase() command

> use web-business
switched to db web-business
> db.dropDatabase()
{ "dropped" : "web-business", "ok" : 1 }
> show dbs
admin 0.000GB
familyDB 0.000GB
local 0.000GB

MongoDB and Play

The premier on MongoDB at the beginning of this blog gives enough familiarity to us to create queries to create, update, read and delete documents. Now, comes the main topic of this blog – how to use mongoDB with Play.  The web application I plan to make is an extension of user registration form I created in the blog on Forms and JSON.  Following is the snapshot of the form. The application sends user’s registration details as a JSON to play application and would return success or failure. It doesn’t store user’s details in the database, something I intend to fix in this blog.

Following is the code

• Routes file

# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~

GET      /                          controllers.Application_Test.index
POST     /newreg                   controllers.Application_Test.registrationRequest
GET     /newreg-success           controllers.Application_Test.registrationRequestSuccess
GET     /newreg-error            controllers.Application_Test.registrationRequestFail


# Map static resources from the /public folder to the /assets URL path
GET     /assets/*file               controllers.Assets.at(path="/public", file)

• Controller – Application_Test 

package controllers

import anorm.{NotAssigned, Pk}
import play.api.data.Form
import play.api.data.Form._
import play.api.data.Forms
import play.api.mvc.Controller
import play.api.mvc._
import play.api.data.Forms._
import play.api.data.validation.Constraints._
import play.api.db.DB
import models._

import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.util.Try
import play.api.libs.json
import play.api.libs.json._
import play.api.Logger
import play.api.data.validation.ValidationError
import play.api.libs.functional.syntax._
import play.api.libs.json.Reads._
import play.api.libs.json.Writes._

object Application_Test extends Controller {

  def index = Action {
        Ok(views.html.HTMLTest())
  }

  implicit val addressRead:Reads[Address] =(
    (JsPath \ "street").read[String] and
      (JsPath \ "country").read[String]
    )(Address.apply _)

  /* do not need to declare this implicitas we will pass this as argument*/

 /*though read expects an implicit r:Reads, the function returns a Reads. So compiler will use this as it can generate Reads*/
  def checkPasswordCase: Reads[String] = {
    Reads.StringReads.filter(ValidationError("Password not in lowercase"))(str => {
      (str.matches("""[a-z]+"""))
    })
  }

  def checkPasswordLength(min:Int): Reads[String] = {
    Reads.StringReads.filter(ValidationError("Invalid password length"))(str => {
      (str.length >= min)
    })
  }

  implicit val user2Read:Reads[User2] = (
    (JsPath \ "id").read[Int] and
      (JsPath \ "user" \ "first-name").read[String] and
      (JsPath \ "user" \ "last-name").read[String] and
      /* read in general takes an implicit argument. In following statement, we have passed argumet onlyLowercase to reads*/
      /* read is Reads.at[T](this)(r)*/
      (JsPath \ "user" \ "password").read[String]((checkPasswordCase) andKeep (checkPasswordLength(8))) and
      (JsPath \ "user" \ "email").read[List[String]] and
      (JsPath \ "user" \ "address").read[Address]
    )(User2.apply _)

  implicit val ackWrites:Writes[Acknowledgment] = (
    (JsPath \ "status").write[Int] and
      (JsPath \ "message").write[String]
  )(unlift(Acknowledgment.unapply))

  def registrationRequest = Action(parse.json) { request => {
    Logger.debug("received message:" + request)
    Logger.debug("received message:" + request.body)
    val jr:JsResult[User2] = request.body.validate[User2]
    Logger.debug( "jr is "+jr)

    jr match {
      case s:JsSuccess[User2] => {
        val user = s.get

        val ack = Acknowledgment(1,"Welcome "+user.firstName+" "+user.lastName)
        Logger.debug("success ack Json:"+Json.toJson(ack))
          Ok(Json.toJson(ack))
      }
      case f:JsError => {
        Logger.debug("error: "+JsError.toFlatJson(f))
        val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
        Logger.debug("fail ack:"+Json.toJson(ack))
        BadRequest(Json.toJson(ack))
      }
    }
    }
  }

  def registrationRequestSuccess = Action { request =>
    Ok(views.html.regconf("success"))
  }

  def registrationRequestFail = Action { request =>
    Ok(views.html.regconf("Error"))
  }


  def getFullName(jsv: JsValue, poi:List[String]): Option[Vector[String]] = {

    /* in the inner recursive function, pass the keys still to be looked for and values found so far*/

    def inner(ijsv:JsValue,ikeysTBD:List[String], ivaluesSoFar:Vector[String]):Option[Vector[String]] = {
     ikeysTBD match { // see if we have looked for all names
       case Nil => Option(ivaluesSoFar) // we have looked for all parameters.
         //more keys to be found.
       case key::t => (ijsv \ key).validate[String].map(value=> {inner(ijsv, t, ivaluesSoFar ++ Vector(value))}).getOrElse(None)
     }

    }
    inner(jsv,poi, Vector[String]())
  }

}

• View – HTMLTest.scala.html

<pre class=”lang:html decode:true “>

<!DOCTYPE html>
<html lang=”en” xmlns:font-variant=”http://www.w3.org/1999/xhtml”&gt;
<head>
<meta charset=”UTF-8″>
<meta http-equiv=”X-UA-COMPATIBLE” content=”IE=edge”>
<meta name=”viewport” content=”width=device-width,initial-scale=1″>
<title>HTML Test</title>
link rel=”stylesheet” type=”text/css” href=”@routes.Assets.at(“stylesheets/bootstrap.min.css”)”
<style type=”text/css”>

html, body {
height:100%;
margin:0;padding:0
}

.center-form {
width:100%;
margin:auto;
position:relative;
top:50%;
transform: translateY(-50%)
}
</style>

script src=”@routes.Assets.at(“javascripts/jquery-3.1.1.min.js”)” /script
script src=”@routes.Assets.at(“javascripts/bootstrap.min.js”)” /script
</head>
<body>

<!–logo column–>
<!–empty column–>

</div>

<!– for medium and large screens, Second row of Bootstrap grid contains the form for username and password. Total 3 columns (12/4). –>

<!–form–>

ID

First Name

Last Name

Email

Password

Confirm password

Street Name

Country

<button type=”submit” class=”btn btn-primary”>Sign Up</button>
</form>
</div>

<!–empty column–>

</div>
</div>

script src=”@routes.Assets.at(“javascripts/myScripts.js”)” /script
</body>
</html>

</pre>

• View – regconf.scala.html

<pre class=”lang:html decode:true “>

@(title:String)

<!DOCTYPE html>

<html>

<head>

<title>@title</title>

link rel=”stylesheet” media=”screen” href=”@routes.Assets.at(“stylesheets/main.css”)”

link rel=”shortcut icon” type=”image/png” href=”@routes.Assets.at(“images/favicon.png”)”

script src=”@routes.Assets.at(“javascripts/jquery-1.9.0.min.js”)” type=”text/javascript” /script

</head>

<body>

@if(title == “success”) {

<h1> Thanks for registering.</h1>

}else {

<h1> Error.</h1> }

</body>

</html>

</pre>

• Models – User2.scala

package models

import play.api.db._
import play.api.Play.current
import anorm._
import anorm.SqlParser._

case class Address (street: String, country:String)

case class User2(
                  id:Int,
                  firstName:String,
                  lastName:String,
                  password:String,
                  email:List[String],
                  address:Address
                )

case class Acknowledgment (
                           status:Int,
                           message:String
                           )

• Javascript – myScripts.js

    function createAddressObject(form){
       var addressObject = {};
      // var jqf = $(form);

       var street = $('#registration-form [name=street]')
       var sname = street.attr('name');
       var sval =street.val();
       console.log('street found'+sname+sval);
       addressObject[sname]=sval;

       var country = $('#registration-form [name=country]')
       var cname = country.attr('name');
       var cval =country.val();
       console.log('country found'+cname+cval);
       addressObject[cname]=cval;
        return addressObject ;
    }

    function createUserObject(form){
     var userObject = {};
    // var jqf = $(form);

    var fn= $('#registration-form [name=first-name]')
            var fnn = fn.attr('name');
            var fnval =fn.val();
            console.log('fn found'+fnn+fnval);
            userObject[fnn]=fnval;

    var ln= $('#registration-form [name=last-name]')
            var lnn = ln.attr('name');
            var lnval =ln.val();
            console.log('ln found'+lnn+lnval);
            userObject[lnn]=lnval;

    var em= $('#registration-form [name=email]')
            var email = [];
            var emn = em.attr('name');
            var emval =em.val();
            console.log('email found'+emn+emval);
            email[0]=emval;
            userObject[emn]=email;

    var pwd= $('#registration-form [name=password]')
            var pwdn = pwd.attr('name');
            var pwdval =pwd.val();
            console.log('pwd found'+pwdn+pwdval);
            userObject[pwdn]=pwdval;
    userObject['address']=createAddressObject(form);
      console.log('userObject: '+userObject)
      return userObject ;
   }

    function objectifyForm2(form){
     var formObject = {};
     //var jqf = $(form);

     var id = $('#registration-form [name=id]')
        console.log('id found'+id);
        formObject[id.attr('name')]=Number(id.val());

     formObject['user']=createUserObject(form);



     return formObject;

     }

    $(document).ready(function(){
        // click on form submit
        $('#registration-form').on('submit',function(e){
        e.preventDefault();
       /*
       The .serializeArray() method uses the standard W3C rules for successful controls to determine which elements
       it should include; in particular the element cannot be disabled and must contain a name attribute.
       Encode a set of form elements as an array of names and values.
       */
       var details2 = JSON.stringify(objectifyForm2(this));
        console.log('details2: '+details2)
            // send ajax
            $.ajax({
                url: '/newreg', // url where to submit the request
                type : "POST", // type of action POST || GET
                datatype : "json", // data type
                /* this doesn't work*/
                //contentType: "application/json; charset=utf-8",
                /*this does*/
                contentType: "text/json; charset=utf-8",
                data : details2,
                success : function(result) {
                    // you can see the result from the console
                    // tab of the developer tools
                    console.log(result);
                    window.location = "/newreg-success"
                },
                error: function(xhr, resp, text) {
                    console.log(xhr, resp, text);
                    window.location = "/newreg-error"
                }
            })
        });
    });

Adding mongoDB in Play

Before we can use mongoDB in our code, we would first need to finish few administrative tasks like adding and configuring mongoDB in Play framework.

• We need a driver to access mongoDB. The database driver is a software component which is our interface to the database. We will use it to connect to the database and perform the CRUD operations on the database. I will use Reactive Mongo driver in the example. Add mongoDB dependency in build.sbt

libraryDependencies ++= Seq(
  jdbc,
  anorm,
  cache,
  "org.reactivemongo" %% "play2-reactivemongo" % "0.10.2"
)

• Next, we need to tell our application where is the database. Add following line to Application.conf

mongodb.servers = ["localhost:27017"]
mongodb.db = "website-db"

The first line tells our application where the mongoDB server (deamon) is running. The second line is the name of our database.

• We also need a MongoDB Play plugin. The plugin provides convenient APIs to work with the database. While the driver helps us in connecting the database, we interact with Add following code to play.plugins

400:play.modules.reactivemongo.ReactiveMongoPlugin

• Finally, let us create the database and collection our application will work on. Using mongo shell, create a database and a collection

> use website-db
switched to db website-db
> db.createCollection("users")
{ "ok" : 1 }
> db.getCollectionNames()
[ "users" ]

Connecting to mongoDB

To connect to the database from Play application, add following lines of code

• Import mongoDriver and MongoConnection in Application_Test controller

import reactivemongo.api.MongoDriver
import reactivemongo.api.MongoConnection

• Create an instance of Driver (MongoDriver) and open connection (MongoConnection) to the mongoDB. Add these lines of code in registrationRequest action in Application_Test controller

Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))

Assuming that the mongoDB instance is running on localhost (your machine), you should see the following trace in the mongod window when user information is submitted.

2017-04-12T20:43:33.517+0100 I NETWORK [thread1] connection accepted from 127.0.0.1:54922 #10 (10 connections now open)

Connecting to database and collections

Now that we have the connection to mongoDB, we can connect to databases inside it.

import reactivemongo.api.collections.default.BSONCollection

val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")

At this point, we have a reference to the collection we want to work on.   BSONCollection provides the APIs to perform the CRUD operations like find, insert, update, remove on the collection. It also has commands that operate on the collection itself – create, rename, drop

Querying a Collection

Next, we need to write a query. This query will be used in methods provided by BSONCollection. A query is created using BSONDocument. BSONDocument defines the structure. In MongoDB Premier section, we looked at several shell commands which would take parameters in curly braces {}, eg:

> db.users.insert({somethingDifferent:'f'})
> db.users.remove({somethingDifferent:'f'})
> db.users.insert({id: 2,user: {firstname:"William",lastname:"Goldberg",email:["wg@wwe.com","wg1@wwe.com"],password:"YouAreNext",address:{street:"44 some other street", country:"USA"}}})WriteResult({ "nInserted" : 1 })

In crude terms, BSONDocument represents the curly braces and the contents inside it. For example, to find/read a document with “firstname” equal to John, the query in Mongo shell would be

db.users.find({“user.firstname”:”John”})

In BSONDocument, we will create the document (curly braces part) as

import reactivemongo.bson.BSONDocument

val query = BSONDocument("user.firstname"->"John")

Note: BSONDocument(“user.firstname”->”John”) is different from BSONDocument(“user”-> BSONDocument(“firstname”->user.firstName)).  The first creates following structure (you may check by running mongodb in verbose mode, mongodb -v) { user.firstname: "John" }.The latter query2 creates following structure { user: { firstname: "John" } }. The above two are not the same. First compares the fields of embedded document using dot notation. Second compares document as a whole against a embedded document

Next, we will pass this query to BSONCollection’s find method. ‘find’ finds the documents matching the given criteria. The ‘find’ method has two flavors. It can take a selector as an argument or a selector and a projection as arguments

find[S, P](selector: S, projection: P)(implicit swriter: BSONDocumentWriter[S], pwriter: BSONDocumentWriter[P]): GenericQueryBuilder[BSONDocument, BSONDocumentReader, BSONDocumentWriter]

find[S](selector: S)(implicit swriter: BSONDocumentWriter[S]): GenericQueryBuilder[BSONDocument, BSONDocumentReader, BSONDocumentWriter]

‘find’ returns a BSONQueryBuilder. The query is not executed yet. BSONQueryBuilder would allow us to refine the query (add sorting or create a projection for example).

To run the query, use Cursor’ or ‘one’. Cursor returns multiple results (in format Cursor [BSONDocument) while ‘one’ returns the first Document (in format Future[Option[BSONDocument]]. Refer to blog on Concurrency to read about Future. I’ll use the ‘one’ method to get on document

val result = collection.find(query).one

Because our code uses Future, we would also need to make some changes to the Action handler in Play

• Change Action to Action.async.

‘one’ returns a Future[Option[BSONDocument]]. The Future[Option[A]] type combines the notion of concurrency(from Future) and the notion of failure(from Option) giving you a concurrent computation that might fail. In play, Action.async (instead of just Action) is more convenient way to handle the complexities of a concurrent task. In blog on Concurrency, I discussed how we would need to wait for the concurrent task to finish before the main thread returns/exits. Using Action, we would have to write code to start the concurrent task (Future) and wait for it to finish before returning from the Action code, something like follows:

def registrationRequest = Action(parse.json) { request => {
//start future
//wait for future to finish 
//process future's result
} //now can exit action

Action.async would handle all these complexities for us. All we would have to do is write code to start the future and handle its result. Rest would be taken care by Action.async One difference between Action and Action.async is that Action.async returns a Future. It returns scala.concurrent.Future[play.api.mvc.SimpleResult] instead of play.api.mvc.SimpleResult which is returned by Action. We return Future from our code by wrapping it in Future.successful. Future.successful creates an already completed Future object with specified result (in our example, it would be BadRequest of type SimpleRequest). Let us make the above two changes, see the following complete Action code (actually Action.sync code!)

def registrationRequest = Action.async(parse.json) { request => {
    Logger.debug("received message:" + request)
    Logger.debug("received message:" + request.body)
    val jr:JsResult[User2] = request.body.validate[User2]
    Logger.debug( "jr is "+jr)

    //parsed incoming JSON
    jr match {
      //JSON is OK. Now proceed with database processing
      case s:JsSuccess[User2] => {

        val user = s.get

        //open database and connect to collection
        Logger.debug("opening database connection")
        val driver = new MongoDriver()
        val connection = driver.connection(List("localhost"))
        val db = connection.db("website-db")
        val collection = db.collection[BSONCollection]("users")

        //create query
        val query = BSONDocument("user.firstname"->user.firstName)
        
        Logger.debug("query is:"+query)

        //Get one document. result is of type Future[Option[BSONDocument]]
        val result = collection.find(query).one

        //map would be called if Future was successful in execution. Successful
        //execution means the database was queried successfully. The result
        //from database could still be either the document was found or 
        //not. This information is stored in the Option inside Future.
        
        result.map(option => option match {
          case None => { //no record. Can add
            Logger.debug("No record from mongo: Can add")
            val ack = Acknowledgment(1, "Welcome " + user.firstName + " " + user.lastName)
            Logger.debug("success ack Json:" + Json.toJson(ack))
            Ok(Json.toJson(ack))
          }
          case Some(x) => {//duplicae record
            Logger.debug("error from Mongo. Duplicate:" + x)
            val ack = Acknowledgment(0, "duplicate: " + x.toString())
            Logger.debug("fail ack:" + Json.toJson(ack))
            BadRequest(Json.toJson(ack))
          }
        }
        )
      }
      case f:JsError => Future.successful({ 
        Logger.debug("error: "+JsError.toFlatJson(f))
        val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
        Logger.debug("fail ack:"+Json.toJson(ack))
        BadRequest(Json.toJson(ack))
      })
    }
    }
  }

The code above has two return points, one after JrSuccess and other after JrFailure. We didn’t need to wrap JsSuccess in Future.successful because ‘result’ is a Future and result.map would also return a Future (refer to Concurrency blog). Thus we only needed to wrap JrFailure. Revising ‘map’ in Future, ‘map’ creates a new future by applying a function to the successful result of this future. In other words, ‘map’ would return a Future (code on right of => passed in {} to map. It will run concurrently). The code might contain an input argument (left side of =>. It would be the output of the Future on which map was called). The return value of map is a Future[S] where S would depend upon the return value of the code passed to map. In above example, result.map({…}) would take the output of the ‘result’ Future (an Option), use it as an input argument to code on right side of => in {}. The return type of the Future created by the map is Future[SimpleResult] because both Ok and BadRequest (last lines of None and Some parts) are of type SimpleResult. The code in {} of map is executed as a Future (a concurrent process).

To test the above code, perform following steps

• start mongod in verbose mode (mongodb -v)
• Add a record in mongoDB using shell in ‘website-db’ database and ‘users’ collection

use website-db

db.users.insert({id: 1,user: {firstname:"John",lastname:"Cena",email:["jc@wwe.com","jc1@wwe.com"],password:"YouCantSeeMe",address:{street:"34 some street", country:"USA"}}})

• start the play application and enter a user with first-name John from web browser

• On clicking ‘Sign Up’, you should see Error page
• If you repeat the above steps with a different name, you’ll get success page.

Handling multiple documents using Cursors

The example above handled a single Document. To get multiple documents from the database, we could use the ‘cursor’ method. It returns Cursor[BSONDocument]. It has two methods, collect and enumerate.

• collect[List]() which returns a future list of documents. These are stored in-memory. Collect will collect  all the documents into a collection of type List. We could pass a Vector as well instead of a List.
• enumerate() which returns an Enumerator of documents. These are handled as streams.

Following changes in code would show all the users in the database

• Routes file

GET      /all                          controllers.Application_Test.list

• Controller file – Application_Test

def list = Action.async { request=> {
  Logger.debug("Listing all users")
  Logger.debug("opening database connection")
  val driver = new MongoDriver()
  val connection = driver.connection(List("localhost"))
  val db = connection.db("website-db")
  val collection = db.collection[BSONCollection]("users")

  val query = BSONDocument()  val findCursor: Cursor[BSONDocument] = collection.find(query).cursor[BSONDocument]  val findFutureList = findCursor.collect[List]()
  findFutureList.map((list:List[BSONDocument]) => {
    Logger.debug("list count:"+list.length)
    Ok(views.html.list(list))
  })
}
}

• New file in views folder – list.scala.html

<pre class=”lang:html decode:true “>

@(list: List[reactivemongo.bson.BSONDocument])

<!DOCTYPE html>

<html lang=”en”>

<head>

<meta charset=”UTF-8″>

<title>Title</title>

</head>

<body>

@list.map(elem=> { @{elem} })

</body>

</html>

</pre>

Note: When importing packages in view, the first line must be argument line. Import must be after it.

@(list: List[reactivemongo.bson.BSONDocument])

Or

@(list: List[reactivemongo.bson.BSONDocument])
@{ import reactivemongo.bson._} 

To test the code, let us first add few records in mongoDB using shell. Execute following command on shell. ‘insertMany’ add multiple Documents at the same time. Notice that we pass multiple documents in an array []

db.users.insertMany([{id: 1,user: {firstname:"John",lastname:"Cena",email:["jc@wwe.com"],password:"YouCantSeeMe",address:{street:"34 some street", country:"USA"}}},{id: 2,user: {firstname:"Dwayne",lastname:"Johnson",email:["dj@wwe.com"],password:"WhatRockIsCooking",address:{street:"35 some street", country:"USA"}}}])

Now type URL http://localhost:9000/list. You should see a page similar to following

The output above isn’t in readable. This will be fixed when in De/serialise section below

Adding a new document in a Collection

Continuing our previous example, we would like to insert the user if it doesn’t exist. To insert a new document, we use BSONCollection’s ‘insert’ method which inserts a document into the collection and wait for the reactivemongo.core.commands.LastError result. LastError can be used to check whether the insertion was successful. Like ‘one’, ‘insert’ returns a Future.

insert(document: BSONDocument)(implicit ec: ExecutionContext): Future[LastError]

First, import following packages

import reactivemongo.core.commands._
import reactivemongo.bson._

The logic of the Action.async function is:

1) get JSON from client (browser) and validate it

val jr: JsResult[User2] = request.body.validate[User2]

2) If JSON is correct, check if user already exists (look up firstname)

jr match {
case s: JsSuccess[User2] => {

 val user = s.get
 Logger.debug("opening database connection")
 val driver = new MongoDriver()
 val connection = driver.connection(List("localhost"))
 val db = connection.db("website-db")
 val collection = db.collection[BSONCollection]("users")

 val query = BSONDocument("user.firstname" -> user.firstName)

 Logger.debug("query is:" + query)

 //result is of type Future[Option[BSONDocument]]
 val findFuture:Future[Option[BSONDocument]] = collection.find(query).one

3) If user does not exist, add the user, else do not add the user

‘map’ the Future returned by ‘find’ command. If the value returned from that Future is None, it means that the user is new.  Create the document and insert it using ‘insert’ command. If the value returned is Some, it means that the user already exists. We will not call ‘insert’ in this case. The code is not trival though and requires more explanation. ‘insert’ returns a Future[LastError]. As we will call ‘insert’ inside ‘map’, the value returned by map would be Future[Future[LastError]] because map returns a Future which takes the output of an existing Future (on which map is called i.e. findFuture). To keep the return type same for both None and Some, the code for Some also returns LastError and the code is wrapped around Future.successful. The first argument to LastError (false) denotes error.

val insertFuture: Future[Future[LastError]] = findFuture.map(option => option match {
  case None => {
    //no record. Can add
    Logger.debug("No record from mongo: Can add")
// create document
    val newUserDoc = BSONDocument(
      "id" -> user.id,
      "user" -> BSONDocument(
        "firstname" -> user.firstName,
        "lastname" -> user.lastName,
        "email" -> BSONArray(user.email(0)),
        "password" -> user.password,
        "address" -> BSONDocument(
          "street" -> user.address.street,
          "country" -> user.address.country
        )
      )
    )

    //addResult is of type Future[LastError]
    collection.insert(newUserDoc)
  }
  case Some(x) => {
    //duplicate record
    Logger.debug("error from Mongo. Duplicate:" + x)
    Future.successful({
      Logger.debug("findFutureResultmap. Not adding a duplicate")
      LastError(false, None, None, None, None, 0, false)
    })

  }
})

4) Reply to the client (browser)

The use of flatMap needs some explanation. I explained earlier that map returns a Future[S] where the data-type of S would depend on the return value of code executed by map (passed in {} to map). In above example, both None and Some return Future[LastError] (because ‘insert’ returns Future[LastError] and Future.successful(…) would also return Future[LastError]). Thus S is equal to Future[LastError]. Thus findFuture.map would return Future[Future[LastError]]. Thus insertFuture is of type Future[Future[LastError]]. But Action.sync need to return Future[SimpleResult]. Thus we need to convert Future[Future[]] to Future[] and LastError to SimpleResult.

flatMap would convert Future[Future[]] to Future[]. flatMap creates a new future by applying a function to the successful result of this future, and returns the result of the function as the new future. Like ‘map’, flatMap takes the output of the Future on which ‘map’ is called (flatMap is called on insertFuture of type Future[Future[LastError]], so its output is of type Future[LastError]. Refer to the previous explanation on how map works i.e. S is equal to Future[LastError]). Thus ‘lef’ is of type Future[LastError]). It would then apply the function passed to flatMap on ‘lef’ (as lef is a Future, we can call lef.map. As ‘lef’ is Future[LastError], the output of ‘lef’ is of type LastError i.e. ‘le’ is of type LastError. The function passed to map converts ‘le’ (of type LastError) to Ok or BadRequest ( depending on whether le.ok is true or false.) of type SimpleRequest. As flatMap returns the result of the function (passed to it) as the new future, we will get Future[SimpleRequest] as the return type of flatMap.

insertFuture.flatMap(lef => {
  lef.map(le => {
    if (le.ok) {
      Logger.debug("insertFuture map")
      val ack = Acknowledgment(0, "insert success: ")
      Logger.debug("insert success:" + Json.toJson(ack))
      Ok(Json.toJson(ack))
    }
    else {
      Logger.debug("not inserting")
      val ack = Acknowledgment(0, "duplicate: ")
      Logger.debug("fail ack:" + Json.toJson(ack))
      BadRequest(Json.toJson(ack))
    }
  })

If we had used ‘map’ instead of ‘flatMap’, the code wouldn’t compile as the output would have been Future[Future[SimpleResult]] which is not what Action.async is expected to return. ‘map’ would have created a Future, that Future execute code of lef.map. The code of lef.map would have created another Future to execute code of if/else block. The if/else block would have created output of type SimpleResult. Thus the overall output would have been Future[Future[SimpleResult]]. flatMap only picks the result of the function passed to it and ignores any intermediate Futures which get created. flatMap returns only the result part whereas map returns the Future holding the result.

case f: JsError => Future.successful({
  Logger.debug("error: " + JsError.toFlatJson(f))
  val ack = Acknowledgment(0, JsError.toFlatJson(f).toString())
  Logger.debug("fail ack:" + Json.toJson(ack))
  BadRequest(Json.toJson(ack))
})

To test the code, add ‘John’ from client. It should fail as we had already added John in ‘find’ example. Now add a new user (say, Dwayne Johnson). The application should add it. You should also be able to verify it from mongo shell (db.users.find()).

Updating and Removing a document

To update a document, use the update method. To remove the document, use the remove method. The both return Future[LastError]. Following is a modification of an example from ReactiveMongo’s documentation.

val selector = BSONDocument("firstname" -> "John")

val modifier = BSONDocument(
  "$set" -> BSONDocument(
    "lastName" -> "London",
    "firstName" -> "John"),
    "$unset" -> BSONDocument(
      "id" -> 1))

// get a future update
val futureUpdate = collection.update(selector, modifier)

The $unset operator deletes a particular field. By default, multi-select is false. You can also specify whether if the update should concern all the documents that match he selector by setting ‘multi’ to ‘true’.

// get a future update
val futureUpdate = collection.update(selector, modifier, multi = true)

Remove a document

val selector = BSONDocument(
  "firstname" -> "John")

val futureRemove = collection.remove(selector)

By default, remove() deletes all the documents that match the selector. You can change this behavior by setting the firstMatchOnly parameter to true:

val futureRemove = collection.remove(selector, firstMatchOnly = true)

de/serialization type classes

We can read (deserialise) from a BSONDocument into a case class and write (serialise) a case class into a BSONDocuments by extending the BSONDocumentReader[T] and theBSONDocumentWriter[T] respectively. When we extend BSONDocumentReader, we would need to implement its ‘read’ method. When we extend BSONDocumentWriter, we need to implement its ‘write’ method.

BSON Data types

Before discussing de/serialisation, it would be useful to understand the BSON data types available in reactive mongo library. BSON protocol supports several data types. Scala has implemented these data types as separate class. Following is the list of Scala classes for the different BSON data types. All these classes (trait) extend BSONValue. These are defined in reactivemongo.bson package. We have already used few of them. For example we have used BSONDocument extensively. BSONDocument represents a document. A BSONDocument is basically an immutable list of key-value pairs. A BSONDocument accepts tuples of String and BSONValue. Since it is the most used BSON type, one of the main focuses of the ReactiveMongo BSON library is to make manipulations of BSONDocument as easy as possible.

• BSONDocument – set of key-value pairs
• BSONArray – sequence of values
• BSONBinary – binary data
• BSONBoolean – boolean
• BSONDateTime – UTC Date Time
• BSONDouble – 64-bit IEEE 754 floating point
• BSONInteger – 32-bit integer
• BSONJavaScript – JavaScript code
• BSONJavaScriptWS – JavaScript scoped code
• BSONLong – 64-bit integer
• BSONNull – null
• BSONObjectID – 12 bytes default id type in MongoDB
• BSONRegex – regular expression
• BSONString – UTF-8 string

In de/serialisation, we would essentially be converting between Scala types and BSON types using readers and writers. Reactivemongo has already some predefined (implicit) handlers that are available when you import reactivemongo.bson._, including:

• String <-> BSONString
• Int <-> BSONInteger
• Long <-> BSONLong
• Double <-> BSONDouble
• Boolean <-> BSONBoolean

We have already used implicit writers. For example, following code converts “John” (a String) into BSONString using implicit writer for String.

val query = BSONDocument("user.firstname"->"John")

We will use these implicit handlers to create more complex and custom readers and writers.

Deserialisation

Earlier in this blog, I discussed use of Cursor. In that example, we retrieved multiple documents from MongoDB and displayed them on a web-page. The output looked as follows:

The missing part of the above example is deserialization of BSONDocument. Deserialisation is the process of reading a BSONDocument. Let us add that part.

First we need to define the case class which will map to BSONDocument. We already have defined it as follows:

case class Address (street: String, country:String)

case class User2(
                  id:Int,
                  firstName:String,
                  lastName:String,
                  password:String,
                  email:List[String],
                  address:Address
                )

There are some differences between the structure of the case class User2 and the Document we add in Mongodb. Following is the structure of the Mongodb document’

{ 

id:

user: {

  address:{

  }

}

}

As evident, the mongoDB document contains embedded document, user which contains embedded document address. The deserialise code we will write will read address document, then user document, then id field and will use the values of these to create User2. In order to read address and user documents, we will have to create additional case classes which will map to these documents. We already have case class for address (Address). All we need is case class for user document (It is similar to User2 except id field. The way I have defined things looks (is) complicated (and unnecessary) but it makes for a good example as many times, your model might not map to the structure of the document)

/* add this in User2.scala*/

import reactivemongo.bson
import reactivemongo.bson._
import reactivemongo.api.collections.default._
import play.modules.reactivemongo.json.BSONFormats._

case class User(
 firstName:String,
 lastName:String,
 password:String,
 email:List[String],
 address:Address
 )

First step is to deserialise address document and map it to Address case class. The deserialisation code would extend BSONDocumentReader and override the ‘read’ method. The read method takes a BSONDocument as argument (the document we want to read). BSONDocument has a ‘getAs’ method which takes a key of the field we want to read and return an Option which would either contain the value of the key (Some) or None if key is incorrect

def getAs[T](s: String)(implicit reader: BSONReader[_ <: BSONValue, T]): Option[T]

Returns the BSONValue associated with the given key, and converts it with the given implicit BSONReader. The implicit reader tells the getAs function how to read/interpret the value. Implicit readers for basic types like String, Int are alredy available. We can extend these to create readers for complex types (as we will see below). Add following code in User2.scala

object User2{
  implicit object AddressReader extends BSONDocumentReader[Address] {
    def read(doc: BSONDocument): Address = {
      val street = doc.getAs[String]("street").get
      Logger.debug("street is"+street)
      val country = doc.getAs[String]("country").get
      Logger.debug("country is"+country)
      Address(street,country)
    }
  }

• implicit object AddressReader extends BSONDocumentReader[Address] tells compiler that this is a reader for Address that is using this reader, a BSONDocument can be mapped to Address case class
• read takes the BSONDocument as argument and returns Address. It contains the logic for how to read the keys from BSONDocument and create Address
• In read method, we use getAs to extract keys (street, address). Assuming that they exist, get is called on Some (returned by getAs) to extract the value. The type parameter to getAs (String) tells compiler how to interpret the value of the key
• The function returns Address at the end.

Similar to Address deserialisation, let us add code for User and User2. Add following implicit readers in User2.scala

/* reader for user*/
implicit object UserReader extends BSONDocumentReader[User] {
  def read(doc:BSONDocument):User = {
    val firstName = doc.getAs[String]("firstname").get
    Logger.debug("firstname is"+firstName)
    val lastName = doc.getAs[String]("lastname").get
    Logger.debug("lastname is"+lastName)
    val email = doc.getAs[List[String]]("email").get
    Logger.debug("email is"+email)
    val password = doc.getAs[String]("password").get
    Logger.debug("password is"+password)
    val address = doc.getAs[Address]("address").get
    User(firstName, lastName, password,email , address)
  }
}


  implicit object User2Reader extends BSONDocumentReader[User2] {
    def read(doc: BSONDocument): User2= {
        val _id = doc.getAs[BSONObjectID]("_id").get
        Logger.debug("_id is"+_id)
        val id = doc.getAs[BSONNumberLike]("id").get.toInt
        Logger.debug("id is"+id)
        val user = doc.getAs[User]("user").get
        User2(id, user.firstName, user.lastName, user.password, user.email, user.address)
    }

Note in above example, how we reuse Address’s reader to create User and User’s reader to create User2.

 

Finally, now as we have the readers, we can change the Action code and tell the compiler to use our reader when fetching BSONDocument from database. Following is the new Action code.

def list = Action.async { request=> {
  Logger.debug("Listing all users")
  Logger.debug("opening database connection")
  val driver = new MongoDriver()
  val connection = driver.connection(List("localhost"))
  val db = connection.db("website-db")
  val collection = db.collection[BSONCollection]("users")

  val query = BSONDocument()
  val findCursor: Cursor[User2] = collection.find(query).cursor[User2]
  val findFutureList = findCursor.collect[List]()
  findFutureList.map((list:List[User2]) => {
    Logger.debug("list count:"+list.length)
    Ok(views.html.list(list))
  })
}
}

The list.scala.html also need to be changed. The argument passed to it is

@(list: List[models.User2])

If you now list the users, we will see them properly

Serialisation

Serialization is the process of creating a BSONDocument. It works opposite to deserialisation. We need to inherit from BSONDocumentWriter and override its ‘write’ method. The ‘write’ method takes the datatype we want to serialise and returns the corresponding BSONDocument. Add following two implementations in User2 object

implicit object AddressWriter extends BSONDocumentWriter[Address] {
  def write(address:Address): BSONDocument = {
    Logger.debug("serialising address:" +address)
    BSONDocument(
      "street" -> address.street,
      "country" -> address.country)
  }
}

implicit object User2Writer extends BSONDocumentWriter[User2] {
  def write(user2:User2): BSONDocument = {
    Logger.debug("serialising user2:" +user2)
    BSONDocument(
      "id" -> user2.id,
      "user" -> BSONDocument(
        "firstname" -> user2.firstName,
        "lastname" -> user2.lastName,
        "email" -> BSONArray(user2.email(0)),
        "password" -> user2.password,
        "address" -> AddressWriter.write(user2.address)
      )
    )
  }
}

Note: In above example, I didn’t write a writer for User (lazy!). Also, note how AddressWriter’s write method was used to serialise Address.

One last change we will have to make is in registrationRequest action. We can now use User2Writer.write to create document for User2

val newUserDoc = User2Writer.write(user)

We can now add and list the documents using our custom readers and writers

• Add a new user

• List to verify it is added correctly