Playing with JSON

Note: In this blog, a reference to a javascript has been made but WordPress does not allow Javascript on sites hosted on WordPress.com. In the HTML code, the places where you see script and /script and link in the code, enclose it with the HTML angle brackets <> and </>

You’ll see

script src="@routes.Assets.at("javascripts/jquery-1.9.0.min.js")" type="text/javascript" /script

replace with

<s ....type="text/javascript"> </s..>

Replace

link rel="stylesheet" type="text/css" href="@routes.Assets.at("stylesheets/bootstrap.min.css")"

with

<l  .... >

In this blog, I’ll discuss how to use JSON to send data from a client (a browser) to PlayFramework’s server-side application. I do not intend to cover advantages of JSON. Several other blogs on the internet cover that topic well in detail.

JSON is a format for storing and transporting data. JSON is often used when to exchange data between the client (browser) and web server. Simply put, we use JSON (Javascript Object Notation) to structure our data in a format. For example, say have created following user registration page. Once the user clicks ‘Sign Up’ button, the data entered by the user need to be sent to the server side application. One way to send the data could be to structure the data as per JSON notation.

json_user_registration

In the blog about Forms in Play, I discussed how to create HTML forms and map them to Form in Play. The examples in that blog used POST as a mechanism to send data to the server. In this blog, the data will first be converted into a JSON at the client side (using Javascript, JQuery) and will then sent to the server. We will still use POST to send the data but the representation of the data would be in JSON. First, we need to create the JSON representation of data entered by the user. It would be:

{

  "user": {

    "firstname":"John",

    "lastname":"Cena",

    "email":"jc@wwe.com",

    "password":"YouCantSeeMe",

    "address":{

        "street":"34 some street",

        "country":USA

    }
  }  
}

JSON representation of data consists of objects consisting of key-value pairs. In the example above “user” is a JSON object consisting of key-value pairs like firstname:John, email:jc@wwe.com.

The key in key-value is always of type string. Value could be string, number, another JSON object, a JSON array (array of JSON values), boolean (true/false) or null.

In above example, ‘user’ JSON object also contains another JSON object, address. The address JSON object contains key-value pairs like country:USA. In general, curly braces ‘{}’ denote a JSON object and colon ‘:’ denotes key-value pairs. Square brackets are used to create an array.

In the following example, email in user JSON object contains an array of strings.

{

  "user": {

    "firstname":"John",

    "lastname":"Cena",

    "email":["jc@wwe.com", "jc1@wwe.com"],

    "password":"YouCantSeeMe",

    "address":{

        "street":"34 some street",

        "country":USA
     }
  }
}

Coming back to the topic of this blog, we want to send data in a Form using JSON. We will use json package (play.api.libs.json) provided in play framework to handle JSON data. The package contains following data types which map to the data types of JSON standard.

  • JsObject – user and address in example above are JsObject
  • JsNull – for null JSON value
  • JsBoolean – for true/false values
  • JsNumber – for represent all types of numbers
  • JsArray – for array of values. Note that it is not necessary for all values in an array to be of same type. For example, following is a valid array
[ "some string", true, null, 4.4, "another string]
  • JsString – String data type

In addition, there are two internally used types in json package.

  • JsUndefined – to represent errors
  • JsValue – super-type of all other types

In the following example of json,

  • rootJson, location, residents(0) and residents(1) are Json objects (they have curly braces {}, signifying that it is a json object)
  • name is key, “Watership Down” is its value. Other keys are lat, long, age, role, fname, lname, married, srole. location and residents(0) and residents(1) are also keys
  • residents is an array consisting of two json objects (square brackets denote array, []. Indexes of objects are 0 and 1). The Jsons at index 0 and 1 are different.
rootJson = {
             "name" : "Watership Down",
             "location" : {
             "lat" : 51.235685,
             "long" : -1.309197
             },
             "residents" : [ {
               "name" : "Fiver",
               "age" : 4,
               "role" : null
             }, {
               "fname" : "Bigwig",
               "lname" : "giwgib",
               "married" : false,
               "srole" : "Owsla"
               } ]
      }

Getting started

The first few lines of code we need to write is of the HTML form showed earlier.

<pre class=”lang:html decode:true “>
 <!DOCTYPE html>
<html lang=”en” xmlns:font-variant=”http://www.w3.org/1999/xhtml”&gt;
<head>
<meta charset=”UTF-8″>
<meta http-equiv=”X-UA-COMPATIBLE” content=”IE=edge”>
<meta name=”viewport” content=”width=device-width,initial-scale=1″>
<title>HTML Test</title>
link rel=”stylesheet” type=”text/css” href=”@routes.Assets.at(“stylesheets/bootstrap.min.css”)”

<style type=”text/css”>

html, body {
height:100%;
margin:0;padding:0
}

.center-form {
width:100%;
margin:auto;
position:relative;
top:50%;
transform: translateY(-50%)
}

</style>

script src=”@routes.Assets.at(“javascripts/jquery-3.1.1.min.js”)” /script
script src=”@routes.Assets.at(“javascripts/bootstrap.min.js”)” /script

</head>
<body>

<!–empty column–>

</div>
<!– for medium and large screens,
Second row of Bootstrap grid contains the form for username and password. Total 3 columns (12/4). –>

<!–form–>

ID
First Name
Last Name
Password
Email
Confirm password
Street Name
Country

<button type=”submit” class=”btn btn-primary”>Sign Up</button>
</form></div>
<!–empty column–>

</div>
</div>
</body>
</html>

</pre>

The corresponding route, model and controller code are as follows:

routes

GET      /                          controllers.Application_Test.index

controllers

package controllers

import play.api.data.Form
import play.api.data.Form._
import play.api.data.Forms
import play.api.mvc.Controller
import play.api.mvc._
import play.api.data.Forms._
import play.api.data.validation.Constraints._
import models._
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.util.Try


object Application_Test extends Controller {
  def index = Action {
    Ok(views.html.HTMLTest(user2Form))
  }
}

Model

case class Address (street: String, country:String)

case class User2(
 firstName:String,
 lastName:String,
 password:String,
 email:List[String],
 address:Address
 )

Sending JSON from browser

Now that we have the basic HTML form, we want to send data from the form to the server in JSON format (instead of standard Form submission). To send JSON from browser, use following Jquery and Javascript code. You may write the code in a separate file (myScripts.js) and include a reference to the script in HTML code

  • Javascript/JQuery (myScripts.js)
The following code is picked from an example on Stack Overflow. It converts HTML form data into JSON
function objectifyForm(formArray) {//serialize data function

  returnArray = {};
  for (var i = 0; i < formArray.length; i++){
    returnArray[formArray[i]['name']] = formArray[i]['value'];
  }
  return returnArray;
}

$(document).ready(function(){
    // click on form submit
    $('#registration-form').on('submit',function(e){
    e.preventDefault();
    var details = JSON.stringify(objectifyForm($(this).serializeArray()));
    console.log(details)
        // send ajax
        $.ajax({
            url: '/newreg', // url where to submit the request
            type : "POST", // type of action POST || GET
            datatype : "json", // data type
            /* this doesn't work*/
            //contentType: "application/json; charset=utf-8",
            /*this does*/
            contentType: "text/json; charset=utf-8",
            data : details,
            success : function(result) {
                // you can see the result from the console
                // tab of the developer tools
                console.log(result);
                window.location = "/newreg-success"
            },
            error: function(xhr, resp, text) {
                console.log(xhr, resp, text);
                window.location = "/newreg-error"
            }
        })
    });
});

We will also need to include a reference for the script in HTML (at the bottom of HTML, just before tag)

script src="@routes.Assets.at("javascripts/myScripts.js")"> /script

Handling browser message submission on server-side (Action companion object)

Before getting into specifics of how JSON is processed in Play, I’ll spend some time to explain how messages from clients are processed in general. Play uses Action companion object to handling incoming HTTP requests.

Object Action extends ActionBuilder[Request]

The Action object takes a Request message (the complete HTTP request) and returns a Result message. A Request or Response can be used to access HTTP’s method, response code, Headers, body, URI etc.

Action has several ‘apply’ methods which provide flexibility in the way a message from the client can be processed.

  • Ignore message from client/browser
final def apply(block: ⇒ Result): Action[AnyContent]

Here, the incoming message from the browser is handled without processing the message itself. For example, if we define an Action as follows, we simply return a string back to the client without considering any data/message which the client might have sent.

def homePage = Action { /*no parameter to Action */
  Ok("you are at home page")
}

#Routes file
GET      /                          controllers.Application_Test.homePage

Action_No_Request_Message

  • Processing message from browser using AnyContent
final def apply(block: (Request[AnyContent]) ⇒ Result): Action[AnyContent]

The above way of using Action creates a Request depending on the Content-Type value in the incoming message. AnyContent is a trait provides methods that adapt request body based on Content-Type header in request. For example, if the content type is set as ‘application/x-www-form-urlencoded’, the Request’s body parameter would be created by creating an instance of AnyContentAsFormUrlEncoded case class which extends AnyContent trait. To see this in action, remove myScripts.js in HTML code and submit the form. Also make the following changes in routes and controller.

Add these line to routes file (remove homePage rule)

#GET      /                          controllers.Application_Test.homePage
GET       /                          controllers.Application_Test.index
POST      /newreg                    controllers.Application_Test.registrationRequest

We added newreg rule because when the above form is submitted to the server, newreg action is called on server (see jquery code above).

url: '/newreg', // url where to submit the request
  • RegistrationRequest controller code. This example also uses Logger to log/print on play console
import play.api.Logger

def registrationRequest = Action { request => { /*we use the version of Action in which we have access to incoming request*/
 Logger.debug("received message:"+request) Logger.debug("received message:"+request.body) Ok("success")
}
}
  • Output on console where you run play server
[debug] application - received message:POST /newreg
[debug] application - received message:AnyContentAsFormUrlEncoded(Map(email -> List(jc@wwe.com), country -> List(USA), first-name -> List(John), street-name -> List(some street), last-name -> List(Cena), password -> List(f)))

AnyContent supports multiple data formats like

  • application/x-www-form-urlencoded
  • text/json or application/json
  • multipart/form-data
  • Raw (Used when no Content-Type matches)
  • text/plain
  • application/xml

JSON is already in the list! Thus if we send Form’s data in JSON format, the server would accept and parse it. Before we test this, include myScripts.js in HTML and add following code in routes and controller and submit the form. The console output should look as follows:

GET     /newreg-success           controllers.Application_Test.registrationRequestSuccess
GET     /newreg-error           controllers.Application_Test.registrationRequestFail

We added newreg-success and newreg-fail rule because if JSON is handled successfully by server, we redirect to newreg-success page. Otherwise, the page is redirected to newreg-error

window.location = "/newreg-success"
/* code in controller*/
def registrationRequestSuccess = Action {request =>
 Ok(views.html.regconf("success"))
}

def registrationRequestFail = Action { request =>  Ok(views.html.regconf("Error"))}
[debug] application - received message:POST /newreg
[debug] application - received message:AnyContentAsJson({"first-name":"John","last-name":"Cena","email":"jc@wwe.com","password":"f","street-name":"somestreet","country":"USA"})

regconf.scala.html

 

<pre class=”lang:html decode:true “>

@(title:String)

<!DOCTYPE html>

<html>
<head>
<title>@title</title>
<link rel=”stylesheet” media=”screen” href=”@routes.Assets.at(“stylesheets/main.css”)”>
link rel=”shortcut icon” type=”image/png” href=”@routes.Assets.at(“images/favicon.png”)”
script src=”@routes.Assets.at(“javascripts/jquery-1.9.0.min.js”)” type=”text/javascript”/script
</head>
<body>
@if(title == “success”) {
<h1> Thanks for registering.</h1>
}else {
<h1> Error.</h1>
}

</body>
</html>

</pre>

As the body of Request is set according to the Content-Type header, to access elements of the Json message, we would asJson method of AnyContent trait which would give an Option[JsValue] (we discussed above that JsValue is super type of all Json types). JsValue provides methods (like \, \\, as, asOpt etc.) which can be used to access key-value from the Json. In previous example, we can access the key’first-name’ from Json as follows:

def registrationRequest = Action  /*(parse.json)*/ { request => {
  Logger.debug("received message:"+request)
  Logger.debug("received message:"+request.body)
  request.body.asJson.map (jsv => jsv \ "first-name").asOpt[String].map(nameValue=>{      Logger.debug("name is "+nameValue); Ok(nameValue)    }).getOrElse{BadRequest("no name")}  ).getOrElse {BadRequest("bad json")}
 }
}
[debug] application - received message:POST /newreg
[debug] application - received message:AnyContentAsJson({"first-name":"John","last-name":"Cena","email":"jc@wwe.com","password":"f","street-name":"somestreet","country":"USA"})
[debug] application - name is John

In above code:

  • method ‘\’ of JsValue is used to traverse the Json and extract value corresponding to ‘first-name’ key.
  • method asOpt is used to convert value returned by ‘\’ to a Scala value. In the example, ‘\’ would return JsString. asOpt would convert JsString to Option[String] or would return None in case of an error.  An alternative to asOpt[T] is ‘as[T]’ method but ‘as’ throws an exception instead or returning None.
  • the purpose of getOrElse is to send a BadRequest message if the message from browser contains bad JSON or if the JSON doesn’t contain ‘first-name’ key.
  • request.body.asJson gives Option[JsValue]. If Option is successful then map will be executed. Otherwise getOrElse will be executed returning BadRequest(“bad json”)
  • if Option[JsValue] is successful, it will give a JsValue (named jsv). We call ‘\’ method on jsv (JsValue).  ‘\’ method takes the keyand returns the corresponding value. If name is not present in JSON, we return BadRequest
  • If key is present, \ will return the corresponding value. We call asOpt on the value and parse it as a String. If parsing is successful, we print and return the value of ‘first-name’.
  • Using body parsers 

Action method also takes body parsers as argument. Using body parsers makes the code simple as they would correctly parse the content before we start handling the browser’s message in our code.

def apply[A](bodyParser: BodyParser[A])(block: (Request[A]) ⇒ Result): Action[A]

We can convert the above code in which we handled json by using a prebuilt json parser.

def registrationRequest = Action (parse.json) { request => {
  Logger.debug("received message:"+request)
  Logger.debug("received message:"+request.body)
  (request.body \ "first-name").asOpt[String].map(nameValue=>{
      Logger.debug("name is "+nameValue); Ok(nameValue)
    }).getOrElse{BadRequest("no name")}
 }
}
[debug] application - received message:POST /newreg
[debug] application - received message:{"first-name":"Bill","last-name":"Goldberg","email":"bg@wwe.com","password":"g","street-name":"some new street","country":"Trump's USA"}
[debug] application - name is Bill

Note how in above example, the body of request is already parsed as JsValue by parse.json. Thus we could directly call the ‘\’ method on body to extract ‘first-name’

JSON Object

Play framework provides a Json object which is has helper functions to handle JsValues. It is defined in play.api.libs.json.Json and provides methods to create and parse Json. Some of the methods are:

  • Json.parse:  parses a string to JsValue.
scala> val json: JsValue = Json.parse("""
 | {
 | "name" : "Watership Down",
 | "location" : {
 | "lat" : 51.235685,
 | "long" : -1.309197
 | },
 | "residents" : [ {
 | "name" : "Fiver",
 | "age" : 4,
 | "role" : null
 | }, {
 | "fname" : "Bigwig",
 | "lname" : "giwgib",
 | "married" : false,
 | "srole" : "Owsla"
 | } ]
 | }
 | """)
json: play.api.libs.json.JsValue = {"name":"Watership Down","location":{"lat":51.235685,"long":-1.309197},"residents":[{"name":"Fiver","age":4,"role":null},{"fname":"Bigwig","lname":"giwgib","married":false,"srole":"Owsla"}]}
  • Json.obj(): simplified syntax to create a JsObject. See example below.
  • Json.arr(): simplified syntax to create a JsArray. See example below.
  • Json.stringify or Json.prettyPrint: creates the string representation of a JsValue
scala> val json: JsValue = Json.obj(
| "name" -> "Watership Down",
| "location" -> Json.obj("lat" -> 51.235685, "long" -> -1.309197),
| "residents" -> Json.arr(
| Json.obj(
| "name" -> "Fiver",
| "age" -> 4,
| "role" -> JsNull
| ),
| Json.obj(
| "fname" -> "Bigwig",
| "lname" -> "giwgib",
| "married" -> false,
| "role" -> "Owsla"
| )
| )
| )
json: play.api.libs.json.JsValue = {"name":"Watership Down","location":{"lat":51.235685,"long":-1.309197},"residents":[{"name":"Fiver","age":4,"role":null},{"fname":"Bigwig","lname":"giwgib","married":false,"role":"Owsla"}]}

scala> Json.stringify(json)
res6: String = {"name":"Watership Down","location":{"lat":51.235685,"long":-1.309197},"residents":[{"name":"Fiver","age":4,"role":null},{"fname":"Bigwig","lname":"giwgib","married":false,"role":"Owsla"}]}

scala> Json.prettyPrint(json)
res7: String =
{
"name" : "Watership Down",
"location" : {
"lat" : 51.235685,
"long" : -1.309197
},
"residents" : [ {
"name" : "Fiver",
"age" : 4,
"role" : null
}, {
"fname" : "Bigwig",
"lname" : "giwgib",
"married" : false,
"role" : "Owsla"
} ]
}

scala>
  • Json.toJson: converts a Scala structure to a JsValue using the resolved implicit Writes[T]

Json.toJson[T](t: T)(implicit writes: Writes[T])

scala> import play.api.libs.json._
import play.api.libs.json._

scala> val doublejsv = Json.toJson(4.1)
doublejsv: play.api.libs.json.JsValue = 4.1
  • fromJson: converts a JsValue to a Scala structure using the resolved implicit Reads[T]

Json.fromJson[T](json: JsValue)(implicit reads: Reads[T])

scala> val d = Json.fromJson[Double](doublejsv)
d: play.api.libs.json.JsResult[Double] = JsSuccess(4.1,)

Extending the previous example, once we receive a request from the browser, we can return a JSON to the browser by combining the ‘first-name’ and ‘last-name’ field into a single field ‘name’.  The JSON is created using Json.obj as shown below (it is not important to understand how getFullName works. It is a helper function to extract first-name and last-name from the JSON.

/*parse.tolerantText return the Json in request as string in request's body*/
def registrationRequest = Action(parse.tolerantText) { request => {
  Logger.debug("received message:" + request)
  Logger.debug("received message:" + request.body)
  val parametersOfInterest:List[String]=List("first-name", "last-name");

/* request.body is a string (instead of AnyContentAsJson as was the case when parse.json body parser was used)*//*Json.parse would convert the Json string into JsValue*/
/* create json from string*/  val jsv = Json.parse(request.body)
  val v = getFullName(jsv, parametersOfInterest)
  v match {
    case None => BadRequest("no name")
    case Some(values) => {Logger.debug("name is " + values(0)+values(1));
      val json = Json.obj("name" -> (values(0)+" "+values(1)))
      Logger.debug(json.toString())
/* create string from Json*/Ok(Json.stringify(json))}
  }

  }
}

/* helper function to extract first-name and last-name*/
def getFullName(jsv: JsValue, poi:List[String]): Option[Vector[String]] = {

 /* in the inner recursive function, pass the keys still to be looked for and values found so far*/

 def inner(ijsv:JsValue,ikeysTBD:List[String], ivaluesSoFar:Vector[String]):Option[Vector[String]] = {
 ikeysTBD match { // see if we have looked for all names
 case Nil => Option(ivaluesSoFar) // we have looked for all parameters.
 //more keys to be found. 
 case key::t => (ijsv \ key).validate[String].map(value=> {inner(ijsv, t, ivaluesSoFar ++ Vector(value))}).getOrElse(None)
 }

 }
 inner(jsv,poi, Vector[String]())
}

Output
[debug] application - received message:POST /newreg
[debug] application - received message:{"first-name":"John","last-name":"Cena","email":"jc@wwe.com","password":"f","street-name":"somestreet","country":"usa"}
[debug] application - name is JohnCena
[debug] application - {"name":"John Cena"}

In the previous example, validate[T] method instead of asOpt[T] method is used to convert Json value to Scala value. ‘validate’ is a more robust method than asOpt as it provides better information in case of errors.

as, asOpt and validate

JsValue has few methods which convert the node (JsValue) into Scala value T. T could be a basic type like String, Int, Double or custom type (eg. a case class). All these methods rely on Reads to convert JsValue to T. These methods differ in way they handle errors if the conversion cannot be done (say due to wrong data type). Do not worry about implicit Reads declaration. Reads is covered later.

  • def as[T](implicit fjs: Reads[T]): T

Tries to convert the node into a T, throwing an exception if it can’t.

  • def asOpt[T](implicit fjs: Reads[T]): Option[T]

Does not throw an exception. Tries to convert the node into a T. Returns either Some[T] or None.

  • def validate[A](implicit rds: Reads[A]): JsResult[A]

Tries to convert the node into a JsResult[T] (Success or Error).

Note: Reads has an abstract ‘reads’ method which takes a Json and return the value

def reads(json: JsValue): JsResult[A]

In the following example, a JsValue is created from a Double. Afterward, as, asOpt and validate are used to convert the JsValue back to Scala type (Double and Int). Obviously, an error occurs when an attempt is made to convert the Double into an Int. The difference in ways error is handled by the different methods demonstrates the advantages of using ‘validate’ over ‘as’ and ‘asOpt’

scala> val doublejsv = Json.toJson(4.1)
doublejsv: play.api.libs.json.JsValue = 4.1

/* no problem with success scenario in 'as'*/
scala> doublejsv.as[Double]
res12: Double = 4.1

/* Using 'as', additional code will be needed to be written to handle exception*/
scala> doublejsv.as[Int]
play.api.libs.json.JsResultException: JsResultException(errors:List((,List(ValidationError(List(error.expected.int),WrappedArray())))))

/* no problem with success scenario in 'asOpt'*/
scala> doublejsv.asOpt[Double]
res14: Option[Double] = Some(4.1)

/*asOpt handles error elegantly by returning a None. 
However, it does not say what was the error*/

scala> doublejsv.asOpt[Int]
res15: Option[Int] = None

/*no problem with success in validate*/
scala> doublejsv.validate[Double]
res16: play.api.libs.json.JsResult[Double] = JsSuccess(4.1,)

/*validate handles error elegantly and also provides additional information about the error
The additional information tell us that validate[Int] expected an Int (but we used it with Double)*/
scala> doublejsv.validate[Int]
res17: play.api.libs.json.JsResult[Int] = 
    JsError(List((,List(ValidationError(List(error.expected.int),WrappedArray())))))

scala>

Custom Deserialising and Serialising JSON – Read[T], Write[T] and Format[T]

Deserializing is the processing of reading values from JSON. Serialization is creating a JSON. We have been using Serialization and Deserialization all along so far. Play already provides inbuilt methods to convert basic Scala types to JsValue and vice versa. In the example below, we can get the value of the key ‘id’ using ‘as’ method without specifying the Reads argument required by ‘as’.

{
  "id": 1,
  "user": {

    "first-name":"John",

    "last-name":"Cena",

    "email":["jc@wwe.com", "jc1@wwe.com"],

    "password":"YouCantSeeMe",

    "address":{

        "street":"34 some street",

        "country":"USA"
     }
  }
}
scala> val idv = (jsv \ "id").as[Int]
idv: Int = 1

However, how can we get the value of “user”? Note that “user” is not a basic type but is a JSON object composed of Strings, Array and another JSON object. One option is to specify extract each member of “user” JSON object by specifying complete path of each key in “user”

scala> val fnv = (jsv \ "user" \ "first-name").as[String]
fnv: String = John

scala> val lnv = (jsv \ "user" \ "last-name").as[String]
lnv: String = Cena

scala> val st = (jsv \ "user" \ "address" \ "street").as[String]
st: String = 34 some street

scala> val em = (jsv \ "user" \ "email")(0).as[String]
em: String = jc@wwe.com

Reading JSON and mapping it to model

A better approach is to use the basic types to create custom deserializer. For example, let us say we want to map “user” from above JSON to following User2 case class.

case class Address (street: String, country:String)

case class User2(
 id:Int,
 firstName:String,
 lastName:String,
 password:String,
 email:List[String],
 address:Address
 )

Note: If we were submitting the form in standard HTML Form format, then we could have used bindFromRequest to validate that the data is correct. Now that we are using JSON, we would have to write the code to validate the JSON.

In general, the process of Deserializati0n involves following steps

  1. We need the JSON (JsValue) to read from (obviously!). It could be a simple JSON or a complex one (consisting of other JSON objects and arrays)
  2. You need to specify the property (key) you want to read (i.e. get the value of a key)
  3. You need to specify how to read the value (i.e. whether the value should be treated as a string, number etc.)
  4. If reading multiple keys, use combinators like ‘and’.

Let us write code for each of the steps above. I’ll also write the steps in REPL so that we can see intermediate results.

1. We get the JSON in the request.body if we use parse.json

def registrationRequest = Action(parse.json) { request => {...}}

We can create the JSON in REPL by typing following command

scala> val jsv=Json.parse("""
 | {
 | "id": 1,
 | "user": {
 |
 | "first-name":"John",
 |
 | "last-name":"Cena",
 |
 | "email":["jc@wwe.com", "jc1@wwe.com"],
 |
 | "password":"YouCantSeeMe",
 |
 | "address":{
 |
 | "street":"34 some street",
 |
 | "country":"USA"
 | }
 | }
 | }
 | """)
jsv: play.api.libs.json.JsValue = {"id":1,"user":{"first-name":"John","last-name":"Cena","email":["jc@wwe.com","jc1@wwe.com"],"password":"YouCantSeeMe","address":{"street":"34 some street","country":"USA"}}}

2. We need to specify the keys we want to read. Earlier we used JsValue’s ‘\’ method. To create custom deserializer, we will use JsPath. A JsPath represents the location of data in a JsValue structure. For example, to get the value of ‘last-name’ key, we use ‘\’ method of JsPath to specify the path to ‘first-name’ key as follows:

scala> val fnPath = JsPath \ "user" \ "first-name" 
lnPath: play.api.libs.json.JsPath = /first-name

Note: JsPath’s ‘\’ method returns another JsPath.

Note: An alias of JsPath is two underscores (__). For example

scala> __ \ "user" \ "first-name"
res17: play.api.libs.json.JsPath = /user/first-name

3. We have got the path to the key we want to read. Now we will specify how to interpret the value (a number or String or boolean etc.). We will use JsPath’s ‘read’ method to do so.

scala> val lnReads = fnPath.read[String]
lnReads: play.api.libs.json.Reads[String] = play.api.libs.json.Reads$$anon$8@1988737b

4. As “user” consists of other JSON objects and types, we use combinators (‘and’) to extract all the keys.

Import package to use ‘and’ combinator

/*We will need to import the following package to use combinators*/
import play.api.libs.functional.syntax._

User2 consists of Address JSON object. First, create Reads for Address. Address consistes of street (a String) and a country (a string).

scala> implicit val addressRead:Reads[Address] =(
| (JsPath \ "street").read[String] and
| (JsPath \ "country").read[String]
| )(Address.apply _)
addressRead: play.api.libs.json.Reads[Address] = play.api.libs.json.Reads$$anon$8@10d8b24d

Note above that the path of “street” and “country” keys does not refer to “user” i.e. it is not “user” \ “address” \ “street”.

Now create Reads for User2. Two things to notice here is the way email (which is a List) is read and they way previously defined Address’ read is used

scala> implicit val user2Read:Reads[User2] = (
| (JsPath \ "id").read[Int] and
| (JsPath \ "user" \ "first-name").read[String] and
| (JsPath \ "user" \ "last-name").read[String] and
| (JsPath \ "user" \ "password").read[String] and
| (JsPath \ "user" \ "email").read[List[String]] and
| (JsPath \ "user" \ "address").read[Address]
| )(User2.apply _)
user2Read: play.api.libs.json.Reads[User2] = play.api.libs.json.Reads$$anon$8@4ea864b7

At this point, we will have an implicit Reads[User2] which can be used by ‘as’, ‘asOpt’ and ‘validate’. That was our objective – provide a custom Reads which can be used by ‘as’, ‘asOpt’ and ‘validate’ to convert JsValue to User2.

 scala> jsv.validate[User2]
res21: play.api.libs.json.JsResult[User2] = JsSuccess(User2(1,John,Cena,YouCantSeeMe,List(jc@wwe.com, jc1@wwe.com),Address(34 some street,USA)),)

The above examples are for REPL. Following would be the code of controller file for Play web application.

implicit val addressRead:Reads[Address] =(
  (JsPath \ "street").read[String] and
    (JsPath \ "country").read[String]
  )(Address.apply _)

implicit val user2Read:Reads[User2] = (
  (JsPath \ "id").read[Int] and
    (JsPath \ "user" \ "first-name").read[String] and
    (JsPath \ "user" \ "last-name").read[String] and
    (JsPath \ "user" \ "password").read[String] and
    (JsPath \ "user" \ "email").read[List[String]] and
    (JsPath \ "user" \ "address").read[Address]
  )(User2.apply _)

def registrationRequest = Action(parse.json) { request => {
  Logger.debug("received message:" + request)
  Logger.debug("received message:" + request.body)
  val jr:JsResult[User2] = request.body.validate[User2]
  Logger.debug( "jr is "+jr)
  Ok("success")
  }
}

If you try running above code, you’ll see success message on submitting the form but the output on Play’s console would be an Error. This is because the above code does not look into value of ‘jr’ (the result of validate) and returns success (Ok).

[localhost:27017]
[info] play - Application started (Dev)
[debug] application - received message:POST /newreg
[debug] application - received message:{"id":"123","first-name":"John","last-name":"Cena","password":"f","email":"jc@wwe.com","street-name":"new road","country":"uk"}
[debug] application - jr is JsError(List((/user/password,List(ValidationError(error.path.missing,WrappedArray()))), (/user/address,List(ValidationError(error.path.missing,WrappedArray()))), (/id,List(ValidationError(error.expected.jsnumber,WrappedArray()))), (/user/first-name,List(ValidationError(error.path.missing,WrappedArray()))), (/user/last-name,List(ValidationError(error.path.missing,WrappedArray()))), (/user/email,List(ValidationError(error.path.missing,WrappedArray())))))

The reason ‘validate’ fails is because the JSON sent by browser is

{

"id":"123",

"first-name":"John",

...

}

But the server application expects

{

"id":"123",

"user": {

"first-name":"some name",

...

}

The JsError returned by ‘validate’ tells about all the errors. It expects /user/password, /user/address, /user/first-name, /user/last-name, /user/email but the client’s JSON contains /password, /address, /first-name, /last-name, /email. Also the id is sent as String while server side expects an Integer (Number).

JsError(List((/user/password,List(ValidationError(error.path.missing,WrappedArray()))), 
(/user/address,List(ValidationError(error.path.missing,WrappedArray()))), 
(/id,List(ValidationError(error.expected.jsnumber,WrappedArray()))), 
(/user/first-name,List(ValidationError(error.path.missing,WrappedArray()))),
 (/user/last-name,List(ValidationError(error.path.missing,WrappedArray()))), 
(/user/email,List(ValidationError(error.path.missing,WrappedArray())))))

To properly fix the code, we will need to (a) check for errors and send the errors back to the client if the JSON sent by the client isn’t correct (b) (a) send correct JSON from the client.

‘validate’ returns JsResult. JsResult could be either JsSuccess or JsError.

  • JsSuccess[T] represents a successful validation/conversion and wraps the result.
  • JsError represents unsuccessful validation/conversion and contains a list of validation errors.

Thus, the output of ‘validate’ could be checked as follows:

val jr:JsResult[User2] = request.body.validate[User2]

jr match {
  case s:JsSuccess[User2] => {Logger.debug("success"); Ok("success")}
  case f:JsError => {Logger.debug("error"); BadRequest("invalid json: "+f.errors)}
}

Extending validation

The validation written above checks for type of data. If can, for example, validate that JSON has password field and it has type String. What if, we also want to validate that password consists of a minimum of 8 characters, or has a size of at least 8 characters. How can we check that the first-name and last-name consists of only alphabets. Such validations can be done as well. They are a bit complicated esp. if you try to understand how the flow of code is working (i.e. how our logic gets called and by who). This complication occurs because JsPath’s ‘read’ method uses an implicit Reads and creates Reads. Somewhere in the code, ‘reads’ of Read get called (note, Read has ‘reads’ method, JsPath has ‘read’ method. Already confusing?). I couldn’t figure out how the flow of code works but I figure that we can assume that the things will work somehow! Look at following example of how to extend previous validation code

To check that ‘password’ is a String, we would write

(JsPath \ "user" \ "password").read[String]

In general, read takes an implicit ‘Reads’ object as an argument.

def read[T](implicit r: Reads[T]): Reads[T]

In above example Play already provides it so we do not need to pass any argument to ‘read’. To create our custom validation function, we need to supply ‘r’ of type Reads. ‘r’ could be of type Reads( val validation:Reads) or something which generates Reads (def validation => Reads or def validation(arguments)=>Reads). Before writing the validation code, it is worth mentioning about Reads object. As the default validation for Reads is minimal, such as checking for type conversion errors, Play provides Reads object with validation helpers. Here are some that are commonly used:

  • Reads.email – Validates a String has email format.
  • Reads.minLength(nb) – Validates the minimum length of a String.
  • Reads.min – Validates a minimum numeric value.
  • Reads.max – Validates a maximum numeric value.

If the additional validation on password was for minimum size then we could have written it as follows

(JsPath \ "user" \ "password").read[String](minLength[String](2))

The above code confused me for some time. Essentially, it is nothing but calling read and passing minLength to it (minLength is the ‘r’ argument).

minLength is a helper function already provided in Play. Our own helper function can be created as follows (say we want ‘password’ to contain lowercase letters only).

1. Create something of type Reads or which returns Reads

def checkPasswordCase: Reads[String] = {
  Reads.StringReads.filter(ValidationError("Password not in lowercase"))(str => {
    (str.matches("""[a-z]+"""))
  })
}

2. pass it to read method

(JsPath \ "user" \ "password").read[String](checkPasswordCase)

If we want the function to check minimum length as well, we could have done following

def checkPasswordLength(min:Int): Reads[String] = {
  Reads.StringReads.filter(ValidationError("Invalid password length"))(str => {
    (str.length >= min)
  })
}
(JsPath \ "user" \ "password").read[String]((checkPasswordCase) andKeep (checkPasswordLength(8))) and

We know that Reads has an abstract ‘reads’ method. In above code,we have not defined ‘reads’. So how does the above code work? Also, is the value of key JsPath \ “user” \ “password” is accessible to us in our validation code (checkPasswordCase and checkPasswordLength methods) as ‘str’?

  • (JsPath \ “user” \ “password”) returns a new JsPath
  • The new JsPath’s read method calls Reads.at defined in PathReads trait
def read[T](implicit r: Reads[T]): Reads[T] = Reads.at[T](this)(r)
  • Reads.at call’s our validation code. It is passed the value returned by asSingleJsResult (which would be the value of key (JsPath \ “user” \ “password”). It is denoted by _ in reads.reads(_)
def at[A](path: JsPath)(implicit reads: Reads[A]): Reads[A] =
Reads[A](js => path.asSingleJsResult(js).flatMap(reads.reads(_).repath(path)))
  • The call to reads.reads(_) is actually 
    (Reads.StringReads.filter(ValidationError("Invalid password length"))(str => {
    (str.length >= min)).reads(_)

    ‘r’ is either checkPasswordCase or checkPasswordLength. They both use StringReads. StringReads’ ‘reads’ method is defined as:

implicit object StringReads extends Reads[String] {
 def reads(json: JsValue) = json match {
 case JsString(s) => JsSuccess(s)
 case _ => JsError(Seq(JsPath() -> Seq(ValidationError("error.expected.jsstring"))))
 }
}

The implementation of StringReads has ‘reads’ defined in which it checks that the json value passed to it is a String. Then filter method is called. It expects a method of type str=>Boolean. The string value extracted by StringReads is passed to filter.

def filter(error: ValidationError)(f: (String) ⇒ Boolean): Reads[String]

In summary, StringReads implements the ‘reads’ method. It is passed the JSON value of key (JsPath \ “user” \ “password”). It checks that the value is of type String. Then filter method executes our validation logic and returns another ‘Reads`. So, in other words, we are just “transforming” (filtering) an existing Reads which already has ‘reads’ implemented.

Writing JSON from Model

Serialization is the reverse of De-serialization. ‘Writes’ is used to serialize a type to JsValue. Here we convert a model (say our case class) into JSON. Say, we want to send an acknowledgment to the browser after the request from the browser has been processed. The acknowledgment could be represented by following case class.

case class Acknowledgement (
                           status:Int, //0 means error, anything else is success
                           message:String // additional information accompanying  status
                           )

The acknowledgment would tell the browser whether the request was processed successfully and additional information (for example, all the errors in case the request could be processed). We want to send the response in following JSON format:

//for success
{
  "status": 1,
 "message": List("Thanks for registering John Cena")
}

//for failure
{
 "status":0,
 "message":{
             "obj.user.password" : [ {
                "msg" : "error.path.missing",
                "args" : [ ]
             } ],
             "obj.user.address" : [ {
             "msg" : "error.path.missing",
             "args" : [ ]
             } ],
             "obj.id" : [ {
             "msg" : "error.expected.jsnumber",
             "args" : [ ]
             } ],
             "obj.user.first-name" : [ {
             "msg" : "error.path.missing",
             "args" : [ ]
             } ],
             "obj.user.last-name" : [ {
             "msg" : "error.path.missing",
             "args" : [ ]
             } ],
             "obj.user.email" : [ {
             "msg" : "error.path.missing",
             "args" : [ ]
             } ]
          }
}

Status 0 means error. Status non-zero means success. The message is a List of String. In case of error, the message contains another JSON object containing detail of errors. The JSON representing error looks complicated but it is easy to implement as Play provides a JsError object whose toFlatJson method can straightway convert JsError to Json.

Just like ‘read’ was used to read the JsPath, ‘write’ is used to create a JsPath.

implicit val ackWrites:Writes[Acknowledgment] = (
  (JsPath \ "status").write[Int] and
    (JsPath \ "message").write[String]
)(unlift(Acknowledgment.unapply))

The example above creates a Writes which would be used to convert an Acknowledgment data type to JSON. The Integer value in Acknowledgement would be written as value of key “status”. The “message” key will have String as value.

The controller code which sends success or failure as JSON looks like follows:

def registrationRequest = Action(parse.json) { request => {
    Logger.debug("received message:" + request)
    Logger.debug("received message:" + request.body)
    val jr:JsResult[User2] = request.body.validate[User2]
    Logger.debug( "jr is "+jr)

    jr match {
      case s:JsSuccess[User2] => {
      /*success - get the user*/
        val user = s.get
      /*create ack oject*/
        val ack = Acknowledgment(1,"Welcome "+user.firstName+" "+user.lastName)
        Logger.debug("success ack Json:"+Json.toJson(ack))
      /*convert to Json using Writes[Acknowledgment]*/
          Ok(Json.toJson(ack))
      }
      case f:JsError => {
        Logger.debug("error: "+JsError.toFlatJson(f))
       /*user JsError object to convert JsError to Json*/
        val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
        Logger.debug("fail ack:"+Json.toJson(ack))
       /*convert to Json using Writes[Acknowledgment]*/
        BadRequest(Json.toJson(ack))
      }
    }
  }
}

To successfully register, we will have to send correct JSON request from client. Change Myscripts.js to following to send correct JSON. The code selects elemtents from the form, create Javascript object by inserting the elements at correct location and then uses Stringify method to convert to JSON.

 function createAddressObject(form){
    var addressObject = {};
   // var jqf = $(form);

    var street = $('#registration-form [name=street]')
    var sname = street.attr('name');
    var sval =street.val();
    console.log('street found'+sname+sval);
    addressObject[sname]=sval;

    var country = $('#registration-form [name=country]')
    var cname = country.attr('name');
    var cval =country.val();
    console.log('country found'+cname+cval);
    addressObject[cname]=cval;
     return addressObject ;
 }

 function createUserObject(form){
  var userObject = {};
 // var jqf = $(form);

 var fn= $('#registration-form [name=first-name]')
         var fnn = fn.attr('name');
         var fnval =fn.val();
         console.log('fn found'+fnn+fnval);
         userObject[fnn]=fnval;

 var ln= $('#registration-form [name=last-name]')
         var lnn = ln.attr('name');
         var lnval =ln.val();
         console.log('ln found'+lnn+lnval);
         userObject[lnn]=lnval;

 var em= $('#registration-form [name=email]')
         var email = [];
         var emn = em.attr('name');
         var emval =em.val();
         console.log('email found'+emn+emval);
         email[0]=emval;
         userObject[emn]=email;

 var pwd= $('#registration-form [name=password]')
         var pwdn = pwd.attr('name');
         var pwdval =pwd.val();
         console.log('pwd found'+pwdn+pwdval);
         userObject[pwdn]=pwdval;
 userObject['address']=createAddressObject(form);
   console.log('userObject: '+userObject)
   return userObject ;
}

 function objectifyForm2(form){
  var formObject = {};
  //var jqf = $(form);

  var id = $('#registration-form [name=id]')
     console.log('id found'+id);
     formObject[id.attr('name')]=Number(id.val());

  formObject['user']=createUserObject(form);



  return formObject;

  }

 $(document).ready(function(){
     // click on form submit
     $('#registration-form').on('submit',function(e){
     e.preventDefault();
    /*
    The .serializeArray() method uses the standard W3C rules for successful controls to determine which elements
    it should include; in particular the element cannot be disabled and must contain a name attribute.
    Encode a set of form elements as an array of names and values.
    */
    var details2 = JSON.stringify(objectifyForm2(this));
     //var details = JSON.stringify(objectifyForm2(this);
     console.log('details2: '+details2)
         // send ajax
         $.ajax({
             url: '/newreg', // url where to submit the request
             type : "POST", // type of action POST || GET
             datatype : "json", // data type
             /* this doesn't work*/
             //contentType: "application/json; charset=utf-8",
             /*this does*/
             contentType: "text/json; charset=utf-8",
             data : details2,
             success : function(result) {
                 // you can see the result from the console
                 // tab of the developer tools
                 console.log(result);
                 window.location = "/newreg-success"
             },
             error: function(xhr, resp, text) {
                 console.log(xhr, resp, text);
                 window.location = "/newreg-error"
             }
         })
     });
 });

Leave a comment