Minimize Json
TL;DR:
Minimize Json Answer by minimizing the names of the properties transmitted. As a practical example, see http://jsonminimize.apphb.com/ ( although is gzipped , the answers are 3862 versus 3089 bytes – an 20% gain . Your results will vary – because of how I obtain the random obtaining of data )
Prerequisites:
Ajax / Json – http://msprogrammer.serviciipeweb.ro/2011/12/05/jquery-ajax-request-and-mvcdetailed/
AutoMapper – for automating transformation on server code of an object to another : http://automapper.org/
Mandatory for source code: Knockout – just because I needed a templating engine for javascript
Introduction
Let’s say you have an application that shows a report of something( for my example, departments and employees). Let’s say that you have optimized database for speed , the network connection, the data you load, everything.
However, because the end user wants to retrieve entire data, the page loads in 3 seconds. And this is not acceptable.
Looking to the source, you see that thepage makes one Json request that retrieves all data. You can not minimize this data. You tell the user that you can make several JSon request – but he will experience some delays. The user does not want to do this.
You look deeper into JSon answer and see that response from JSon:
{"ok":true,"departments":[{"IdDepartment":0,"NameDepartment":"Department 0","Employees":[{"FirstName":"Andrei","LastName":"Ignat","Login":"Andrei.Ignat","DateOfBirth":"\/Date(-62135596800000)\/","ManagerName":"Knowledge"},{"FirstName":"0 JkerzEZgv","LastName":"0 DjYFekyLF","Login":"0 JkerzEZgv.0 DjYFekyLF","DateOfBirth":"\/Date(24019200000)\/","ManagerName":"manager 0"},{"FirstName":"0 SlBKASBb","LastName":"0 rYGQXWQ","Login":"0 SlBKASBb.0 rYGQXWQ","DateOfBirth":"\/Date(21686400000)\/","ManagerName":"manager 1"}]},{"IdDepartment":1,"NameDepartment":"Department 1","Employees":[{"FirstName":"1 JkerzEZgv","LastName":"1 DjYFekyLF","Login":"1 JkerzEZgv.1 DjYFekyLF","DateOfBirth":"\/Date(24019200000)\/","ManagerName":"manager 0"},{"FirstName":"1 SlBKASBb","LastName":"1 rYGQXWQ","Login":"1 SlBKASBb.1 rYGQXWQ","DateOfBirth":"\/Date(21686400000)\/","ManagerName":"manager 1"}]},{"
What if , instead of “IDDepartment” the server transmits just “I” ? And if , instead of “NameDepartment” just “N” ? The response will be minimized! However, you do not want to affect the page source, so this article is about.
Original Application
The original application is displaying the departments and theirs employees. It is made with Knockout for templating and have this code ( oversimplified for convenience) :
$.ajax({
//code
success: function (returndata) {
if (returndata.ok) {
window.alert("Departments transferred: " + returndata.departments.length);
ko.applyBindings(returndata);
//code
and the knockout templating engine will interpret the templating
<div data-bind=”foreach: departments”>
<h3 data-bind=”text: NameDepartment”><h3>
The steps:
1. Create 2 new classes , DSmall and ESmall with properties names smaller.
2.Map the properties with a Dictionary from Department to DSmall and from Employee to ESmall
3. Map the list of Department to list of DSmall and list of Employees to ESmall (using AutoMapper and the dictionaries from 2)
4. Verify with an console example
5. Transmit over JSon the DSmall and ESmall.
6. In javascript recreate the Department and Employees from DSmall and ESmall.
7.Results
8.Homework
Create 2 new classes , DSmall and ESmall with properties names smaller.
The Departmen have a property named IDDepartment. DSmall will have a property named I. And so on – see below:
public class Department{
public Department()
{
Employees = new List<Employee>();
}
public int IdDepartment { get; set; }
public string NameDepartment { get; set; }
public List<Employee> Employees { get; set; }
}
|
public class DSmall
{
public int I { get; set; }
public string D { get; set; }
public List<ESmall> E{ get; set; }
}
|
public class Employee
{
public string FirstName { get; set; }
public string LastName { get; set; }
public string Login
{
get
{
return FirstName + "." + LastName;
}
}
public DateTime DateOfBirth { get; set; }
public string ManagerName { get; set; }
}
|
public class ESmall
{
public string F { get; set; }
public string L { get; set; }
public string Lo { get; set; }
public DateTime D{ get; set; }
public string M{ get; set; }
}
|
Map the properties with a Dictionary from Department to DSmall and from Employee to ESmall
As I said,the Departmen have a property named IDDepartment. DSmall will have a property named I. And so on. So I came up with this for department:
dDep = new Dictionary<string, string>();
dDep.Add("I", "IdDepartment");
dDep.Add("D", "NameDepartment");
dDep.Add("E", "Employees");
and for converting Employees to ESmall
dEmp = new Dictionary<string, string>();
dEmp.Add("F", "FirstName");
dEmp.Add("L", "LastName");
dEmp.Add("Lo", "Login");
dEmp.Add("D", "DateOfBirth");
dEmp.Add("M", "ManagerName");
The dictionaries will be good also for transmitting to javascript the names in order to not change the names of the properties in templating.
Map the list of Department to list of DSmall and list of Employees to ESmall (using AutoMapper and the dictionaries from 2)
This is the easiest line:
Utils<Employee, ESmall>.CreateMap(dEmp);
Utils<Department, DSmall>.CreateMap(dDep);
I invite you to read the code on utils class. It involves some lambda programming. I will not detail here - just grab the source from https://github.com/ignatandrei/MVCMinimizeJson/
<p> </p> <p> </p> <h3> Verify with an console example </h3> <p> </p>
I have made an console example to show the difference of serializing DSmall and Department. The code does not involve database - just loads some random data.
I have serialized with XMLSerializer and Javascript serializer 99 Department with random employees and then transforming to DSmall and ESmall
#region xml
var ser = new XmlSerializer(typeof(List<Department>));
var sb = new StringBuilder();
using (var sw = new StringWriter(sb))
{
ser.Serialize(sw, d);
}
// Console.WriteLine(sb.ToString());
var nrXML = sb.Length;
#endregion
#region javascript
var jss = new JavaScriptSerializer();
int nrJss = jss.Serialize(d).Length;
#endregion
For javascript the result is impressive: 30%. ( as I say, because of the random, your result may vary a little . Do your research)
Transmit over JSon the DSmall and ESmall.
This was the easiest part. I just put the random data into appliaction( I know, I know... but it was easier). And I have created 2 actions
public JsonResult RequestEmployees()
{
//usually this will be loading from database
//but to compare same data when minimizing,
//I have to put somewhere
List<Department> d = this.HttpContext.Application["data"] as List<Department>;
return Json(new { ok = true, departments = d });
}
public JsonResult RequestEmployeesMinimizing()
{
//usually this will be loading from database
//but to compare same data when minimizing,
//I have to put somewhere
List<Department> d = this.HttpContext.Application["data"] as List<Department>;
var min = LoadData.Minimize(d);
return Json(new { ok = true, departmentsMin = min, LoadData.dDep, LoadData.dEmp });
}
As you see, when transmitting the minimizing version of data, we will send also the dDep ( dictionary department) and dEmp( dictionary employees) of mapped properties to the View - in order to re-instantiate the original Employee and Department from javascript
In javascript recreate the Department and Employees from DSmall and ESmall.
We must do this in order to not modify the template that we already have.So I have created a simple mapping function in Javascript:
function createObjectMapped(minObject, dictProp) {
var obj = new Object();
for (var prop in dictProp) {
obj[dictProp[prop]] = minObject[prop];
}
return obj;
}
and use like this:
returndata.departments = [];
$.each(returndata.departmentsMin,function(index, departmentMin)
{
var dep = createObjectMapped(departmentMin, returndata.dDep);
//mapping employees of the department
dep.Employees = [];
$.each(departmentMin.E, function (index, employeeMin) {
dep.Employees.push(createObjectMapped(employeeMin, returndata.dEmp));
});
returndata.departments.push(dep);
});
7.Results
In Fiddler, for 99 departments with random employees, the original data is 77.765 bytes . The minimized data is 55.796 bytes. My gaining was 28%
8.Homework
Do the same when sending json bulk data - see http://msprogrammer.serviciipeweb.ro/2014/01/05/javascript-mvvm-and-asp-net-mvc/
Source code at https://github.com/ignatandrei/MVCMinimizeJson/
View online at http://jsonminimize.apphb.com/
The original problem and idea was coming from Adrian Petcu Thank you !
Leave a Reply