神刀安全网

JSON Compression by Rotating Data 90°

I have been researching various ways to make small changes to how JSON is sent using APIs without introducing in-line or substitution compression variants.

I have worked out a simple technique to provide huge lossless compression ratios when providing JSON objects.

The Basics

With a standard API return object you often send multiple records with the same key names. This is essentially extra overhead as you must repeat this for as many results you are going to include in the object array.

What if we could provide the same information but in a different way that no longer includes a key name for every key value within an object array but kept it just as accessible?

[     { "firstname": "Tim", "lastname": "Cook", "company": "Apple" },     { "firstname": "Satya", "lastname": "Nadella", "company": "Microsoft" },     { "firstname": "Sundar", "lastname": "Pichai", "company": "Google" } ]

Example JSON with 3 object arrays – 225 bytes

{     "firstname": [ "Tim", "Satya", "Sundar" ],     "lastname": [ "Cook", "Nadella", "Pichai" ],     "company": [ "Apple", "Microsoft", "Apple" ] }

Converted version of the same data – 152 bytes

Results

3 record comparison
Original 225 bytes
Compressed 152 bytes
Saving 73 bytes
Percentage 32%

To test the technique with more data, I duplicated the existing data 5 times to give our new sample set of data 15 records.

If we then repeat the process – the results are even more impressive.

15 record comparison
Original 1,113 bytes
Compressed 496 bytes
Saving 617 bytes
Percentage 55%

Accessing the data

You can access the data almost exactly as you did before, simply shift the position of the index look-up. Example; a simple full name alert:

alert(data[0].firstname + ' ' + data[0].lastname);

Original

alert(data.firstname[0] + ' ' + data.lastname[0]);

Compressed

Both of these examples provide the same information but only require minor changes to the data structure at both ends.

In order to loop the data you need to find the length – this is also slightly different:

var rows = data.length;

Original

var rows = data.firstname.length;

Compressed

Tip: you can query any root node as each will return the same length answer.

Further Compression Options

There are other common ways to compress data:

  • White-space removal
    Obvious, but not always followed
  • Shortening key names
    Less used but very effective
    • firstname › f
    • lastname › l
    • company › c

Bandwidth reduction requires suitable time to plan and manage, but can be very effective especially when you include this new technique.

JSON savings when combined with other techniques:

Method Raw White Space Shortening Both
Original 1,113 bytes 916 bytes 798 bytes 601 bytes
Compressed 496 bytes 425 bytes 475 bytes 404 bytes
Saving 617 bytes 491 bytes 323 bytes 197 bytes
Percentage 55% 54% 40% 33%

Keep in mind all these tests are like for like.

If we now look at our original source data compared to our compressed version using; JSON, white space removal and node name shortening then we end up with a 46% saving overall for just 15 records.

All Techniques Combined

Final savings
Original 1,113 bytes
Compressed 404 bytes
Saving 709 bytes
Percentage 64%

A total saving of 64% , not bad considering all the data is as perfectly readable as it was to start with.

If you only used the white-space and key name shortening you could save 46%. But when you can gain an additional 28% compression with little effort – why not.

One word of caution, this works best with object arrays that have consistent node patterns. You can provide a null for a key value not provided in the original object of course. But keep in mind that too much of this will reduce the benefits of this technique.

Source Data

[     { "firstname": "Tim", "lastname": "Cook", "company": "Apple" },     { "firstname": "Satya", "lastname": "Nadella", "company": "Microsoft" },     { "firstname": "Sundar", "lastname": "Pichai", "company": "Google" },     { "firstname": "Tim", "lastname": "Cook", "company": "Apple" },     { "firstname": "Satya", "lastname": "Nadella", "company": "Microsoft" },     { "firstname": "Sundar", "lastname": "Pichai", "company": "Google" },     { "firstname": "Tim", "lastname": "Cook", "company": "Apple" },     { "firstname": "Satya", "lastname": "Nadella", "company": "Microsoft" },     { "firstname": "Sundar", "lastname": "Pichai", "company": "Google" },     { "firstname": "Tim", "lastname": "Cook", "company": "Apple" },     { "firstname": "Satya", "lastname": "Nadella", "company": "Microsoft" },     { "firstname": "Sundar", "lastname": "Pichai", "company": "Google" },     { "firstname": "Tim", "lastname": "Cook", "company": "Apple" },     { "firstname": "Satya", "lastname": "Nadella", "company": "Microsoft" },     { "firstname": "Sundar", "lastname": "Pichai", "company": "Google" } ]

Original data-set of 15 records.

{"f":["Tim","Satya","Sundar","Tim","Satya","Sundar","Tim","Satya","Sundar","Tim","Satya","Sundar","Tim","Satya","Sundar"],"l":["Cook","Nadella","Pichai","Cook","Nadella","Pichai","Cook","Nadella","Pichai","Cook","Nadella","Pichai","Cook","Nadella","Pichai"],"c":["Apple","Microsoft","Apple","Apple","Microsoft","Apple","Apple","Microsoft","Apple","Apple","Microsoft","Apple","Apple","Microsoft","Apple"]}

Compression using my rotation technique plus white-space and key name shortening.

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » JSON Compression by Rotating Data 90°

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址