GitLocker: The Coding Marketplace

Description:

kson 0.1.0

KSON: Keystripped Schemafied Object Notation============================================KSON is a simple data interchange format based on JSON. Its serializedrepresentation doesn't contain the redundant keys of typical json documents and more compact representations of values are possible throughan extensible encoding/decoding mechanism.KSON uses a simple schema format to describe arbitrarrily nested objectsand encoded types. The minified and gzipped javascript library is < 1K.Usage in javascript------------------- // Load schema definitions var movie_schemas = [ { "id": "role", "fields": ["name", "character"], "meta": [0, 0] },{ "id": "movie", "fields": ["title", "year", "rating", "cover", "actors"], "meta": [0, 0, 0, "prefix:http\://movies.db/covers/|suffix.jpg", "[]role"] } ]; KSON.addSchema(movie_schemas); var movies = [ { "title": "Forrest Gump", "year": 1994, "rating": 8.7, "cover": "http://movies.db/covers/8.jpg", "actors": [ {"name": "Tom Hanks", "character": "Forest Gump"}, {"name": "Robin Wright", "character": "Jenny Curran"}, {"name": "Gary Sinise", "character": "Lieutenant Dan Taylor"} ] }, { "title": "Toy Story", ... } ... ]; // pretty printing is not part of the library console.log(KSON.stringify(movies, "[]movie")); [ "[]movie", "Forest Gump", 1994, 8.7, "8", [ ["Tom Hanks", "Forest Gump"], ["Robin Wright", "Jenny Curran"], ["Gary Sinise", "Lieutenant Dan Taylor"] ], "Toy Story", ... ] // Of course the "movie" and "role" schema definitions may also be // serialized to KSON using the "schema" schema: var schema_data = KSON.stringify(movie_schemas, "[]schema") console.log(schema_data); ["[]schema","role",["name","character"],[0, 0],"movie", ["title","year","rating","cover","actors"], [0,0,0,"prefix:http\://movies.db/covers/|suffix:.jpg","[]role"]] // Likewise schemas can be initialized with a kson document KSON.addSchema(schema_data);FAQ---Q: Why should I use this instead of JSON/MessagePack/Thrift/...?A: Unless you are targeting the browser you probably shouldn't. If you are, then the size and speed of the javascript libraries for these formats may make KSON a favorable option.Q: Is KSON faster/smaller/better than JSON?A: Tests using node.js indicate that parsing is marginally slower. This should be outweighed by faster transmission speed of signifigantly less data. See the node.js benchmarks for more info.Q: Is this a drop in replacement for JSON?A: No, you will need to write schema definitions for your data. Automatic schema detection based on example data may help in getting started. Also see "restrictions" in the Specification.Q: Why yet another a schema format for JSON?A: Existing JSON schema formats include extrenious information to the purpose of KSON and would require larger javascript library for parsing and serialization.Q: Is there language support for php/ruby/java/c...?A: Currently javascript and python are supported. Porting to other languages with existing JSON support should be fairly easy and patches are very welcome.Schema Specification--------------------A KSON schema consists of three fields: id *string* identifier fields *array* of field names meta *array* signifying the types associated with corresponding elements in the *fields* array.A schema only defines one object. Nesting is accomplished by referencingsubschemas via their id in the meta array.An element of *meta* specifies the content of the element of *fields* atthe same index. The lengths of the *fields* and *meta* arrays must be equal.An element of *meta* must be one of the following: 0 field contains a plain value of type *string*, *boolean*, *number*, *null*. "[]" *array* of plain values "schema-id" *string* reference the schema of a nested object "[]schema-id" "codec-id" "[]codec-id"The top level elements of a KSON document must be either an *array* ofobjects or an *object*. A plain value or an *array* of plain valuescannot be serialized.Codecs------Decoders are executed in the order of their definition in the meta field.The decoder is passed the value to decode as the first argument and theintermediate data object as the second. The intermediate object will havethe values of all previous fields decoded and applied.Included codecs: enum:a:b:c:d "a" -> 0, "b" -> 1, ... prefix:egg "eggbacon" -> "bacon" suffix:bacon "eggsausagebacon" -> "eggsausage" bool true -> 1, false -> 0 date new Date() -> 1364938727390 int36 1364938727390 -> "hf1lcnka"Codec chaining prefix:egg|suffix:bacon "eggsausagebacon" -> "sausage" date|int36 new Date() -> "hf1lcnka"Custom codecs are registered using factory functions in the following manner. KSON.coders["my_coder_id"] = function(args) { // args is a string array from a schemas meta field // enum:a:b:c:d -> args == ["a", "b", "c", "d"] return function (val, encode) { // The actual encoding/decoding function return encode ? encode_op(val) : decode_op(val); } }Schema Validation-----------------The python library validates schemas and codec definitions, the javascriptlibrary does not. Invalid schema definition loaded with the javascriptlibrary may parse or stringify bad data without any throwing any errors.Common errors include: - Loading a schema before the custom codecs it uses have been registered. - Loading a schema before the subschemas on which it depends have been loaded. Note that schemas are loaded in the order they appear in a file, so dependent schmas schould come after those they depend uppon. - Circularly dependent schmas (including schemas which depend on themselves) are possible by using a stub schema, which only specifies an id and no fields. This can be replaced by the propper schema with the same id, after its dependencies have been loaded. - Loading a json encoded schema. The argument of the KSON.addSchema function may be either a KSON encoded schema or schemas, an already decoded schema or an array of schemas. It is however fairly simple to load json encoded schemas: KSON.addSchema(JSON.parse(raw_json_schemas));The python library will warn about these and other issues, so use it toInstallation------------ pipinstallksonAutomaticschemadetection: kson --auto-detect data.json --id schema_idDevelopment----------- gitclonegit@bitbucket.org:mbarkhau/kson.git npm install uglify-js -g npminstallbenchmarkmicrotime pip install py.test maketest make bench $ make minify