Response inline below..
Post by Anders RundgrenPost by Jim KloHi Anders,
I did some work very similar some time back for Learning Registry [1]. There are some known issues with the solution we chose and upon investigation of your doc, seems to suffer from similar problems.
Hi Jim,
This sounds interesting!
Post by Jim KloMy first question might be is what is your indented use case for this?
Primary replacing this: http://webpki.org/papers/keygen2/keygen2.junit.run.html
which is a part of: https://code.google.com/p/openkeystore
Not sure if I fully understand what's going on here - but looks like JUnit test cases that are provisioning some PKI - little hard to follow for me; I'm no crypto expert.
Post by Anders RundgrenPost by Jim KloLike JOSE this seems to have very limited use due to a number of issues I'll outline below.
Well, I have done some shortcuts/redefinitions of JSON which should make this scheme usable for a wide class of web-based security-protocols, where the _message_ is the core and a signature is only there to vouch for the message's authenticity.
Yes, that's essentially what we did - take some shortcuts
digitally signing the hash of our canonical bencoded representation was a 'shortcut' for storage and provided a check to ensure that you canonicalized correctly. Our goal however was to ensure that maintain that any JSON parser could still natively access data, and try to use well known algorithms to aid in validation of authenticity and check for forgery. Known sha256 isn't perfect - but good enough
Bencode isn't perfect - but good enough at the time.
We've had several discussions on steps to improve - but most things are equally complex. Crypto is already unnecessarily too hard for most smart developers to understand - adding complex steps for validation of portable data structures is just another notch in that got added - that I'd like to make even more simple.
Post by Anders Rundgrenhttps://code.google.com/p/openkeystore/source/browse/library/trunk/src/org/webpki/json/JSONParser.java
300 lines of code seems supportable. it is brand new so it may have some "warts".
I suspect you could have also just modified something like Jackson and modified toString() and map storage to use a modified JSONObject as well. Regardless, the parser seems straight forward
the magic is in your JSONObject
where you have to provide a crutch to retain ordering via a LinkedHashMap to maintain key ordering - not available in all solutions without some effort
I'm more interested in serialization of simple typed objects via JSON, less concerned about more generic container like implementations:
Foo.java
public class Foo {
int alpha;
String beta;
}
Foo.py
class Foo(object):
def __init__(self, a, b)
self.alpha = a
self.beta = b
Using Java reflection wouldn't necessarily walk properties in the same order as say Python dir(.) would. Producing a canonical output would require some kind of 'crutch' code, IMO. I'm just looking for simplicity. :)
Ideally, at least for our use case which is a federated metadata referratory, having the ability to access our envelope data (which includes the crypto signature) is pretty important for stream processing as well as minimizing storage requirements. Because data is publicly federated, the signature is primarily there to minimize forgery from trusted signatories.
Here's a sample: http://node01.public.learningregistry.net/obtain?by_doc_ID=true&request_ID=15a0b923465249fd80b8b3c0f9f7879e
documents[0].document[0] is our envelope, documents[0].document[0].resource_data is the metadata, everything else is api wrapper.
Post by Anders RundgrenIBM has apparently patented (!!!) a JSON signature scheme ( http://patents.justia.com/patent/20100185869 ) which does canonicalization the hard ("right" if you prefer that...) way. I won't try that because then the advantage of JSON becomes limited.
I've not seen this before
Looks remarkably like XML variants of the same thing. It's amazing what the USPTO grants nowadays
but that's a topic for another discussion
Post by Anders RundgrenPost by Jim Klo// fig 1
{
"alpha": 1,
"beta": "kappa"
}
and
// fig 2
{
"beta": "kappa",
"alpha": 1
}
It would not validate using JSON schema ( http://json-schema.org/ ) and that is where I started (with my 12 years of XSD in the back).
How so?
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"beta": {
"type": "string"
},
"alpha": {
"type": "integer"
}
},
"required": [ "alpha", "beta" ],
"additionalProperties": false
}
As far as I know there is no mechanism in the language to specify key ordering.
Cheers,
- Jim
Post by Anders RundgrenI.e. I consider JSON properties equivalent to XML elements rather than XML attributes.
But that's of course *my* (re)definition of JSON :-)
Cheers
Anders
Post by Jim KloJOSE gets around this because the data is contained within the JWT packaging. Additionally JOSE does signature validation of the base64 encoded material - no the JSON entities. Like you've mentioned this isn't suitable for usage as a data model.
The challenge is there's no good way (that I've uncovered) to canonicalize JSON objects and retain use as a parsed data model that serializes between platforms consistently. Ultimately what you've done is created a canonical format that just treats the entire entity as a string. This wouldn't necessarily work well as moving the data between Java, JavaScript, Ruby, Python, etc
dictionary keys are not ordered consistently
say my system receives fig 1 above and stores it for processing, the key order isn't guaranteed when parsing it from it's native format. In our use case we use CouchDB which stores documents as JSON. While CouchDB just happens to be consistent in storing/retrieving dictionaries in the same order as they submitted - there's not guarantee of that. Clients connecting to the DB also may not reconstruct key order as well depending upon their JSON parsers. Thus in order to make your solution work, Like JOSE, the canonical format must be retained
indefinitely in a portable format.
Post by Jim KloI've covered dictionaries
and FWIW XML shares the same problem with element attributes. Other challenges in signing JSON are brought upon by floating point and boolean values. Floating point precision is problematic unless encoded as strings as floating point precision is not portable across platforms. JavaScript, Erlang, and Python could introduce error when moving objects between systems. Booleans have the same issue - some languages internally store 1 and 0 others actually have native true and false.
Ultimately we chose to use Bencode [2] as part of a canonicalization solution which unfortunately cannot handle floats very well, however in meeting I had with BitTorrent folks some time ago we discussed Tagged Netstrings [3] as a more viable solution.
IMO, the only way to make signing JSON objects reasonable is to have a solution that is trivial to canonicalize the format for reconstruction of the signed object and permits native serialization of the JSON. I don't want to have to keep some string blob around because I need to preserve the canonical format.
To a certain extent the solution to this lies in the development, implementation, and adoption of s "CJSON" or Canonical JavaScript Object Notation, where the representation of the object is already canonical and requires no transformations. Alas, there seems to be very little interest for this, since most of the current work is like JOSE where signatures are only needed short term for one time validation of data sent over the wire.
[1] http://docs.learningregistry.org/en/latest/spec/Identity_Trust_Auth_and_Security/index.html#identity-and-digital-signatures
[2] https://wiki.theory.org/BitTorrentSpecification#Bencoding
[3] http://tnetstrings.org/
Jim Klo
Senior Software Engineer
Center for Software Engineering
SRI International
Post by Anders RundgrenSince Google doesn't support XSD or XML DSig in Android I began looking at other alternatives.
There were none :-( Now there is :-)
https://openkeystore.googlecode.com/svn/resources/trunk/docs/Enveloped-JSON-Signatures.pdf
Comments are welcome!
Cheers
Anders