[2.2] JSON decoding is broken on some unicode chars

[rslinckx]

See this post: http://www.extjs.com/forum/showthread.php?t=40316

Basically, the JSON decoder only does a

eval('('+json+')')

which may lead to problems in some cases.

Consider trying to decode the unescaped string "\u2028". The javascript engine will abort with a syntax error because this is a line termination unicode char.

http://www.json.org/json2.js will correctly escape some unicode chars that lead to problems.

As the mentioned post also point out, the Ext.decode() facility isn't used in some classes such as JsonReader and TreeLoader, and those should be fixed as well to use Ext.decode()

[Animal]

I can type the following on the Firebug console:

eval("({foo:'bar',\u2028bletch:'blivit'})")

And it creates the expected object with the two properties in.

[rslinckx]

Try this one:

eval("({foo:'bar',bletch:'b\u2028livit'})")

[Animal]

Ah got you. But surely that has to be quoted. What you are sending is intended to be evaluated. It is a literal expression of the Javascript language, so you must send a valid literal expression:

eval("({foo:'bar',bletch:'b\\u2028livit'})")

[rslinckx]

I'm expecting Ext.decode() to accept a valid json string. A json string may not be a valid javascript expression, especially if you take into account the browser's javascript engine implementation which do some weird stuff on some weird unicode strings.

I see here two solutions, either change the semantic of the Ext.decode function to 'eval a piece of javascript code', or fix the Ext.decode function to accept any well formed json string (which may contain chars that are not valid javascript expressions)

Note that having an un-escaped \u2028 *is* valid json as per the spec at json.org.

This is the regexp used in json2.js to re-escape some unicode chars that browsers do not handle gracefully:

var cx = /[\u0000\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g

[Animal]

Let's see what Jack and Brian think. I really don't like this technicality that separates JSON from being a Javascript literal. IMNSHO, there's no such thing as JSON. It is a literal expression of the Javascript language, and when you generate some on the server, you are generating source code. But that's just the way I treat it.

[rslinckx]

Well, it's clear from the json.org homepage that json is not source code

JSON is a lightweight data-interchange   format. [...] It is based on a subset of the   JavaScript   Programming Language,   Standard   ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely   language independent [...] These properties make JSON an ideal data-interchange language.

But that's not the issue, the issue is that If i need to use Ext with a server I can't control, I can't ask the server to escape all output, especially if its output is valid json as per the spec

[Animal]

Yes, I know. He's quite a stickler for that JSON standard that is not Javascript.

I recently suggested to add a putLiteral method to the Java JSONObject and JSONArray classes which would allow addition of Javascript expressions without wrapping the resulting property value with quotes. He said that would break the standard.

I can't understand the utility of constricting what you can send when you are sending to a native Javascript interpreter.

I personally want to send real Javascript like

{
    startDate: new Date(1222270993605),
    handler: function(foo) {
        doSomething();
    }
}

And have it create the real, live Javascript object based on that literal.

[Condor]

@Animal: Are you actually using org.json? I thought almost everybody was using json-lib (I know I am).

[Animal]

Yes, I'm using Crockford's org.json classes. json-lib is a Java alternative? I'll look into it.