1. #1
    Ext User
    Join Date
    Dec 2007
    Posts
    6
    Vote Rating
    0
    rslinckx is on a distinguished road

      0  

    Default [2.2] JSON decoding is broken on some unicode chars

    [2.2] JSON decoding is broken on some unicode chars


    See this post: http://www.extjs.com/forum/showthread.php?t=40316

    Basically, the JSON decoder only does a
    Code:
    eval('('+json+')')
    which may lead to problems in some cases.

    Consider trying to decode the unescaped string "\u2028". The javascript engine will abort with a syntax error because this is a line termination unicode char.

    http://www.json.org/json2.js will correctly escape some unicode chars that lead to problems.

    As the mentioned post also point out, the Ext.decode() facility isn't used in some classes such as JsonReader and TreeLoader, and those should be fixed as well to use Ext.decode()

  2. #2
    Sencha - Ext JS Dev Team Animal's Avatar
    Join Date
    Mar 2007
    Location
    Notts/Redwood City
    Posts
    30,506
    Vote Rating
    54
    Animal has a spectacular aura about Animal has a spectacular aura about Animal has a spectacular aura about

      0  

    Default


    I can type the following on the Firebug console:

    Code:
    eval("({foo:'bar',\u2028bletch:'blivit'})")
    And it creates the expected object with the two properties in.

  3. #3
    Ext User
    Join Date
    Dec 2007
    Posts
    6
    Vote Rating
    0
    rslinckx is on a distinguished road

      0  

    Default


    Try this one:
    Code:
    eval("({foo:'bar',bletch:'b\u2028livit'})")

  4. #4
    Sencha - Ext JS Dev Team Animal's Avatar
    Join Date
    Mar 2007
    Location
    Notts/Redwood City
    Posts
    30,506
    Vote Rating
    54
    Animal has a spectacular aura about Animal has a spectacular aura about Animal has a spectacular aura about

      0  

    Default


    Ah got you. But surely that has to be quoted. What you are sending is intended to be evaluated. It is a literal expression of the Javascript language, so you must send a valid literal expression:

    Code:
    eval("({foo:'bar',bletch:'b\\u2028livit'})")

  5. #5
    Ext User
    Join Date
    Dec 2007
    Posts
    6
    Vote Rating
    0
    rslinckx is on a distinguished road

      0  

    Default


    I'm expecting Ext.decode() to accept a valid json string. A json string may not be a valid javascript expression, especially if you take into account the browser's javascript engine implementation which do some weird stuff on some weird unicode strings.

    I see here two solutions, either change the semantic of the Ext.decode function to 'eval a piece of javascript code', or fix the Ext.decode function to accept any well formed json string (which may contain chars that are not valid javascript expressions)

    Note that having an un-escaped \u2028 *is* valid json as per the spec at json.org.

    This is the regexp used in json2.js to re-escape some unicode chars that browsers do not handle gracefully:
    Code:
    var cx = /[\u0000\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g

  6. #6
    Sencha - Ext JS Dev Team Animal's Avatar
    Join Date
    Mar 2007
    Location
    Notts/Redwood City
    Posts
    30,506
    Vote Rating
    54
    Animal has a spectacular aura about Animal has a spectacular aura about Animal has a spectacular aura about

      0  

    Default


    Let's see what Jack and Brian think. I really don't like this technicality that separates JSON from being a Javascript literal. IMNSHO, there's no such thing as JSON. It is a literal expression of the Javascript language, and when you generate some on the server, you are generating source code. But that's just the way I treat it.

  7. #7
    Ext User
    Join Date
    Dec 2007
    Posts
    6
    Vote Rating
    0
    rslinckx is on a distinguished road

      0  

    Default


    Well, it's clear from the json.org homepage that json is not source code

    Code:
    JSON is a lightweight data-interchange   format. [...] It is based on a subset of the   JavaScript   Programming Language,   Standard   ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely   language independent [...] These properties make JSON an ideal data-interchange language.
    But that's not the issue, the issue is that If i need to use Ext with a server I can't control, I can't ask the server to escape all output, especially if its output is valid json as per the spec

  8. #8
    Sencha - Ext JS Dev Team Animal's Avatar
    Join Date
    Mar 2007
    Location
    Notts/Redwood City
    Posts
    30,506
    Vote Rating
    54
    Animal has a spectacular aura about Animal has a spectacular aura about Animal has a spectacular aura about

      0  

    Default


    Yes, I know. He's quite a stickler for that JSON standard that is not Javascript.

    I recently suggested to add a putLiteral method to the Java JSONObject and JSONArray classes which would allow addition of Javascript expressions without wrapping the resulting property value with quotes. He said that would break the standard.

    I can't understand the utility of constricting what you can send when you are sending to a native Javascript interpreter.

    I personally want to send real Javascript like

    Code:
    {
        startDate: new Date(1222270993605),
        handler: function(foo) {
            doSomething();
        }
    }
    And have it create the real, live Javascript object based on that literal.

  9. #9
    Sencha - Community Support Team Condor's Avatar
    Join Date
    Mar 2007
    Location
    The Netherlands
    Posts
    24,246
    Vote Rating
    91
    Condor has much to be proud of Condor has much to be proud of Condor has much to be proud of Condor has much to be proud of Condor has much to be proud of Condor has much to be proud of Condor has much to be proud of Condor has much to be proud of

      0  

    Default


    @Animal: Are you actually using org.json? I thought almost everybody was using json-lib (I know I am).

  10. #10
    Sencha - Ext JS Dev Team Animal's Avatar
    Join Date
    Mar 2007
    Location
    Notts/Redwood City
    Posts
    30,506
    Vote Rating
    54
    Animal has a spectacular aura about Animal has a spectacular aura about Animal has a spectacular aura about

      0  

    Default


    Yes, I'm using Crockford's org.json classes. json-lib is a Java alternative? I'll look into it.