PDA

View Full Version : [FIXED][3.0.0] Ext.urlEncode - empty array value bug



Laurent Mignon
7 Jul 2009, 9:55 PM
Ext version tested:
Ext 3.0-rc3
Browser versions tested against:

FF3 (firebug 1.3.0.10 installedOperating System:

Linux (ubuntu 9.04)Description:

Urlencode doesn't work properly whith empty array value parameter

The bug is related to the already closed thread http://extjs.com/forum/showthread.php?p=35402 (http://extjs.com/forum/showthread.php?p=354025)

Test Case:


Ext.urlEncode({param-list: [], foo: 1})

Steps to reproduce the problem:
NoneThe result that was expected:
"param-list=&foo=1"The result that occurs instead:
"foo=1"Possible fix:

Juts change a litle bit the code provided by the refered thread as



Ext.apply(Ext, {
each: function(array, fn, scope){
if(Ext.isEmpty(array, true))
return;
if(typeof array.length == "undefined" || Ext.isPrimitive(array)){
array = [array];
}
for(var i = 0, len = array.length; i < len; i++){
if(fn.call(scope || array[i], array[i], i, array) === false){
return i;
};
}
},

urlEncode: function(o, pre){
var buf = [], key, item, e = encodeURIComponent;

for(key in o){
item = o[key];
Ext.each(typeof item == 'undefined' || Ext.isEmpty(item, true)? key : item, function(val, i){
buf.push("&", e(key), "=", val != key ? e(val) : "");
});
}
if(!pre){
buf.shift();
pre = "";
}
return pre + buf.join('');
}
});

aconran
8 Jul 2009, 4:38 AM
Confirmed, marking this thread as OPEN.

evant
8 Jul 2009, 6:47 PM
I think that the behaviour currently is not quite right. If the value is undefined it will be included, but ignored if it is null. As you've also suggested, empty arrays don't get included. I think null items should definitely be included here.

Laurent Mignon
8 Jul 2009, 9:39 PM
For my part, I've the feeling that every args should be included. When you post a form, every inputs are posted whatever their values.
In the case of a jsonstore (the one I've debugged), the Ext.urlEncode is used to build url based on params and the url is finally transformed into a post request. (as I understand but maybe am I wrong) . The result is that the number of paramter in the form posted will depend of their values. It's a different behaviour than in 2.x.

Condor
8 Jul 2009, 10:24 PM
If you examine this from a standpoint of a form being submitted with a GET request then:

{param-list: [], foo: 1}
would resemble:

<form method="GET" action="url">
<input type="checkbox" name="param-list" value="1" />
<input type="checkbox" name="param-list" value="2" />
<input type="checkbox" name="foo" value="1" checked="checked" />
</form>
which would indeed generate the following query string when posted:

foo=1

So I do not think that Ext.urlEncode is necessarily wrong.

evant
8 Jul 2009, 10:38 PM
The 2.x version of encode will generate a parameter for an empty array.

Neither of them will generate a parameter for a null value, but will generate one for undefined, which I think is odd.

3.x also doesn't encode dates correctly like 2.x does.

Laurent Mignon
9 Jul 2009, 2:47 AM
If you examine this from a standpoint of a form being submitted with a GET request then:

{param-list: [], foo: 1}would resemble:

<form method="GET" action="url">
<input type="checkbox" name="param-list" value="1" />
<input type="checkbox" name="param-list" value="2" />
<input type="checkbox" name="foo" value="1" checked="checked" />
</form>which would indeed generate the following query string when posted:

foo=1So I do not think that Ext.urlEncode is necessarily wrong.

If but the use case I consider is if I submit with a POST request....
The problem here, is due to the fact that I've initialized my store with a 'POST' method attribute


var listingStore = new Ext.data.Store({
autoLoad: true,
remoteSort: true,
proxy: new Ext.data.HttpProxy({
url: '/printers/listing',
method: 'POST'
}),
but for some reason, the posted form is based on the output of Ext.urlEncode and thus, I don't received all expected parameters.

Condor
9 Jul 2009, 3:46 AM
For a form post it doesn't matter if you use GET or POST. A set of <input type="checkbox"> tags with the same name or a <select multiple="multiple"> tag won't post anything if no selection is made.

Laurent Mignon
9 Jul 2009, 5:37 AM
I agree with you but a form with a set of <input type="text"> tag will post something...
I'm not a specialist and I don't know what is the correct behavior between the Ext2 and Ext3 urlEncode method. I've maybe build my business implementation on a wrong assumption that everything defined in the baseParameters of my store will be posted to the server(in Ext2 it's the case) whatever their values ...

Condor
9 Jul 2009, 5:40 AM
But:

<input type="text" name="param-list" value="" />
<input type="text" name="param-list" value="" />
<input type="text" name="foo" value="1" />
would result in the following query string:

param-list=&param-list=&foo=1
because it is equivalent to:

Ext.urlEncode({param-list: ['', ''], foo: 1})
(not an empty array)

Laurent Mignon
9 Jul 2009, 5:54 AM
OK
My prob is just in the different behavior of the same functionality in Ext2 and Ext3.
Thanks for your explanation.

SeiginoRaikou
10 Jul 2009, 7:54 AM
I see why when looking at it from the point of view of a posting form it would make sense not to include empty arrays, but what about web services?

I'm using Ext.Ajax.request to hit an ASMX web service and am running into the same issue. Correct me if I'm wrong but I don't think I have the option to ignore missing parameters - either urlEncode includes that parameter or I have to stop using arrays as web service parameters and will have serialize/deserialize strings instead.

Laurent Mignon
13 Jul 2009, 3:16 AM
For a form post it doesn't matter if you use GET or POST. A set of <input type="checkbox"> tags with the same name or a <select multiple="multiple"> tag won't post anything if no selection is made.


But:

<input type="text" name="param-list" value="" />
<input type="text" name="param-list" value="" />
<input type="text" name="foo" value="1" />would result in the following query string:

param-list=&param-list=&foo=1because it is equivalent to:

Ext.urlEncode({param-list: ['', ''], foo: 1})(not an empty array)
And how can I represent /post an empty array???

Condor
13 Jul 2009, 3:22 AM
And how can I represent /post an empty array???

You can't (using a form at least).

But doesn't the absence of the parameter not already indicate an empty array?

Laurent Mignon
13 Jul 2009, 3:28 AM
You can't (using a form at least).

But doesn't the absence of the parameter not already indicate an empty array?

The problem is probably more complex...

my-array=null could be represented by the absence of the parameter
my-array=[] could be represented by ?my-array= (but in this case how can I represent my-array=[''])????

Condor
13 Jul 2009, 3:37 AM
It's simply a limitation of url parameter encoding.

Maybe you should switch to JSON encoding parameters (needs more work on the server though).

mschwartz
13 Jul 2009, 5:47 AM
This is clearly a bug.



#!/usr/bin/php
<?php
print json_encode(array());
Output:


$ ./test.php
[]


And:


#!/usr/bin/php
<?php
print json_encode(array('foo' => array()));


Output:



$ ./test.php
{"foo":[]}

evant
13 Jul 2009, 5:49 AM
We've already acknowledged there's an issue, however this has nothing to do with json encoding.

Condor
13 Jul 2009, 5:50 AM
This is clearly a bug.

???

We are talking about URL encoding and not about JSON encoding.

mschwartz
13 Jul 2009, 5:51 AM
???

We are talking about URL encoding and not about JSON encoding.

I think it should do what json_encode() does for arrays.

evant
13 Jul 2009, 5:53 AM
Definitely not, we're talking about URL encoding data to post to the server in a way that mimics a form post.

Condor
13 Jul 2009, 5:57 AM
I think it should do what json_encode() does for arrays.

You are confusing Ext.encode and Ext.urlEncode. Those two are completely different!

@evant: How would you expect the following object to be encoded?

Ext.urlEncode({test1: undefined, test2: null, test3: [], test4: [undefined, null]})

Currently it's:

test1=&test4=undefined&test4=null
which is indeed inconsistent.

IMHO it should probably be:

test1=&test2=&test4=&test4=
(encode undefined and null as empty string)

Or should undefined not be encoded:

test2=&test4=
(only encode null as empty string)

evant
13 Jul 2009, 5:58 AM
In 2.x test3 is also included as an empty parameter. I think it makes sense to maintain this behaviour as well.

Condor
13 Jul 2009, 5:59 AM
In 2.x test3 is also included as an empty parameter. I think it makes sense to maintain this behaviour as well.

It might not be backward compatible, but it is consistent with how a <form> works (as I explained before).

mschwartz
13 Jul 2009, 5:59 AM
<?php

if (isset($_POST['button'])) {
print "<pre>";
var_dump($_POST);
}
else {
?>
<form action="test.php" method="post">
<input type="text" name="x[]">
<input type="text" name="x[]">
<input type="text" name="x[]">
<input type="submit" name="button">
</form>
<?php
}


buttonSubmit Query
x[]
x[]
x[]

Condor
13 Jul 2009, 6:01 AM
<?php

if (isset($_POST['button'])) {
print "<pre>";
var_dump($_POST);
}
else {
?>
<form action="test.php" method="post">
<input type="text" name="x[]">
<input type="text" name="x[]">
<input type="text" name="x[]">
<input type="submit" name="button">
</form>
<?php
}


buttonSubmit Query
x[]
x[]
x[]


That is posting an array of empty strings, e.g. {'x[]': ['', '', '']}.

Now look at how this is posted:

<form action="test.php" method="post">
<input type="checkbox" name="x[]" value="1">
<input type="checkbox" name="x[]" value="2">
<input type="checkbox" name="x[]" value="3">
<input type="submit" name="button">
</form>

mschwartz
13 Jul 2009, 6:02 AM
With method="GET":

http://localhost/~mschwartz/test.php?x[]=&x[]=&x[]=&button=Submit+Query

mschwartz
13 Jul 2009, 6:02 AM
Checkboxes are badly broken in HTML

They don't post anything if not checked.

mschwartz
13 Jul 2009, 6:05 AM
http://localhost/~mschwartz/test.php?button=Submit+Query (http://localhost/%7Emschwartz/test.php?button=Submit+Query)


(with checkboxes)

Condor
13 Jul 2009, 6:14 AM
http://localhost/~mschwartz/test.php?button=Submit+Query (http://localhost/%7Emschwartz/test.php?button=Submit+Query)


(with checkboxes)

Exactly! That is not badly broken, it's just the way checkboxes work!

And it's not only checkboxes, a <select multiple="multiple"> tag works the same way.

That is why:

Ext.urlEncode({'x[]': []})
should result in an empty string!

SeiginoRaikou
13 Jul 2009, 6:15 AM
I'm with Evant, why make a breaking change just because it's technically more in line with a form HTML tag, which is not the only way URL encoding is used?

mschwartz
13 Jul 2009, 6:16 AM
You are confusing Ext.encode and Ext.urlEncode. Those two are completely different!

@evant: How would you expect the following object to be encoded?

Ext.urlEncode({test1: undefined, test2: null, test3: [], test4: [undefined, null]})Currently it's:

test1=&test4=undefined&test4=nullwhich is indeed inconsistent.

IMHO it should probably be:

test1=&test2=&test4=&test4=(encode undefined and null as empty string)

Or should undefined not be encoded:

test2=&test4=(only encode null as empty string)



Ext.urlEncode({test1: undefined, test2: null, test3: [], test4: [undefined, null]})

test1=&test2=&test3[]=&test4[]=&test4[]=

evant
13 Jul 2009, 6:19 AM
@mschwartz

We definitely won't be doing this. If you want to implement this in your app, go ahead, but it's not how we're going to do it.

mschwartz
13 Jul 2009, 6:19 AM
I'm with Evant, why make a breaking change just because it's technically more in line with a form HTML tag, which is not the only way URL encoding is used?

Because it's not encoding what you tell it to?

mschwartz
13 Jul 2009, 6:19 AM
@mschwartz

We definitely won't be doing this. If you want to implement this in your app, go ahead, but it's not how we're going to do it.

It's ok.

You asked how it should encode it.

evant
13 Jul 2009, 8:29 AM
This has been fixed in SVN, the following gives:



var o = Ext.urlEncode({
a: undefined,
b: null,
c: '',
d: 'foo',
e: [],
f: [null, undefined],
g: [1, 2],
h: new Date()
});
console.log(o);




a=&b=&c=&d=foo&e=&f=&f=&g=1&g=2&h=2009-07-14T02:28:28


Marking this as fixed.

mschwartz
13 Jul 2009, 8:37 AM
Try posting that to a PHP script and see what $_REQUEST['f'] is...

evant
13 Jul 2009, 8:49 AM
I would assume the same thing as what you'd get if you posted this?



<html>
<head></head>
<body>
<form method="post" action="foo.php">
<input type="hidden" name="f" value="" />
<input type="hidden" name="f" value="" />
<input type="hidden" name="g" value="1" />
<input type="hidden" name="g" value="2" />
<input type="submit"></submit>
</form>
</body>
</html>

mschwartz
13 Jul 2009, 9:28 AM
I would assume the same thing as what you'd get if you posted this?



<html>
<head></head>
<body>
<form method="post" action="foo.php">
<input type="hidden" name="f" value="" />
<input type="hidden" name="f" value="" />
<input type="hidden" name="g" value="1" />
<input type="hidden" name="g" value="2" />
<input type="submit"></submit>
</form>
</body>
</html>


I think it's broken, still. Not to beat a dead horse.

If you:
Ext.urlEncode({ f: [ 'A', 'B' ] });
I don't think it's going to post correctly.

PHP will give you $_REQUEST['f'] = 'B' only.

I think that a PHP script would expect:
$_REQUEST['f''][0] = 'A'
$_REQUEST['f'][1] = 'B'

evant
13 Jul 2009, 9:33 AM
Sure, but we're not just dealing with PHP here. We're following the standard way to post things as you'd do with a normal HTML post. With PHP so widely adopted, I don't really understand how it can be broken in their implementation?

mschwartz
13 Jul 2009, 10:10 AM
Sure, but we're not just dealing with PHP here. We're following the standard way to post things as you'd do with a normal HTML post. With PHP so widely adopted, I don't really understand how it can be broken in their implementation?

The [] syntax isn't just a PHP thing.

Try this:



<form name="myform">

<select name='kcat[]'>
<option>Option1</option>
<option>Option2</option>
<option>Option3</option>
</select>

<select name='kcat[]'>
<option>Option1</option>
<option>Option2</option>
<option>Option3</option>
</select>

<select name='kcat[]'>
<option>Option1</option>
<option>Option2</option>
<option>Option3</option>
</select>

</form>

<script language="JavaScript" type="text/javascript">
alert(document.forms["myform"]["kcat[]"].length)
</script>


(And I'm not sure why you want to break PHP ...)

mschwartz
13 Jul 2009, 10:26 AM
Another test:



<?php

if (isset($_REQUEST['button'])) {
print "<pre>";
var_dump($_POST);
}
else {
?>
<form action="test.php" method="POST">
<input type="text" name="x" value="1">
<input type="text" name="x" value="2">
<input type="text" name="x" value="3">
<input type="submit" name="button">
</form>
<?php
}


Click submit button, output is:



array(2) {
["x"]=>
string(1) "3"
["button"]=>
string(12) "Submit Query"
}

evant
13 Jul 2009, 4:55 PM
I'm not arguing about what PHP returns, I can clearly see that. However:

1) All you're suggesting is automatically renaming the parameters. If this is the case, then PHP users will need to do the same as they would (and probably have been doing), which would be to append the names with [].
2) The behaviour of the array encoding has been like this throughout Ext 2.x, yet nobody has mentioned it up until now. This by itself doesn't really mean much, however with the amount of people that use Ext with PHP, if it were a major problem we would have heard about it by now.

mschwartz
14 Jul 2009, 4:43 AM
I'm not arguing about what PHP returns, I can clearly see that. However:

1) All you're suggesting is automatically renaming the parameters. If this is the case, then PHP users will need to do the same as they would (and probably have been doing), which would be to append the names with [].
2) The behaviour of the array encoding has been like this throughout Ext 2.x, yet nobody has mentioned it up until now. This by itself doesn't really mean much, however with the amount of people that use Ext with PHP, if it were a major problem we would have heard about it by now.

Probably because REST is more widely used in Java or maybe C# would be my guess.

I find it's an interesting concept to urlEncode an arbitrary javascript object. I don't think there's any standard for encoding anything but a string.

It's not useful to me the way it is, being a PHP guy (these days). If there were an optional 2nd argument to enable/disable PHP style encoding, it might make sense for everyone?

BTW, I'm not a huge fan of REST. Through experience, I've found that POST via XML or JSON is ideal. You don't have to worry about caches and proxies interfering with your RPC type requests, you aren't limited to the max length of a URL which can vary from browser to browser, you can include binary data in your GET type requests, etc.