PDA

View Full Version : [FIXED] [BUG:MAJOR, Beta3] - NumericFilter compares floating point numbers for equality



icfantv
21 Feb 2012, 4:41 PM
Because of machine rounding (and as documented by Number.doubleValue) you cannot compare floating point numbers for equality using == or != without using some sort of epsilon/tolerance, i.e. abs(actual - expected) < epsilon.

Here's the offending method (line 61):


protected boolean validateModel(M model) {
Number val = getValueProvider().getValue(model);
if (rangeMenu.eq != null && rangeMenu.eq.getCurrentValue() != null) {
if (val == null || rangeMenu.eq.getCurrentValue().doubleValue() != val.doubleValue()) {
return false;
}
}

// ** snip **
};

WesleyMoy
21 Feb 2012, 5:11 PM
Thanks for bringing this to our attention. I've filed a ticket against the team to have a look.

icfantv
21 Feb 2012, 6:22 PM
You're welcome. I've been reading a bunch of stuff on the web for how to best go about solving this issue, if possible, in the GXT API.

Per http://www.ibm.com/developerworks/java/library/j-jtp0114 (look at the section titled "BigDecimals for small numbers), it may make sense to use BigDecimal internally where appropriate, i.e., for floating point number types.