modal BasicDialogs on Mac Firefox (18.104.22.168) do not display with a grey opaque mask. Instead, you get some colored horizontal lines on the screen. I even tried looking at the dialog example at
I believe this worked at some point, but doesn't currently. I tested on 2 different Mac machines. Mine with Firebug installed, one with no extensions installed.
Tweaking the CSS rules - it appears to be related to opacity. Set the opacity to 0 or 1, and no lines. Anything else and you get lines.