[Jprogramming] i. on floating point

John Baker bakerjd99 at gmail.com
Wed Jul 1 19:04:14 UTC 2015


I have observed that this sometimes fails

fl=.  78.9999 .....  some floating list
sl=. 78.9999 .... some subset of (fl)

(666.666 #~ #sl) (fl i.  sl) } fl  NB. searching floats

where this always works

(666.666 #~ #sl) ((<.fl) i. <. sl) } <. fl  NB. searching integers

When I came across this I thought this could be fixed by comparison
tolerance.

(666.666 #~ #sl) (fl (i. !.0) sl) } fl

but no luck.

Either this is a bug in J or I am not properly setting comparison tolerance
to 0.


Floating point - one of life's more intractable necessary evils.

-- 
John D. Baker
bakerjd99 at gmail.com


More information about the Programming mailing list