[Jprogramming] difficulty with extended precision arithmetic
bill lam
bbill.lam at gmail.com
Tue Jan 19 17:36:43 HKT 2010
mar, 19 Jan 2010, R.E. Boss skribis:
> T=:10000000000000306999999999997669x
>
> 9!:11 ]20 NB. print precision
>
> T^0.5
> 3162277660168428
>
Isn't the result of T^0.5 a ieee floating point? In that case, increase its
print precision above 16 does not increase accuracy.
--
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3
More information about the Programming
mailing list