[Jprogramming] Lack of Precision?

[Devon McCormick]

Geoff -

J's extended precision applies to integers only.  If you think about it,
you'll see
that while it's relatively simple to make integers arbitrarily large,
floating point
is a much more difficult proposition: how would you know when to stop
in the potentially infinite expansion of a floating point number?  Even
something
as simple as %3 is a problem.

This restriction to integers is why something like Ahmad's
<.@%: 2x*10x^200
works: "<.@%:" applies both the square root and enforces the
integer limitation at the same time.

On 2/26/07, Geoff Canyon <geoffc at inspiredlogic.com> wrote:
>
> (this is my first J code)
>
> I'm executing this to get the square root of 2:
>
> x:%:x:2e100
>
> I also tried:
>
> <.((%:2)*10^100)
>
> In both cases I'm looking for extended precision (100 places if J
> supports it).
>
> In both cases it appears the digits take a left turn about 20 digits
> in -- they don't match publicly available resources on the web.
>
> Is there something else I should be doing, or is there a limit to J's
> precision?
>
> Thanks!
>
> Geoff
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>



-- 
Devon McCormick
^me^ at acm.
org is my
preferred e-mail