[Jprogramming] Puzzling result

[Erling Hellenäs]

So, the perceived problem is related to comparison tolerance. The 
question is then how it is used and if this use is rational?

With comparison tolerance at 0, when we add another digit, we get a new 
incorrect result, now related to lost precision?

9!:19]0
      196x|57290824867848391x
99
     ({.196 0.5)|57290824867848391
104

Cheers,

Erling

Den 2017-09-13 kl. 14:38, skrev Don Guinn:
> I think I found the culprit.
>
>     ({.196 0.5)|5729082486784839
> 0
>     9!:18''
> 5.68434e_14
>     2^.9!:18''
> _44
>     9!:19]0
>
>     ({.196 0.5)|5729082486784839
> 147
>     9!:19]5.68434e_15
>
>     ({.196 0.5)|5729082486784839
> 147
>     ({.196 0.5)|57290824867848391
> 0
>     9!:19]5.68434e_16
>
>     ({.196 0.5)|57290824867848391
> 104
>
> On Tue, Sep 12, 2017 at 4:57 PM, Raul Miller <rauldmiller at gmail.com> wrote:
>
>>     R2=: |
>>     R1=: (] - [ * [: <.!.0 ] % [)"0
>>     R0=: (|&x:)"0
>>
>>     (14^2) R1 5729082486784839
>> 147
>>     (14^2) R2 5729082486784839
>> 0
>>
>>     X=: (0.1-0.1)+i.1000
>>     Y=: 5729082486784839+i:1000
>>
>>     datatype X
>> floating
>>
>>     +/,X (R0/ ~: R2/) Y
>> 1845046
>>     +/,X (R0/ ~: R1/) Y
>> 0
>>
>> Good question.
>>
>> Thanks,
>>
>> --
>> Raul
>>
>>
>> On Tue, Sep 12, 2017 at 4:26 AM, Erling Hellenäs
>> <erling at erlinghellenas.se> wrote:
>>> Residue1=: ] - [ * [: <.!.0 ] % [
>>>
>>> Residue2=: |
>>>
>>> (14^2) Residue1 5729082486784839
>>>
>>> 147
>>>
>>> (14^2) Residue2 5729082486784839
>>>
>>> 0
>>>
>>> Why is Residue2 losing precision when Residue1 is not?
>>>
>>> /Erling
>>>
>>>
>>>
>>> Den 2017-09-12 kl. 02:04, skrev Don Kelly:
>>>> I believe that Raul has hit it on the head.  14^2 will be floating as
>> will
>>>> generally be the case with % so it is possible that errors add or
>> subtract
>>>> to give an erroneous result. Note that
>>>>
>>>> (14^2) (] - [ * [: <. ] % [)5729082486784839x
>>>>
>>>> 147
>>>>
>>>>
>>>> (14^2x) (] - [ * [: <. ] % [)5729082486784839
>>>>
>>>> 147
>>>>
>>>> and
>>>>
>>>> (14^2x)|5729082486784839
>>>>
>>>> 147
>>>>
>>>>
>>>> does it more simply
>>>>
>>>>
>>>> Don
>>>>
>>>> On 2017-09-11 6:50 AM, Erling Hellenäs wrote:
>>>>> (14^2x) (] - [ * [: <. ] % [)5729082486784839x
>>>>>
>>>>> 147
>>>>>
>>>>> (14^2) (] - [ * [: <. ] % [)5729082486784839
>>>>>
>>>>> _49
>>>>>
>>>>> (14^2) (] - [ * _1 + [: <. ] % [)5729082486784839
>>>>>
>>>>> 147
>>>>>
>>>>> /Erling
>>>>>
>>>>>
>>>>> Den 2017-09-11 kl. 15:40, skrev Erling Hellenäs:
>>>>>>  From what I see, floor gives an incorrect result which I correct with
>>>>>> the " _1 + " There is not other problem. No loss of precision. /Erling
>>>>>>
>>>>>>
>>>>>> Den 2017-09-11 kl. 15:27, skrev Raul Miller:
>>>>>>> I do not know what you are thinking, and you have not illustrated
>> your
>>>>>>> thoughts sufficiently for me to grasp them.
>>>>>>>
>>>>>>> So, I thought I'd work through this:
>>>>>>>
>>>>>>>      (14^2) (] - [ * _1 + [: <. ] % [)5729082486784839
>>>>>>> 147
>>>>>>>      (14^2x) (] - [ * _1 + [: <. ] % [)5729082486784839x
>>>>>>> 343
>>>>>>>
>>>>>>> Clearly, the exact answer is different from the quick answer.
>>>>>>>
>>>>>>> If we compare
>>>>>>>      load'debug/dissect'
>>>>>>>      dissect '(14^2) (] - [ * _1 + [: <. ] % [)5729082486784839'
>>>>>>>      dissect '(14^2x) (] - [ * _1 + [: <. ] % [)5729082486784839x'
>>>>>>>
>>>>>>> We can see the initial off-by-one errors, but the big difference
>> shows
>>>>>>> up at the end (where we subtract from 5729082486784839)
>>>>>>>
>>>>>>> The value we are subtracting is:
>>>>>>>
>>>>>>>      (14^2x) ([ * _1 + [: <. ] % [)5729082486784839x
>>>>>>> 5729082486784496
>>>>>>>      (14^2) ([ * _1 + [: <. ] % [)5729082486784839
>>>>>>> 5.72908e15
>>>>>>>
>>>>>>> Looking at the bits needed to represent that as an integer:
>>>>>>>
>>>>>>>      2^.(14^2) ([ * _1 + [: <. ] % [)5729082486784839
>>>>>>> 52.3472
>>>>>>>
>>>>>>> we can see that that's right on the edge of being the number of
>>>>>>> representable digits in a 64 bit float.
>>>>>>>
>>>>>>> But, also, it's the result of multiplying
>>>>>>>
>>>>>>>      14^2
>>>>>>> 196
>>>>>>>
>>>>>>> by an off-by-one amount. And...
>>>>>>>
>>>>>>>      343-196
>>>>>>> 147
>>>>>>>
>>>>>>> ...
>>>>>>>
>>>>>>> Anyways, I'm not sure what you were thinking, but I guess we can take
>>>>>>> this as a good example of the idea that errors can actually
>>>>>>> accumulate.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>> ------------------------------------------------------------
>> ----------
>>>>>> For information about J forums see http://www.jsoftware.com/
>> forums.htm
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>> For information about J forums see http://www.jsoftware.com/forums.htm
>>>>
>>>> ----------------------------------------------------------------------
>>>> For information about J forums see http://www.jsoftware.com/forums.htm
>>>
>>> ----------------------------------------------------------------------
>>> For information about J forums see http://www.jsoftware.com/forums.htm
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm