[Jprogramming] reduce revisited

Fri Feb 26 01:13:47 UTC 2016

Ok, but, these look different to me:

   (i. 250000 2) + reduceS 1 2 3 4
62499750001 62500000002 62499750003 62500000004
   (i. 250000 2) + reduceC [ 1 2 3 4
62499750001 62500000002 3 4
   (i. 250000 4) + reduceC [ 1 2 3 4
124999500001 124999750002 125000000003 125000250004

But, sure, if we only care about a description of the operation, as
opposed to the results they generate, I suppose there is some kind of
equivalence here.

Thanks,

-- 
Raul

On Thu, Feb 25, 2016 at 7:04 PM, Pascal Jasmin <godspiral2000 at yahoo.ca> wrote:
> all of the expressions sum 250001 rows of 4.  Some of the expressions manipulate the x side to create 4 item rows.  That all of the timings are nearly equivalent shows that such manipulations are relatively fast native operations.  Its a major improvement compared to the only way I knew how to compute such expressions prior to today.
>
>
>     15 timespacex '(i. 250000 ) + reduceS  1 2 3 4'
> 0.0179739 1.88783e7
>
>
>     10 timespacex '(i. 250000 ) + reduce  1 2 3 4'
> 0.125398 4.03457e7
>
>   3 timespacex '(i. 250000 ) + reducE  1 2 3 4'
> 0.282656 2.10458e6
>
>
> (i.10) + reduceS i.4
> 45 46 47 48
>
> ----- Original Message -----
> From: Raul Miller <rauldmiller at gmail.com>
> To: Programming forum <programming at jsoftware.com>
> Sent: Thursday, February 25, 2016 6:45 PM
> Subject: Re: [Jprogramming] reduce revisited
>
> These timings, for the most part, are not comparable.
>
> You have four expressions and three different results. So only two of
> the four timings can be meaningfully compared.
>
> --
> Raul
>
>
> On Thu, Feb 25, 2016 at 5:49 PM, Pascal Jasmin <godspiral2000 at yahoo.ca> wrote:
>>> 5 timespacex ' +/ (i. 250000 4), 1 2 3 4'
>>> 0.0200582 2.51692e7
>>> 5 timespacex '(i. 250000 2) + reduceS 1 2 3 4'
>>> 0.0205952 2.51699e7
>>
>> for that first expression, there are no fills, but 1m cells appended.
>>
>> the 2nd expression (equal speed) does 250000 row reshapes
>>
>>
>>
>>> 5 timespacex '(i. 250000 2) + reduceC [ 1 2 3 4'
>> 0.0224864 2.517e7
>>
>>
>> this expression fills with 0s each row of x.
>>
>>> 5 timespacex '(i. 250000 4) + reduceC [ 1 2 3 4'
>> 0.019306 2.517e7
>>
>>
>> equivalent to first expression.  No fills needed.  Should be just as fast.
>>
>>
>> ____
>> But wouldn't timings involving reduce-with-fill depend on how much
>> fill needs to get generated?
>>
>> Thanks,
>>
>>
>>
>> On Thu, Feb 25, 2016 at 4:51 PM, Pascal Jasmin <godspiral2000 at yahoo.ca> wrote:
>>> a couple of implementations I have posted before for u/ when x items and initial y are of different shape.
>>>
>>> reduce =: 1 : '<"_1@[ ([: u (&.>)/(>@:) ,) <@:]'
>>> reducE =: 1 : (':'; 'o=. y for_i. x do. o =. i u o end.')
>>>
>>> reducE is explicit, and left to right.  reduce boxes each side, and uses the same right to
>>> left order as /.  reduce is 2-3x faster for +
>>>
>>> There's an even faster method, but before I mention it, I'd like to go over how the rank 0 operators work in J
>>>
>>> (1 + 1 2 3 4) -: 1 1 1 1 + 1 2 3 4
>>>
>>> `a scalar u"0 a_larger_shape` gets expanded to match the larger shape, and its result is identical to the operation with the "pre-expanded" shape.
>>>
>>> 1 2 + 1 2 3 4 is rank error.
>>>
>>> But there are a couple of obvious expansions that would not create an error
>>>
>>> 1 2 1 2 + 1 2 3 4 or
>>> 1 2 0 0 + 1 2 3 4
>>>
>>> note that this is not an error
>>>
>>> 1 2 + i.2 2 but the result (with shape ommitted) is identical to
>>>
>>>
>>> 1 1 2 2 + 0 1 2 3
>>>
>>> The new reduce adverb is at its core a conjunction.  Where one of its verbs is how to expand the x items such that they are similar (or compatible) shape to y.
>>>
>>> reduceC =: 2 : 'u/@:(v , ])'
>>> reduceS =:  reduceC ($@] $"1 _1 [)
>>>
>>> reduceS is an adverb that generically expands (i.2 2) + reduceS 1 2 3 4 to
>>>
>>> +/ 0 1 0 1 , 2 3 2 3 ,: 1 2 3 4
>>>
>>> which may be better than an error, but is unusual, and so the conjunction version is useful to taylor the expansion as you prefer.
>>>
>>> some custom applied conjunctions
>>>
>>> reduceC ($@] {.!._"1 _1 [) NB. take with fill set to _ to match shape of y.
>>> reduceC ($@] {. 2 # [) NB. expand 1 2 to 1 1 2 2,
>>>
>>> NB. but make sure there are 4 items (end matches shape of y)
>>>
>>> 5 timespacex ' +/  (i. 250000 4), 1 2 3 4'
>>> 0.0200582 2.51692e7
>>> 5 timespacex '(i. 250000 2) + reduceS 1 2 3 4'
>>> 0.0205952 2.51699e7
>>> 5 timespacex '(i. 250000 2) + reduceC (2 #"1 [) 1 2 3 4'
>>> 0.0231547 2.51707e7
>>>
>>> 5 timespacex '(i. 250000 2) + reduce 2 2 $ 1 2 3 4'
>>> 0.134078 4.66376e7
>>>
>>>
>>> 5 timespacex '(i. 1000000) + reduce  1 2'
>>> 0.563169 1.61364e8
>>> 5 timespacex '(i. 1000000) + reduceS  1 2'
>>> 0.0407191 4.1947e7
>>>
>>>
>>>
>>> another technique that is pretty funny how its fast, but works well especially when y is a string, and x is not, is to convert everything to strings (using linear representations of x, lets you have x items with widely varying shapes)
>>>
>>> NB. swaps 2 items described by x in y.
>>> amV  =: (0 {:: [)`(1 {:: [)`]}
>>>
>>> swap =: (((0 { [) ;~ ]  {~  1 { [) amV ] amV~  (1 { [) ;~ ]  {~  0 { [)
>>>
>>>
>>> timespacex '(52 | i. 2000 2) swap reduce Alpha_j_'
>>> 0.0180793 412160
>>>
>>> timespacex '(":("1) 52 | i. 2000 2) (".@[ swap ])/ Alpha_j_'
>>> 0.00243008 2.32998e6
>>>
>>> timespacex '(52 | i. 2000 2) swap reduceC ([) i.52'
>>> 0.0193757 2.2007e6
>>>
>>> a [ for v argument to reduceC is same as prepending to y and letting it fill.  The swap function accesses x parameters by index and so it is harmless to expand it.
>>> ----------------------------------------------------------------------
>>> For information about J forums see http://www.jsoftware.com/forums.htm
>
>>
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm