[Jprogramming] Mass amend
greg heil
gheil.j at gmail.com
Wed May 26 21:56:06 UTC 2021
Harvey
Another take
on viewing such
DOHLCV data
~greg heil
https//picsrp.github.io
---
NB. Date Open High Low Close Volume
f=: filer=:|:":each ".;._2]0 :0 NB. from Raul
'2020-03-06';126.70;128.33;124.52;127.73;8193300
'2020-03-09';120.16;122.41;117.28;117.81;10757500
'2020-03-10';122.78;124.88;115.76;124.77;11410700
'2020-03-11';121.01;122.58;116.38;117.97;8446500
'2020-03-12';109.65;109.80;102.28;102.81;12512700
)
NB. adverb & verbs to make data numeric
cBoxs =: 1 : '( u &.> x{y) x} y' NB. adverb to update box array
dedash =: #~ '-' &~: NB. monadic hook to dedash
dedash =: 3 : '(#~''-''&~:)y' NB. explicit ver
numb =: ". NB. make numeric
Note'egs, uses on f data'
nx =: }.i.6
nx numb cBoxs f NB. just numeric rows
0 dedash cBoxs f NB. just date row
(i.6) numb cBoxs 0 dedash cBoxs f NB. both chained
)
NB. additional verbs to scale data rows
NB. can be used adlib on numeric data with adv cBoxs
same=: [
sqrt=: %:
pct =: 100&*
ln =: ^.
l10 =: 10&^.
pln =: <. @ pct @ ln NB. int part of % for LN
pl10 =: <. @ pct @ l10 NB. " for l10
Note'eg usage'
fn=: (i.6) numb cBoxs 0 dedash cBoxs f
1 pl10 cBoxs fn
)
--
from: chris burke <cburke at jsoftware.com>
to: Programming forum <programming at jsoftware.com>
date: May 17, 2021, 6:37 AM
subject: Re: [Jprogramming] Mass amend
> I'm at a point where I would like to learn how to write GUIs in J and how to write visually oriented J programs (such as stock market charting programs). I think it may be challenging because, from my initial investigating, there seems to be *extremely* little information (including coding samples) available about these aspects of J programming.
In the Jqt session, examples of gui code are at:
Help|Studio|Qt Demos
Help|Studio|Showcase
Tools|Package Manager
The Qt interface is documented starting from page
https://code.jsoftware.com/wiki/Guides/Window_Driver
--
from: HH PackRat <hhpackrat at gmail.com>
to: programming at jsoftware.com
date: May 16, 2021, 10:36 PM
subject: Re: [Jprogramming] Mass amend
>The following (which I did show) was the untransposed original version, downloaded from Yahoo Finance:
Date Open High Low Close Volume
>+-----------------------------------------------+
>¦2020-03-06¦126.70¦128.33¦124.52¦127.73¦8193300 ¦
>+----------+------+------+------+------+--------¦
>¦2020-03-09¦120.16¦122.41¦117.28¦117.81¦10757500¦
>+----------+------+------+------+------+--------¦
>¦2020-03-10¦122.78¦124.88¦115.76¦124.77¦11410700¦
>+----------+------+------+------+------+--------¦
>¦2020-03-11¦121.01¦122.58¦116.38¦117.97¦8446500 ¦
>+----------+------+------+------+------+--------¦
>¦2020-03-12¦109.65¦109.80¦102.28¦102.81¦12512700¦
>+-----------------------------------------------+
>What I did then was to transpose it (and called it filer = file r) and applied the following instructions which retained only the Date, High, Low, and Close data:
filer=. 5 {. filer
filer=. 1 0 1 1 1 # filer
>These were followed by the J code I showed in my previous email.
>hanks so much, Raul, for your response! I'm 75 and continue to learn new things from replies to questions I've posed to this J programming group. I have always loved learning during my entire life so far, and J is a delightful challenge for my mental skills. Older folk like me often work crossword puzzles or Sudoku to keep their minds sharp--I do J programming for various computer software challenges! I programmed as a hobby from 1975 to 2009, using various versions of BASIC over the years. (I owned an original MITS Altair 8800, for which Bill Gates and Paul Allen wrote a BASIC interpreter and thus started Microsoft. Subsequently, I owned an Amiga 1000, and multiple IBM PC clones in succeeding years.) In 2006 my library's IT department head suggested that I might take a look at J, and, as they say, the rest is history.
>Although I'm now generally comfortable programming in J (except for occasional stumpers for which I need to ask for help in this group), I'm at a point where I would like to learn how to write GUIs in J and how to write visually oriented J programs (such as stock market charting programs). I think it may be challenging because, from my initial investigating, there seems to be *extremely* little information (including coding samples) available about these aspects of J programming.
Again, thanks so much, Raul!
--
from: Raul Miller <rauldmiller at gmail.com>
to: Programming forum <programming at jsoftware.com>
date: May 15, 2021, 11:00 PM
subject: Re: [Jprogramming] Mass amend
Ok...
>You left out your example data for 'filer', but you gave enough detail that I think I can see what you are trying to do.
>That said, I think that I would split this up into two different routines:
>One routine would deal with unpacking the "rows" (or "inverted columns) from your data and amending the result. (And possibly either converting back to character or ensuring that both character and numeric arguments are supported or possibly not - for now, I'll follow the pattern from your example.)
>The other routine would deal with the numeric "rescaling". We could either have different routines here (one for each different kind of rescaling) or we could use a switch statement and pack them all together like you did. Either way works. This routine would be an argument for the other routine. (So the other routine would be an adverb.)
In other words:
NB. sample data
NB. Date Open High Low Close Volume
filer=:|:":each ".;._2]0 :0
'2020-03-06';126.70;128.33;124.52;127.73;8193300
'2020-03-09';120.16;122.41;117.28;117.81;10757500
'2020-03-10';122.78;124.88;115.76;124.77;11410700
'2020-03-11';121.01;122.58;116.38;117.97;8446500
'2020-03-12';109.65;109.80;102.28;102.81;12512700
)
hiloclose=: 0 0 1 1 1 0
amender=:{{
ndx=. I.x NB. index vector from bit vector
update=. u 0&".@> x#y
(<"0 update) ndx} y
}}
rescaler=:{{
select. pricetype=. x
case. 1 do. y
case. 2 do. 10 * %: y
case. 3 do. 100 * 10 ^. y
end.
}}
Example use:
hiloclose 1&rescaler amender filer
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦126.7 ¦120.16 ¦122.78 ¦121.01 ¦109.65 ¦
>+----------+----------+----------+----------+----------¦
>¦128.33 ¦122.41 ¦124.88 ¦122.58 ¦109.8 ¦
>+----------+----------+----------+----------+----------¦
>¦124.52 ¦117.28 ¦115.76 ¦116.38 ¦102.28 ¦
>+----------+----------+----------+----------+----------¦
>¦127.73 ¦117.81 ¦124.77 ¦117.97 ¦102.81 ¦
>+----------+----------+----------+----------+----------¦
>¦8193300 ¦10757500 ¦11410700 ¦8446500 ¦12512700 ¦
>+------------------------------------------------------+
hiloclose 2&rescaler amender filer
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦126.7 ¦120.16 ¦122.78 ¦121.01 ¦109.65 ¦
>+----------+----------+----------+----------+----------¦
>¦113.283 ¦110.639 ¦111.75 ¦110.716 ¦104.785 ¦
>+----------+----------+----------+----------+----------¦
>¦111.589 ¦108.296 ¦107.592 ¦107.88 ¦101.134 ¦
>+----------+----------+----------+----------+----------¦
>¦113.018 ¦108.54 ¦111.7 ¦108.614 ¦101.395 ¦
>+----------+----------+----------+----------+----------¦
>¦8193300 ¦10757500 ¦11410700 ¦8446500 ¦12512700 ¦
>+------------------------------------------------------+
hiloclose 3&rescaler amender filer
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦126.7 ¦120.16 ¦122.78 ¦121.01 ¦109.65 ¦
>+----------+----------+----------+----------+----------¦
>¦210.833 ¦208.782 ¦209.649 ¦208.842 ¦204.06 ¦
>+----------+----------+----------+----------+----------¦
>¦209.524 ¦206.922 ¦206.356 ¦206.588 ¦200.979 ¦
>+----------+----------+----------+----------+----------¦
>¦210.629 ¦207.118 ¦209.611 ¦207.177 ¦201.204 ¦
>+----------+----------+----------+----------+----------¦
>¦8193300 ¦10757500 ¦11410700 ¦8446500 ¦12512700 ¦
>+------------------------------------------------------+
>Now... of course, you might actually want to avoid the amends that I assumed you wanted from your textual description. Or you might want other things to be different. So this is only an approximation of what you might do.
>Also, beware that copy and paste may replace some space characters with characters which look like spaces but which are not (and may eliminate some newline characters). Worst case you'll have to type the code in by hand.
>And, of course, rather than have the three different rescaling verbs be 1&rescaler 2&rescaler and 3&rescaler you might instead give each of them a significant name.
Still... perhaps this would be useful to you.
Take care,
Raul
--
from: HH PackRat <hhpackrat at gmail.com>
to: programming at jsoftware.com
date: May 15, 2021, 10:01 PM
subject: Re: [Jprogramming] Mass amend
>Hello--I was out of the house for much of today, so I haven't been able to respond until now.
>Thanks to all who responded with suggestions! Unfortunately, none of them worked as is, probably because you're used to a different way of approaching the problem and also because I probably should have indicated a fake example to show what I was looking for. Here are the final versions of the code to accomplish my goal--please don't gasp over the fact that I could create such "horrendous" looking lines of J code. They work, and that's all I care about (the 2 files at the beginning are file r1 and file r2 derived from file r). I got the ideas here and there from the code you submitted. The following lines of code successfully accomplished my goal:
filer1=. 1 0 0 0 # filer NB. dates
filer2=. 0 1 1 1 # filer NB. work with high/low/close data
if. 1 = pricetype do. NB. arithmetic scale
price=. filer2
elseif. 2 = pricetype do. NB. square-root scale
price=. < every 10&*%: ". > every filer2 NB. x10
elseif. 3 = pricetype do. NB. logarithmic scale
price=. < every 100&*10&^. ". > every filer2 NB. x100
end. NB. if
>Mike Day asked, "You’ve already had some helpful, constructive, replies. All I’ve got to add is to wonder why you’re working with boxed data if your data are as regular as your example suggests."
>I work with boxed data because that's the way J imports stock market data:
Date Open High Low Close Volume
>+-----------------------------------------------+
>¦2020-03-06¦126.70¦128.33¦124.52¦127.73¦8193300 ¦
>+----------+------+------+------+------+--------¦
>¦2020-03-09¦120.16¦122.41¦117.28¦117.81¦10757500¦
>+----------+------+------+------+------+--------¦
>¦2020-03-10¦122.78¦124.88¦115.76¦124.77¦11410700¦
>+----------+------+------+------+------+--------¦
>¦2020-03-11¦121.01¦122.58¦116.38¦117.97¦8446500 ¦
>+----------+------+------+------+------+--------¦
>¦2020-03-12¦109.65¦109.80¦102.28¦102.81¦12512700¦
>+-----------------------------------------------+
>There are 3 types of data: Date, OHLC (rationals w/decimal fractions), V (integers only). (I realize that integers are a subset of rationals, but Volume can never have decimal fractions, so I considered it a third type of data for my purposes.) Sorry for the appearance of the table (it's copy-and-paste data), but that's the main problem with proportional fonts, which I have no control over. (Or, more accurately, I'm not aware of any control over fonts that I may have in Gmail.) The above table is the starting point for my J program, which spends the remainder of the code manipulating the data in various ways. (For the following, Open prices and Volume are omitted.) In case you're interested, the following 3 tables show the results for arithmetic ("normal") scaling, square-root scaling (x10), and logarithmic scaling (x100) for IBM stock in the middle of the March 2020 "crash":
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦128.33 ¦122.41 ¦124.88 ¦122.58 ¦109.80 ¦
>+----------+----------+----------+----------+----------¦
>¦124.52 ¦117.28 ¦115.76 ¦116.38 ¦102.28 ¦
>+----------+----------+----------+----------+----------¦
>¦127.73 ¦117.81 ¦124.77 ¦117.97 ¦102.81 ¦
>+------------------------------------------------------+
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦113.28 ¦110.64 ¦111.75 ¦110.72 ¦104.79 ¦
>+----------+----------+----------+----------+----------¦
>¦111.59 ¦108.30 ¦107.59 ¦107.88 ¦101.13 ¦
>+----------+----------+----------+----------+----------¦
>¦113.02 ¦108.54 ¦111.70 ¦108.61 ¦101.40 ¦
>+------------------------------------------------------+
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦210.83 ¦208.78 ¦209.65 ¦208.84 ¦204.06 ¦
>+----------+----------+----------+----------+----------¦
>¦209.52 ¦206.92 ¦206.36 ¦206.59 ¦200.98 ¦
>+----------+----------+----------+----------+----------¦
>¦210.63 ¦207.12 ¦209.61 ¦207.18 ¦201.20 ¦
>+------------------------------------------------------+
>And here are the same scalings for the Dow-Jones Industrial Average (which has far higher prices):
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦25994.38 ¦24992.36 ¦25020.99 ¦24604.63 ¦22837.95 ¦
>+----------+----------+----------+----------+----------¦
>¦25226.61 ¦23706.07 ¦23690.34 ¦23328.32 ¦21154.45 ¦
>+----------+----------+----------+----------+----------¦
>¦25864.77 ¦23851.02 ¦25018.16 ¦23553.22 ¦21200.61 ¦
>+------------------------------------------------------+
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦1612.28 ¦1580.90 ¦1581.80 ¦1568.59 ¦1511.22 ¦
>+----------+----------+----------+----------+----------¦
>¦1588.29 ¦1539.68 ¦1539.17 ¦1527.36 ¦1454.46 ¦
>+----------+----------+----------+----------+----------¦
>¦1608.25 ¦1544.38 ¦1581.71 ¦1534.71 ¦1456.04 ¦
>+------------------------------------------------------+
>+------------------------------------------------------+
>¦2020-03-06¦2020-03-09¦2020-03-10¦2020-03-11¦2020-03-12¦
>+----------+----------+----------+----------+----------¦
>¦441.49 ¦439.78 ¦439.83 ¦439.10 ¦435.87 ¦
>+----------+----------+----------+----------+----------¦
>¦440.19 ¦437.49 ¦437.46 ¦436.79 ¦432.54 ¦
>+----------+----------+----------+----------+----------¦
>¦441.27 ¦437.75 ¦439.83 ¦437.21 ¦432.63 ¦
>+------------------------------------------------------+
>The program is a "ZigZag" (or "Hi-Lo") program to determine where price highs and lows occur in a set of market data, giving the user 5 different sets of options to instruct the program how to operate. The end result of the program is NOT a chart with lines joining the highs and lows as would be typical with a zigzag program (I currently know nothing about creating market charts in J). Rather, the output of the program is a list of dates, each followed by the letter "H" (for a high on the date) or the letter "L" (for a low on the date) which can then be combined together with other kinds of data to achieve various purposes.
>Again, a big THANK YOU to everyone who suggested various solutions to my query! I greatly appreciate you all!
--
from: 'Michael Day' via Programming <programming at jsoftware.com>
to: programming at jsoftware.com
date: May 15, 2021, 2:51 PM
subject: Re: [Jprogramming] Mass amend
>Depends what Harvey means by "values", I suppose. I assumed he meant scalars. Otherwise my observation is pretty useless!
Cheers,
Mike
--
from: Joey K Tuttle <jkt at qued.com> via gmail.com
to: programming at jsoftware.com
date: May 15, 2021, 11:01 AM
subject: Re: [Jprogramming] Mass amend
(],%:) &.> <"0 ]i. 2 5
>+-------------+-------------+-------------+-------------+---+
>|0 0 |1 1 |2 1.414213562|3 1.732050808|4 2|
>+-------------+-------------+-------------+-------------+---+
>|5 2.236067977|6 2.449489743|7 2.645751311|8 2.828427125|9 3|
>+-------------+-------------+-------------+-------------+---+
- joey
--
from: 'Mike Day' via Programming <programming at jsoftware.com>
to: programming at jsoftware.com
date: May 15, 2021, 2:16 AM
subject: Re: [Jprogramming] Mass amend
Sorry... another early send!
>You’ve already had some helpful, constructive, replies. All I’ve got to add is to wonder why you’re working with boxed data if your data are as regular as your example suggests.
>If you really do have m rows of n boxed numeric values, why not just open it up with > ? You can do a lot of processing on the whole array. Box it all up again at the end if really necessary with <“0 (boxed scalars) or <“1 (boxed rows)
But perhaps I’ve misunderstood your requirements,
Cheers,
Mike
--
from: Hauke Rehr <hauke.rehr at uni-jena.de> via gmail.com
to: programming at jsoftware.com
date: May 15, 2021, 1:15 AM
subject: Re: [Jprogramming] Mass amend
>The fact that scaling a logarithm is equivalent to using a different base doesn’t mean changing the base mirrors the semantics of every occurrence of (factor&*&^.) Sometimes you just want to /scale this value up/. And the base you’d come up with that is equivalent to this wouldn’t have any meaning to the problem at hand.
Other than that:
>You’re right but I think HH wants to keep things simple, mirroring what he had in mind and can easily understand.
--
from: Raul Miller <rauldmiller at gmail.com>
to: Programming forum <programming at jsoftware.com>
date: May 15, 2021, 1:04 AM
subject: Re: [Jprogramming] Mass amend
Hmm... why would we do that?
>^. gives the natural log, if we wanted to work with base 10 logs we'd be using 10&^. instead of a bare ^. and if we're treating logs as percentages we'd probably be working with log10
(But what is the domain where we'd be wanting logs-as-percent?)
Thanks,
--
from: Hauke Rehr <hauke.rehr at uni-jena.de> via gmail.com
to: programming at jsoftware.com
date: May 15, 2021, 12:21 AM
subject: Re: [Jprogramming] Mass amend
hh_constant =: 100
scaled_logs =: hh_constant&*&^. each original
Am 15.05.21 um 06:46 schrieb Raul Miller:
> logs=: ^. each original
--
from: Raul Miller <rauldmiller at gmail.com>
to: Programming forum <programming at jsoftware.com>
date: May 14, 2021, 9:47 PM
subject: Re: [Jprogramming] Mass amend
>To expand on greg heil's suggestion, and to try and interpret what you suggested, it sounds like you want something like this:
original=: <"0 i.3 1000
roots=: %:each original
logs=: ^. each original
>But you also talked about amending all of the values, so maybe you want
original=: roots
or
original=: logs
>Though, obviously, if you do both of those updates to the 'original', one of them would overwrite the other.
>So, if that's not what you want, perhaps you could provide a concrete example of what you want using a smaller array? (It's straightforward to scale up an approach in size, but examples which fit in an email message need to be small enough to fit in an email message).
Thanks,
More information about the Programming
mailing list