#213164 - 2018-01-27 12:02 AM
functions and time of execution
|
AndreLuiz
Getting the hang of it
Registered: 2015-10-07
Posts: 89
Loc: Brasil, João pessoa
|
[PTBR]
Olá para toda da comunidade KiXtart.
Bom KiXtart é uma das minhas linguagens favoritas.
E eu gostaria que todos pudessem fazer parte dessa publicação que se trata de uma busca por melhres algortmos.
Bom é o seguinte, a um tempo atrás eu tinha feito uma função que fazia o mesmo que o Redim faz hahah(não conhecia o comando).
E com depois descobri quando fiz essa alteração eu acabei percebendo que almentou o tempo de execução do script.
Fiz uma função parecida com o Ascan(não sabia que ela existia também), ai quando descobri e alterei tive uma melhora muito significativa para o tempo de execução do meu script.
E isso foi com o instr, join, etc...
Esses dias quis fazer o seguinte:
Redim preserve $Var[1][3]
E não consegui, ele me joga um erro.
Poderia fazer isso acima com uma função, mas ai eu vou ter um custo de tempo na hora da execução.
Ah um forma de fazer só que mais rápido.
E pedi que todo vinhesem responder este tópico afim de demontrar tipos de otmização para o kixtart.
Acho que seria muito benéfico que o kixtart posuisse algumas funções nativas como manipulação de array(removeId), uma forma mais produtiva de concatenar strings, alterar o split para trabalhar de uma outra forma também, ou uma função expecifica para isso:
$x = Split("KiX", "") Join($x, "-"); K-i-X
E tudo isso porque acredito que essas pequenas coisas que nós já fazemos na linguagem kixtart, podem ficar ainda mais rápida se forem implementada na linguagem nativamente. O KiXtart é desenvolvido em que linguagem é C?
Oque vocês acham, vocês podem me dá direcionamento melhor sobre este assunto?
[ENG]
Hello to all of the KiXtart community
Good KiXtart is one of my favorite languages.
And I would like everyone to be part of this publication that it is a search for better algorithms.
Well it's the following, a while ago I had done a function that did the same as Redim does hahah (did not know the command).
And then I discovered when I made this change I finally realized that it increased the execution time of the script.
I did a similar function with Ascan (did not know it existed as well), then when I discovered and changed it I had a very significant improvement to the execution time of my script.
And that was with instr, join, etc ...
These days I wanted to do the following:
Redim preserve $ Var [1] [3]
And I could not, he throws me a mistake.
I could do this above with a function, but then I'll have a cost of time at the time of execution.
Ah a way to do just that faster.
And I asked that they all come to answer this topic in order to demonstrate types of optimization for kixtart.
I think it would be very beneficial if kixtart had some native functions like array manipulation (removeId), a more productive way of concatenating strings, changing the split to work in another way as well, or a specific function for this:
$ x = Split ("KiX", "") Join ($ x, "-"); K-i-X
And all this because I believe that these little things we already do in the kixtart language can get even faster if they are implemented in the language natively. Is KiXtart developed in what language is C?
What do you think, can you give me better direction on this subject?
|
|
Top
|
|
|
|
#213165 - 2018-01-27 04:11 PM
Re: functions and time of execution
[Re: AndreLuiz]
|
Glenn Barnas
KiX Supporter
Registered: 2003-01-28
Posts: 4396
Loc: New Jersey
|
1 - RTFM!! Fall asleep with your face in the pages!  When I first started using Kix, I created a spreadsheet (Excel 97 ??) with the function/command name, a very short description, and a field that indicated it's potential usefulness. Knowing what's available is half the challenge (as you have seen). I took this one step further just this week, creating a script to index all the functions in my library to create a handy reference table.
2 - Review the function library (www.kixtart.org/udf) to see them all indexed. As much as we all say to treat them like "black boxes", they can provide a wealth of brain food - interesting ideas and concepts for often unrelated functionality.
3 - Keep sharing and asking questions - we've all learned from each other, collaborated on projects, and the result has been some pretty awesome code.
4 - Join our Golf tournaments - we could use some more people on the course! Several of these have directly resulted in useful UDFs that I use quite often in my production code. After more than 20 years of writing Kix scripts, I'm amazed that I always learn something new in the Golf rounds.
Of course, the quality of your posts show that you probably already know most of this stuff, but a reminder never hurts. Plus, others less familiar would not be hurt by reading this and following such guidelines.
This is the best place to post such questions as it's the forum that Ruud always scans for issues and ideas. Several functions that have been suggested have made it into Kix over the years, Replace being one of the newest that I can recall.
Here's some answers to your questions above.
ReDim Preserve $Var[1][3]
This won't work because there are two separate arrays referenced by this - an "array of arrays". If you think about this, the inner array doesn't need to be a fixed size. I can create, for example, an outer array of family members, the inner array containing a list of their children. My mom had 5 brothers and sisters, so her outer array would have 6 elements (0-5). She had 2 kids, 2 of her sisters had 3 kids, one sister had 4 kids, her brother had 2, and her other brother had none. Thus, the inner arrays would each be different lengths.
If you want to redefine both array sets because your inner arrays are all a consistent size, you'd need to process the inner arrays first, then the outer array.
For $X = 0 to UBound($Var)
$aTmp = $Var[$X] ; Load inner array into a temp
ReDim Preserve $aTmp[3] ; resize temp array, preserving data
$Var[$X] = $aTmp ; return resized data to outer array
Next
ReDim Preserve $Var[1] ; resize outer array
SPLIT? Sure - the Split(string, null) returning an array of chars would be pretty cool, but here's a quick alternative. For this kind of thing to make it into a Kix update, it has to either be fairly trivial to implement, have a fair demand for the functionality, and overall, must not break scripts developed on older versions. This might have potential.
Break On
$A = TxtToAry('KIX')
Ubound($A) ?
Join($A, '-') ?
; Convert a text string to an array of characters
; Glenn Barnas
Function TxtToAry($_S)
Dim $_P
For $_P = 1 to Len($_S)
ReDim Preserve $TxtToAry[$_P - 1]
$TxtToAry[$_P - 1] = SubStr($_S, $_P, 1)
Next
Exit 0
EndFunction
Glenn
_________________________
Actually I am a Rocket Scientist! 
|
|
Top
|
|
|
|
|
#213166 - 2018-01-27 07:48 PM
Re: functions and time of execution
[Re: Glenn Barnas]
|
AndreLuiz
Getting the hang of it
Registered: 2015-10-07
Posts: 89
Loc: Brasil, João pessoa
|
[PTBR]
Olá Glenn! pensei que ninguém tivesse visto minha biblioteca de coleção :D, obrigado pelo reconhecimento.
Bom
Sei que fazer isto:
For $X = 0 to UBound($Var)
$aTmp = $Var[$X] ; Load inner array into a temp
ReDim Preserve $aTmp[3] ; resize temp array, preserving data
$Var[$X] = $aTmp ; return resized data to outer array
Next
ReDim Preserve $Var[1] ; resize outer array
É trivial, de fato! Mas a ideia não é essa, é ser rápido, eu tenho uma sub-matriz que tem muitos dados carregados nela!
Veja, se caso eu faço isso que você me propõe(que eu já faço isso atualmente como solução), eu crio um novo objeto(string), manipulo aumentando ele com o redim(que é um comando que tem um custo de execução), e depois jogo para variável de novo(este processo é até leve), e é isso perco bastante tempo, principalmente se ele estiver dentro de um laço for.
Oque quero com isso é diminuir tempo de execução! entende?
Bom e sobre o seu código de quebrar um string em char, ele é lento pelo fato de usar um "redim PRESERVE", dentro de um laço for, uma melhor alternativa para não ter que usar o redim a quantidade de vezes do comprimento de uma string, seria fazer isso:
break on
$x=StrToArray("KiX")
ubound($x)?
Join($x, "-")?
get$
function StrToArray($text)
Dim
$len
$len = Len($text)-1
Redim
preserve $StrToArray[$len]
for $i=0 to $len
$StrToArray[$i] = Substr($text, $i+1, 1)
next
endfunction
Isso torna tudo mais rápido, claro que para um string "KiX" tanto faz(acredito), mas para uma string com 20\+ caracteres, você percebera uma enorme diferença!
Mas mesmo assim, creio que se isso fosse feito por debaixo da infraestrutura de KiX seria 4x mais rápido.
Ah, obrigado pelo convite, depois eu entro lá para ver oque se passa lá, mas não sei exatamente do que se trata.
Ah, você ou alguém tem alguma noticia de quando chegará uma nova versão do KiXtart? E oque trará de novo, ou alguma coisa do tipo.
Obrigado por responder!
[ENG]
Hello Glenn! Think of my collection library: D, thanks for the recognition.
Good
I know what to do is:
For $X = 0 to UBound($Var)
$aTmp = $Var[$X] ; Load inner array into a temp
ReDim Preserve $aTmp[3] ; resize temp array, preserving data
$Var[$X] = $aTmp ; return resized data to outer array
Next
ReDim Preserve $Var[1] ; resize outer array
It's trivial, indeed! But an idea is not this, it's fast, I have a sub-array that has a lot of data loaded into it!
See, if I do that you propose to me, I create a new object (string), manipulating it with redeim (which is a command that has a running cost), and then play to variable again (this process is until light), and that's it I lose a lot of time, especially if it's inside a loop for.
What I want with this is to reduce execution time! you see?
Good and about your code breaking a string in char, it is slow by the fact of using a "redeim PRESERVE" inside a loop to, a better alternative to not having to use the redeim a number of times of length of a string, would do this:
break on
$x=StrToArray("KiX")
ubound($x)?
Join($x, "-")?
get$
function StrToArray($text)
Dim
$len
$len = Len($text)-1
Redim
preserve $StrToArray[$len]
for $i=0 to $len
$StrToArray[$i] = Substr($text, $i+1, 1)
next
endfunction
This is your newest language, of course for a "KiX" string (I believe), but for a string with 20+ characters, you've noticed a huge difference!
But even so, I think if it were done underneath KiX's infrastructure it would be 4x faster.
Oh, thanks for the invitation, then I'll go in there to see what's going on there, but I do not know exactly what it's all about.
Oh, do you or anyone else have any news of when a new version of KiXtart will arrive? And what it will bring again, or something.
Thanks for answering! : D
|
|
Top
|
|
|
|
|
#213167 - 2018-01-27 11:04 PM
Re: functions and time of execution
[Re: AndreLuiz]
|
Glenn Barnas
KiX Supporter
Registered: 2003-01-28
Posts: 4396
Loc: New Jersey
|
This was just a quick example that I threw together. I don't think the performance is related to the ReDim function, however.
I replaced "'KIX' with a file of about 25Kbytes using the FileIO function. The file load consistently takes 15-16ms to load 282 lines / 25328 chars into an array of lines.
Running this with the original logic, performing a ReDim for each character took about 3.7-3.9 seconds.
I modified the function to redim only when the counter exceeds the array size, and with a ReDim value of 15000, it resizes the array only 3 times (twice during processing and once at the end to trim to actual size. This was a common method in earlier versions where array management seemed to require a lot of overhead.
So - I converted the array of source data to a single large string ($D = Join($Array, @CRLF) and passed that to the function, timing with millisecond accuracy. The process took 3.766 seconds - consistently +/- 20ms.
I then changed the logic to not use SubStr() on such a large string. Passing the 282 lines to the function one at a time, and even including the overhead of adding the small arrays to a single large array as originally done took just 235ms, with +/- 10ms consistency.
Here's my test code - you'll need the FileIO and TimeDiff functions and either insert them or change the Call reference path.Break On
Call '%KIXLIBPATH%\FileIO.kxf'
Call '%KIXLIBPATH%\TimeDiff.kxf'
$S = @DATE + ' ' + @Time + '.' + Right('000' + @MSECS, 3)
$D = FileIO('C:\Temp\Test.txt', 'R')
$E = @DATE + ' ' + @Time + '.' + Right('000' + @MSECS, 3)
'Data Load -' ?
' Size: ' Len(Join($D, @CRLF)) ?
'Start: ' $S ?
' End: ' $E ?
'Total: ' TimeDiff($S, $E, ,1) ? ?
$S = @DATE + ' ' + @Time + '.' + Right('000' + @MSECS, 3)
$A = TxtToAry(Join($D, @CRLF))
$E = @DATE + ' ' + @Time + '.' + Right('000' + @MSECS, 3)
'Conversion by Single Text Block-' ?
'Start: ' $S ?
' End: ' $E ?
'Total: ' TimeDiff($S, $E, ,1) ?
'Array: ' 1 + Ubound($A) ? ?
$ = FileIO('C:\Temp\test1.txt', 'W', $A)
Dim $aF
$S = @DATE + ' ' + @Time + '.' + Right('000' + @MSECS, 3)
$C = -1
For Each $Line in $D
$A = TxtToAry($Line + @CRLF)
$T = UBound($A)
$F = UBound($aF)
ReDim Preserve $aF[1 + $F + $T]
For $I = 0 to $T
$C = $C + 1
$aF[$C] = $A[$I]
Next
Next
$E = @DATE + ' ' + @Time + '.' + Right('000' + @MSECS, 3)
'Conversion by Line -' ?
'Lines: ' 1 + UBound($D) ?
'Start: ' $S ?
' End: ' $E ?
'Total: ' TimeDiff($S, $E, ,1) ?
'Array: ' 1 + Ubound($aF) ? ?
$ = FileIO('C:\Temp\test2.txt', 'W', $aF)
; Convert a text string to an array of characters
; Glenn Barnas
Function TxtToAry($_S)
Dim $_P, $_C
$_C = -1
For $_P = 1 to Len($_S)
$_C = $_C + 1
If $_C >= UBound($TxtToAry)
ReDim Preserve $TxtToAry[100 + $_C]
EndIf
$TxtToAry[$_P - 1] = SubStr($_S, $_P, 1)
Next
ReDim Preserve $TxtToAry[$_C]
Exit 0
EndFunction
Clearly, it seems that the performance difference comes from the SubStr function handling very large strings and not from the ReDim.
Glenn
_________________________
Actually I am a Rocket Scientist!
|
|
Top
|
|
|
|
Moderator: Lonkero, ShaneEP, Jochen, Radimus, Glenn Barnas, Allen, Ruud van Velsen, Mart
|
0 registered
and 264 anonymous users online.
|
|
|
