#191058 - 2008-12-12 03:08 PM
Read a file and randomly pick entries
|
Willmaykett
Fresh Scripter
Registered: 2007-01-07
Posts: 21
|
Hi all,
I hope some talented person can help me.
I have an application the generates a text file with a series of entries in it and I need to generate a file that randomly picks x amount of entries (where x is a number I need). For example
Widget1 widget2 Widget5 Widget9 Widget10
Now, the number of entries, ie widgets, can vary, from zero to 20 or more in it.
What I need to do is be able to RANDOMLY pick a predetermined number of widget entries. So that if I say pick 2 entries, I might get Widget5 and 9, then next time it might be 1 and 10.
I think I need to: 1. Read the file and create an array. 2. Work out how many entries are in the array. 3. Pick a predetermined number that I want. 4. Randomly pick some of the entries based on the number I want. 5. Then generate a new file with those random entries.
Make sense.
I need some serious help in making some code, and would seriously appreciate the help!
Thanks, Wil
|
|
Top
|
|
|
|
#191063 - 2008-12-12 03:56 PM
Re: Read a file and randomly pick entries
[Re: Willmaykett]
|
eriqjaffe
Hey THIS is FUN
Registered: 2004-06-24
Posts: 214
Loc: Arlington Heights, IL USA
|
quick & dirty:
break on
global $RandomCallCounter
$Array = RandomizeArray(ReadFile("\\path\to\infile.txt"))
$arraysize = ubound($array) + 1
? "The array contains "+$arraysize+" elements. How many do you want to pick? "
gets $howmany
$ = open(1,"\\path\to\newfile.txt",5)
for $x = 1 to ($howmany)
$ = WRITELINE(1,$Array[$x]+@CRLF)
next
$ = close(1)
This relies on the ReadFile() and RandomizeArray() UDFs, both of which can be found in the UDF forum. May not be the neatest way, but it'll get the job done.
Edited by eriqjaffe (2008-12-12 03:57 PM)
|
|
Top
|
|
|
|
#191076 - 2008-12-12 11:48 PM
Re: Read a file and randomly pick entries
[Re: Gargoyle]
|
Willmaykett
Fresh Scripter
Registered: 2007-01-07
Posts: 21
|
Hi guys,
Wow, thanks for the tips. Would never have been able to do what I wanted in such a small amount of code.
Thanks guys, really appreciate it !
I ran the code from eriqjaffe and it worked a treat. What extra would I need to do so that if 3 entries existed in the file, and I picked 5, that all 3 entries will be used? Currently it throws an error about being out of bounds.
The code from Gargoyle threw an error all the time, and I think I worked out it was some syntax errors. But then it generated only a series of numbers, not a list of what was in the file. Further than that, I got stumped . 
Thanks heaps for your work so far!
Edited by Willmaykett (2008-12-13 01:00 AM)
|
|
Top
|
|
|
|
#191109 - 2008-12-13 08:59 AM
Re: Read a file and randomly pick entries
[Re: Gargoyle]
|
Willmaykett
Fresh Scripter
Registered: 2007-01-07
Posts: 21
|
Hi guys,
Interestingly, it still didn't work. I had to change all the [] to () then it worked, BUT, it still isn't giving me what I want.
When I create a file with the following entries: Line1 Line2 Line3 Line4 Line5
and I run the code
srnd(@msec)
$Pick = 4 ; how many do you want
$Counter = 0
Open(1,"c:\data\file.txt")
$Line = Readline(1)
While @error = 0
redim preserve $Array[$Counter]
$Array[$Counter] = $Line
? $Line
$Counter = $Counter + 1
$Line = Readline(1)
Loop
$Size = Ubound($Array)
For $Output = 0 to $Pick
$Array(RND($size)) ?
Next
Sleep 3
the first part of the array gives me the list of all the lines, but there is a 0 in it, and the random part gives me 5 random numbers, not 4 as expected, and not the actual data
Is something a bit amiss?
Thanks
|
|
Top
|
|
|
|
#191117 - 2008-12-13 01:36 PM
Re: Read a file and randomly pick entries
[Re: Gargoyle]
|
Glenn Barnas
KiX Supporter
   
Registered: 2003-01-28
Posts: 4402
Loc: New Jersey
|
This one works - the corrections and enhancements are commented. Note that there is nothing preventing the same data from being displayed more than once - if duplicate selections is a problem, you can modify the code slightly to select an array element pointer. If the element contains data, you output it and clear the element. If the element is empty, it's been used, so reduce the counter by one instead of outputting the value.
break on
Dim $Array[0] ; Must initialize the array first!
srnd(@msec)
$Pick = 4 ; how many do you want
$Counter = -1 ; alternate, pre-increment method uses -1 init
If Open(1,"c:\data\file.txt") = 0 ; trapped the result code
$Line = Readline(1)
While @error = 0
$Counter = $Counter + 1 ; moved to pre-increment
redim preserve $Array[$Counter]
$Array[$Counter] = $Line
$Line ?
$Line = Readline(1)
Loop
$ = Close(1) ; closed the file
EndIf
$Size = Ubound($Array)
$Size + 1 ' elements were loaded' ? ; show element count
For $Output = 1 to $Pick ; counter, not array index, so 1-based
$Array[RND($size)] ? ; corrected d"(" and "["
Next
If you simply add the element count to the original code, you always get 0 (-1 ubound). That's because the array was not declared before being ReDim'd. This resulted in no data in the array.
I modified the loop to a pre-increment model (personal preference). Both work, but in pre-increment, the array index pointer always accurately reflects the array count, which could eliminate the need for the UBound in certain code applications.
I added an If to the Open to trap the result code, preventing the extraneous zero from appearing on the console, as well as added the Close call.
The Element Count message pointed to the original problem of an uninitialized array.
Finally, the parens and square braces were messed up again. I also changed the For loop to be 1-based. This is why you were getting an extra item. The $Output value is a counter of the number of selections, so should be 1-based. The Rnd function returns a value between 0 and the max value (array size) and is the array element index, which is properly zero-based.
Here's an alternate loop to prevent duplicate selections:
For $Output = 1 to $Pick
$Index = RND($size) ; select a random element
If $Array[$Index] ; does it contain data?
$Array[$Index] ? ; then display it
$Array[$Index] = '' ; and clear it to prevent re-use
Else
$Output = $Output - 1 ; reduce the output count so we try again
EndIf
Next This assumes you don't need the array data. If you need to maintain the array data, you can duplicate the array and use the duplicate for controlling the output.
Glenn
_________________________
Actually I am a Rocket Scientist!
|
|
Top
|
|
|
|
#191180 - 2008-12-15 07:43 AM
Re: Read a file and randomly pick entries
[Re: Glenn Barnas]
|
Willmaykett
Fresh Scripter
Registered: 2007-01-07
Posts: 21
|
Thanks Glenn, really appreciate the help so far!
What I intend doing is running this script as a scheduled task once a week to randonly pull out the required number of entries from my file. What sometimes may happen is that the extract may contain only 4 entries, but my "picker" will normally be set for 5.
I am going to use the "alternate loop" code above, and make the following additions:
$Size = Ubound($Array)
$Size + 1 ' elements were loaded' ? ; show element count
$SizeTotal = $Size + 1
$SizeTotal ?
If $SizeTotal =< $Pick
"Entries in file less than or equal to $PICK" ?
Sleep 3
Exit
EndIf
What I also noticed is that the entries picked don't appear to be that "random"
for example, My file with: Widget1 Wigdet2 Widget3 Widget4 Widget5
and pick of 2 always gives me Widget1 Widget2
Thought it would be a bit more random than that...
Thanks heaps for all you help so far.
Edited by Willmaykett (2008-12-15 09:29 AM) Edit Reason: Made changes to code
|
|
Top
|
|
|
|
#191184 - 2008-12-15 12:11 PM
Re: Read a file and randomly pick entries
[Re: Richard H.]
|
Willmaykett
Fresh Scripter
Registered: 2007-01-07
Posts: 21
|
This is my (mostly Glenn's actually) code so far.
break on
Dim $Array[0] ; Must initialize the array first!
SRND(@msec)
$Pick = 2 ; how many do you want
$Counter = -1 ; alternate, pre-increment method uses -1 init
If Open(1,"c:\data\file.txt") = 0 ; trapped the result code
$Line = Readline(1)
While @error = 0
$Counter = $Counter + 1 ; moved to pre-increment
redim preserve $Array[$Counter]
$Array[$Counter] = $Line
$Line = ReadLine(1)
Loop
$ = Close(1) ; closed the file
EndIf
$Size = Ubound($Array)
$Size + 1 ' elements were loaded' ? ; show element count
$SizeTotal = $Size + 1
$SizeTotal ?
If $SizeTotal =< $Pick
"Entries in file less than or equal to $PICK" ?
Sleep 3
Exit
EndIf
For $Output = 1 to $Pick
$Index = RND($size) ; select a random element
If $Array[$Index] ; does it contain data?
$Array[$Index] ? ; then display it
$Array[$Index] = '' ; and clear it to prevent re-use
Else
$Output = $Output - 1 ; reduce the output count so we try again
EndIf
Next
Sleep 3
So I am not sure where I am supposed to put what you suggest. With my test file, it always returns: Widget1 Widget2
and never a "random" selection with any of the other text in the file?
Thanks
|
|
Top
|
|
|
|
#191188 - 2008-12-15 03:17 PM
Re: Read a file and randomly pick entries
[Re: Willmaykett]
|
Richard H.
Administrator
   
Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
|
Ok, here is the way I would do it:
Break ON
$fso = CreateObject("Scripting.FileSystemObject")
$txt = $fso.OpenTextFile("C:\data\file.txt",Not 0)
$Array=Split($txt.ReadAll,@CRLF)
$txt.Close
$=SRND(@MSECS)
$=RND(0)
$Pick=2
For $Output=1 to $Pick
$Array=Split(udfSqueeze(Join($Array,@CRLF),@CRLF),@CRLF)
$iChose=IIf(UBound($Array)>0,RND(UBound($Array)),0)
"Selection "+$Output+": "+$Array[$iChose]+@CRLF
$Array[$iChose]=""
Next
Function udfSqueeze($s,$q)
$udfSqueeze=$q+$s+$q
While InStr($udfSqueeze,$q+$q)
$udfSqueeze=Join(Split($udfSqueeze,$q+$q),$q)
Loop
$udfSqueeze=Split($q+$udfSqueeze+$q,$q+$q)[1]
EndFunction
The method strips array elements out once they have been selected, meaning that you won't hit a blank element and you won't need to retry.
Retrying is not a huge problem when your select set is a small proportion of the entire set, but as the proportion grows you will get more and more hits on an empty array element.
Of course you need to balance this against the work needed by all the split/joins.
In this example if you have 4 lines in the file and you attempt to pick 10 lines then the script will not fail, it will report null values for picks 5-10.
Edited by Richard H. (2008-12-15 03:18 PM) Edit Reason: Removed debugging code
|
|
Top
|
|
|
|
#191194 - 2008-12-16 04:15 AM
Re: Read a file and randomly pick entries
[Re: NTDOC]
|
Willmaykett
Fresh Scripter
Registered: 2007-01-07
Posts: 21
|
Hi guys,
Well, it looks like I got something working, using Richard's code and I added in the writeline stuff to generate the required file. I have tested it and it works a treat!
Thanks to all you guys who have helped me out so much. I will try an sit down and understand what it really is doing.... 
For a total newbie like me, with NO programming experience, any recommendations on resources for "scripting for dummies" like me..that starts from the beginning??
Once again, Thanks heaps you guys.
|
|
Top
|
|
|
|
Moderator: Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart
|
0 registered
and 1077 anonymous users online.
|
|
|