Page 1 of 2 12>
Topic Options
#183134 - 2007-12-02 05:01 AM File listing
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
I've got to find files and return their paths that match a pattern. The folder and subfolders contain several million files with thousands of folders. I've started running this script several hours ago, but I don't know if this is ever going to finish. Is there a more efficient way to do this? Any help would be appreciated.

 Code:
$type=Tscm0005,Cxac0005,Tsfm0033,Tsec0002,tsec0006,tsec0008,tsec0010,tscm0002,tsfm0081,tsfm0083,tsfm0084,tstm0001

	For Each $reporttype in $type	
		$reports = dirplus("f:\reports\trans\monthold","/s /a-d /m " + $reporttype)
		For Each $file in $reports   
			$=Writelog2('c:\admin\'+ $reporttype +'.txt',$file.name +";"+ $file.path)
		Next
	Next
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#183136 - 2007-12-02 09:01 AM Re: File listing [Re: brewdude6]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
That is one (IMHO good) way to look at it.
You could also take a DIRPlus() of everything and search in the result.
 Code:
$reports = dirplus("f:\reports\trans\monthold","/s /a-d")
For Each $file in $reports
	For Each $reporttype in $type
		If InStr($file.Name,$reporttype)
...

Top
#183138 - 2007-12-02 09:30 AM Re: File listing [Re: Witto]
Bryce Offline
KiX Supporter
*****

Registered: 2000-02-29
Posts: 3167
Loc: Houston TX
"The folder and subfolders contain several million files with thousands of folders."

Ouch.... Since DIRPlus has to crawl and look at each file, building an array as it goes... yeah it could take some time.

You might want to try a low tech method....

 Code:
shell '%comspec% /c "f:\reports\trans\monthold" /s /a-d | find /i "' + $reporttype + '" > Files.txt'


If this directory structure is as big as you say it is, your files.txt file could be several megs large by time it is finished....

Top
#183139 - 2007-12-02 09:38 AM Re: File listing [Re: Bryce]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
KiXtart also has Dir() function, but I think it cannot recurse subdirectories.
Top
#183140 - 2007-12-02 11:12 AM Re: File listing [Re: Witto]
NTDOC Administrator Offline
Administrator
*****

Registered: 2000-07-28
Posts: 11629
Loc: CA
sure it can, you just have to code it to do it.
Top
#183141 - 2007-12-02 01:29 PM Re: File listing [Re: NTDOC]
Arend_ Moderator Offline
MM club member
*****

Registered: 2005-01-17
Posts: 1896
Loc: Hilversum, The Netherlands
Or use Fso, works the same as Dir just imo it's faster.

 Code:
Dim $FSO, $Path
$FSO = CreateObject("Scripting.FileSystemObject")
$Path = $FSO.GetFolder("C:\")

FindFiles($Path, "boot.ini")

Function FindFiles($Root, $sfile)
  Dim $SubFolder, $file
  For Each $file In $Root.Files
    If $file.name = $sfile
      ? $file.path
    Endif
  Next
  For Each $SubFolder In $Root.SubFolders
    FindFiles($SubFolder, $sfile, $dest)
  Next
EndFunction

Top
#183143 - 2007-12-02 01:55 PM Re: File listing [Re: Arend_]
Sealeopard Offline
KiX Master
*****

Registered: 2001-04-25
Posts: 11165
Loc: Boston, MA, USA
I think Bryce's SHELL method may be the fastest with post-processing of the text file in KiXtart then. Still, rather large files/arrays for KiXtart to deal with.
_________________________
There are two types of vessels, submarines and targets.

Top
#183149 - 2007-12-02 03:47 PM Re: File listing [Re: Bryce]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
Is there a "dir" directive missing?
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#183153 - 2007-12-02 04:39 PM Re: File listing [Re: brewdude6]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
 Originally Posted By: brewdude6
Is there a "dir" directive missing?


Got it. I agree with Bryce and Sealeopard, the Dirplus routine was churning for over 13 hours with no output. This method is working fine.

 Code:
	Shell '%comspec% /c dir "f:\reports\trans\monthold" /s /a-d | find /i "' + $reporttype + '" > e:\admin\"'+$reporttype+'".txt'
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#183157 - 2007-12-02 06:33 PM Re: File listing [Re: Bryce]
brewdude6 Offline
Hey THIS is FUN

Registered: 2000-10-21
Posts: 280
Loc: Nashville, TN
Any way to do multiple find parameters for the single "dir" directive? I'd like to find all matching strings in one pass if possible.
_________________________
I could have made a neat retort but didn't, for I was flurried and didn't think of it till I was downstairs.
-Mark Twain

Top
#183159 - 2007-12-02 08:50 PM Re: File listing [Re: brewdude6]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
 Code:
If InStr($file.Name,Tscm0005) Or InStr($file.Name,Cxac0005) Or 
   InStr($file.Name,Tsfm0033) Or InStr($file.Name,Tsec0002) Or 
   InStr($file.Name,tsec0006) Or InStr($file.Name,tsec0008) Or 
   InStr($file.Name,tsec0010) Or InStr($file.Name,tscm0002) Or 
   InStr($file.Name,tsfm0081) Or InStr($file.Name,tsfm0083) Or 
   InStr($file.Name,tsfm0084) Or InStr($file.Name,tstm0001)

Top
#183161 - 2007-12-02 08:52 PM Re: File listing [Re: brewdude6]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
 Code:
If InStr($file.Name,Tscm0005) Or InStr($file.Name,Cxac0005) Or 
   InStr($file.Name,Tsfm0033) Or InStr($file.Name,Tsec0002) Or 
   InStr($file.Name,tsec0006) Or InStr($file.Name,tsec0008) Or 
   InStr($file.Name,tsec0010) Or InStr($file.Name,tscm0002) Or 
   InStr($file.Name,tsfm0081) Or InStr($file.Name,tsfm0083) Or 
   InStr($file.Name,tsfm0084) Or InStr($file.Name,tstm0001)

Top
#183162 - 2007-12-02 08:54 PM Re: File listing [Re: brewdude6]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
 Code:
If InStr($file.Name,Tscm0005) Or InStr($file.Name,Cxac0005) Or 
   InStr($file.Name,Tsfm0033) Or InStr($file.Name,Tsec0002) Or 
   InStr($file.Name,tsec0006) Or InStr($file.Name,tsec0008) Or 
   InStr($file.Name,tsec0010) Or InStr($file.Name,tscm0002) Or 
   InStr($file.Name,tsfm0081) Or InStr($file.Name,tsfm0083) Or 
   InStr($file.Name,tsfm0084) Or InStr($file.Name,tstm0001)

Top
#183163 - 2007-12-02 08:56 PM Re: File listing [Re: Witto]
Arend_ Moderator Offline
MM club member
*****

Registered: 2005-01-17
Posts: 1896
Loc: Hilversum, The Netherlands
3 times is the charm ?
Top
#183164 - 2007-12-02 08:59 PM Re: File listing [Re: Arend_]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
No, just bad response from the kixtart.org website. Response like "page not found" and try to resend the answer.
Top
#183165 - 2007-12-02 09:02 PM Re: File listing [Re: Witto]
Arend_ Moderator Offline
MM club member
*****

Registered: 2005-01-17
Posts: 1896
Loc: Hilversum, The Netherlands
lol \:\) you can delete 2 of them though \:\)
Top
#183168 - 2007-12-02 09:08 PM Re: File listing [Re: Arend_]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
I can delete the content, not the post.
Top
#183169 - 2007-12-02 09:09 PM Re: File listing [Re: Witto]
Arend_ Moderator Offline
MM club member
*****

Registered: 2005-01-17
Posts: 1896
Loc: Hilversum, The Netherlands
Huh, you can delete the post...I think, if you edit the post you can click the button below called "Delete". I thought that deleted the entire post.
Top
#183170 - 2007-12-02 09:13 PM Re: File listing [Re: Arend_]
Witto Offline
MM club member
*****

Registered: 2004-09-29
Posts: 1828
Loc: Belgium
UBB Error
This post may no longer be deleted.

Please click back to return to the previous page.

Top
#183171 - 2007-12-02 09:14 PM Re: File listing [Re: Witto]
Arend_ Moderator Offline
MM club member
*****

Registered: 2005-01-17
Posts: 1896
Loc: Hilversum, The Netherlands
Too late to delete them
Top
Page 1 of 2 12>


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 400 anonymous users online.
Newest Members
batdk82, StuTheCoder, M_Moore, BeeEm, min_seow
17885 Registered Users

Generated in 0.074 seconds in which 0.026 seconds were spent on a total of 14 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org