Page 1 of 1 1
Topic Options
#212287 - 2017-02-11 03:14 PM Compare files and remove common lines
Sweeny Offline
Fresh Scripter

Registered: 2016-03-18
Posts: 18
Loc: Hampshire
To the great and wise people of the Kixtart forum,
I am struggling with a concept/approach to a scripting problem. Can someone give me a direction on how best to tackle this problem? I would like the script to compare 2 files and given the contents of one delete lines that match out of the other. So if one line exists in one its deleted out of the other. Or if easier compare 2 files, show the difference then remove the lines that match out of one of the files.
Many thanks for any information,
Tom

Top
#212288 - 2017-02-11 05:00 PM Re: Compare files and remove common lines [Re: Sweeny]
Allen Administrator Online   shocked
KiX Supporter
*****

Registered: 2003-04-19
Posts: 4562
Loc: USA
Untested and off the cuff to get you going. You will need readfile() and possibly writefile()

Readfile() -
http://www.kixtart.org/forums/ubbthreads.php?ubb=showflat&Number=84660

Writefile() -
http://www.kixtart.org/forums/ubbthreads.php?ubb=showflat&Number=84659

How to use UDFs -
http://www.kixtart.org/forums/ubbthreads.php?ubb=showflat&Number=81943#Post81943

The rest of the UDFs are here -
http://www.kixtart.org/forums/ubbthreads.php?ubb=postlist&Board=7&page=1


 Code:
;read files and put into arrays
$file1=Readfile(File1)
$file2=Readfile(File2)

;loop through files removing matches
for $i=0 to ubound($file2)
  for each $line in $file1
    $index=ascan($file2,$line)
    if $index>-1
      $file2[$index]=""
    endif  
  next
next

;build new arrary based on file2
for each $line in $file2
  if $line<>""
    redim preserve $output[$j]
    $output[$j]=$line
    $j=$j+1
  endif  
next

;use $output array to display your records, or create/overwrite a file

Top
#212290 - 2017-02-11 07:49 PM Re: Compare files and remove common lines [Re: Allen]
Glenn Barnas Administrator Offline
KiX Supporter
*****

Registered: 2003-01-28
Posts: 4401
Loc: New Jersey
Using the FileIO() UDF will make this a bit easier. FileIO can read a file into an array, then write that array to a file (same or different). The advantage, besides having a single UDF, is that when FileIO is used to write, you can suppress writing blank lines with a simple parameter. Thus, simply enumerate the first (master) file. Each line should then be searched for in the other file (array). If there's a match, simply set the element in the second array to an empty value. The example below illustrates this using a flat delimited dataset for the master data.

 Code:
; variables should be declared!
$aFData1 = FileIO($Source1, 'R') ; read the first (Master) file
$aFData2 = FileIO($Source2, 'R') ; read the second file

; Convert the master file to a delimited string to simplify matching the second file array
; The source will look like "~line1~line2~line3~...~"
; The delimiter character chosen MUST NOT appear in the source data. Use "Chr(3)"
; as a non-text delimiter if this is an issue (replace "'~'" with "Chr(3)" in this example)
$Source = '~' + Join($aFData1, '~')

; Enumerate the file to "clean up" - lines that exist in file 1 will be removed
For $P = 0 to UBound($aFData2)
  ; If the current line "~line#~" is found in $Source, clear it!
  If InStr($Source, '~' + $aFData2[$P] + '~')
    $aFData2[$P] = ''
  EndIf
Next

; All dups have been removed from the second file - write it back, trimming blank lines
$aFData2 = FileIO($Source2, 'W', 2)
FileIO() has some useful options, such as suppression of blank lines, suppression of final line terminator, and ability to output LF-delimited files for use on *nix systems.

Glenn

PS - the latest version of FileIO() can be downloaded from my http://www.innotechcg.com web site. Grab it now as I'm shutting this domain down and migrating the Kix library to a personal domain in the next few days.

_________________________
Actually I am a Rocket Scientist! \:D

Top
#212291 - 2017-02-12 02:43 PM Re: Compare files and remove common lines [Re: Glenn Barnas]
Sweeny Offline
Fresh Scripter

Registered: 2016-03-18
Posts: 18
Loc: Hampshire
Thank you both!!! I will test in the coming days and adapt to fit purposes, any issues I'll be back ;\) I can't thank enough \:\)
Cheers,
Tom

Top
Page 1 of 1 1


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 533 anonymous users online.
Newest Members
M_Moore, BeeEm, min_seow, Audio, Hoschi
17883 Registered Users

Generated in 0.035 seconds in which 0.015 seconds were spent on a total of 13 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org