#66359 - 2002-06-08 05:57 AM
Unicode Converter UDF
|
Howard Bullock
KiX Supporter
Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
|
I was attempting to write a UDF that would convert a (unicode file to an ascii file format) and an (ascii file format to a unicode file).
I have been successful in going from unicode to Ascii, but seem to have hit a limitation with KiXtart going from Ascii to unicode. KiXtart doesn't seem to handle Chr(0) at all when adding it to a string. Does anyone have any insight?
I know this isn't a true unicode converter but it works for single byte languages.
code:
UnicodeConvert("c:\data\scripts\unicode.sql")
Function UnicodeConvert($File) Dim $FH, $x, $null, $rc
$FH = OpenFile($File, "R") $x = ReadLine($FH) if @error = 0 if instr($x, Chr(255)+Chr(254)) $Data = Right($x, 1) While (@error=0) $x = ReadLine(1) if @error = 0 if Asc($x)<>0 $Data = $Data + $x else $null = $null + 1 if $null = 4 $Data = $Data + @CRLF $null = 0 endif endif endif Loop else $Data = Chr(255)+Chr(254) Gosub "ProcessLine" $x = ReadLine($FH) While (@error=0) Gosub "ProcessLine" $x = ReadLine(1) Loop endif else exit 1 endif
$rc = close($FH) Del $File $FH = OpenFile($File, "W", 1, 1) $rc = WriteLine($FH, $Data) $rc = close($FH) ? For $i=1 to len($Data) Asc(substr($Data,$i,1)) " " Next
exit 0
:ProcessLine For Each $x IN Split($x,"") $Data = $Data + $x + Chr(0) Next $Data = $Data + Chr(13) + Chr(0) + Chr(10) + Chr(0)
return
Endfunction
;FUNCTION OpenFile() ; ;AUTHOR Howard A. Bullock (hbullock@tycoelectronics.com) ; ;ACTION Opens file based on specified parameters. Supports overwriting a file. ; ;SYNTAX OpenFile(FileName, R|W, [0|1], [0|1]) ; ;PARAMETERS $FileName (Required) - String value ; $ReadWrite (Required) - String value ; $Create (Optional) Default(0) Do not create a new file. (1 or 0) ; $OverWrite (Optional) - Default(0) Append (1 or 0) ; ; ;REMARKS This function attempts to open the specified file. @error is set on exit. ; The UDF will search for the first available file number. ; ;RETURNS File handle (integer 1-10) if successful; nothing if error ; ;DEPENDENCIES None ; ;EXAMPLES ; Open file for write; do not create the file; append ; $FH = OpenFile("junk.txt", "W") ; if @error = 0 ; Writeline($FH, "Some text") ; endif ; ; Open file for write; create the file; delete old file ; $FH2 = OpenFile("junk2.txt", "W", 1, 1) ; if @error = 0 ; Writeline($FH2, "Some text") ; endif; ; Function OpenFile($FileName, $ReadWrite, optional $Create, optional $Overwrite) Dim $FileName, $ReadWrite, $Create, $Overwrite Dim $FH, $RC, $Mode
; Validate input parameters if VarType($Create) = 0 $Create = 0 endif if VarType($Overwrite) = 0 $Overwrite = 0 endif if not (VarType($FileName) = 8 and ($ReadWrite = "R" or $ReadWrite = "W") and ($Create = 0 or $Create = 1) and ($Overwrite = 0 or $Overwrite = 1) ) exit 1 endif if $Create < $OverWrite exit 1 endif
; Build Mode value if $Create = 0 $Mode = 0 else $Mode = 1 endif if $ReadWrite = "R" $Mode = $Mode + 2 else $Mode = $Mode + 4 endif
;Handle OverWrite option if $OverWrite = 1 if exist ($Filename) del $Filename endif endif
; Open file $FH=1 $RC=Open ($FH, $FileName, $Mode) ? "@error @serror" if $RC <> 0 select case $RC=2 exit $RC case $RC=-3 while $RC = -3 and $FH < 11 $RC=Open ($FH, $FileName, $Mode) if $RC = -3 $FH=$FH +1 endif Loop if $RC = -3 exit $RC endif case $RC=-2 exit $RC case $RC=-1 exit $RC case 1 exit $RC endselect endif
$OpenFile = $FH exit 0 Endfunction
[ 08 June 2002, 15:24: Message edited by: Howard Bullock ]
|
Top
|
|
|
|
#66361 - 2002-06-08 03:18 PM
Re: Unicode Converter UDF
|
Howard Bullock
KiX Supporter
Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
|
Yeah I did some other testing this morning and found that chr(0) doesn't seem to work at all. Bug in my book.
Sorry for the long lines. I was tired and forgot to change TABS to spaces.
{edit} OK officier, I have paid my fine and performed my community service by removing the long lines. [ 08 June 2002, 15:25: Message edited by: Howard Bullock ]
|
Top
|
|
|
|
#66362 - 2002-06-08 03:47 PM
Re: Unicode Converter UDF
|
Anonymous
Anonymous
Unregistered
|
The Ascii zero is in the manual under 'General Syntax Rules', I quote: Strings can contain any characters, except the \0 (NULL) and \x1a (end of file) characters.
With FSO you can open a file as unicode for read or write.
I knocked up the following to read and write to/from an array
code:
$files = @ScriptDir+"\output.uni",@ScriptDir+"\output.txt" $txt = "hello there","how are you","today?","I am " WriteFile($files[0],$txt,"u") AppendFile($files[0],"Unicode"+@CRLF+"so there") WriteFile($files[1],$txt) AppendFile($files[1],"Ascii"+@CRLF+"ending with EOL"+@CRLF) ; FOR EACH $fn IN $files $arr = ReadFile($fn) ? $fn +', Lines = '+Ubound($arr) FOR $ajh = 0 to Ubound($arr) ? ""+$ajh+" : "+$arr[$ajh] NEXT ? NEXT GETS $rc EXIT ; ;======================================== ; Creates a text file from $text (can be an array) ; If $type = "u" then a Unicode text file is created. ; New-line is NOT forced after text ; (to force New-Line $text[n] should be an empty element ; or if $text is a string it should end with @CRLF) ; FUNCTION WriteFile($fn,$text,OPTIONAL $type) IF Ubound($text) < 1 $text =Split($text,@CRLF) ENDIF IF $type = "U" $type = -1 ELSE $type = 0 ENDIF DIM $oFSO,$oFile,$i $oFSO = CreateObject("Scripting.FileSystemObject") IF @Error EXIT @Error ENDIF $oFile = $oFSO.OpenTextFile($fn,2,-1,$type) IF @Error EXIT @Error ENDIF FOR $i = 0 TO UBound($text) -1 $oFile.WriteLine($text[$i]) NEXT $oFile.Write($text[UBound($text)]) $oFile.Close ENDFUNCTION ; ;======================================== ; Appends $text (can be an array) onto an existing text file ; New-line is NOT forced before or after text ; (to force New-Line $text[n] should be an empty element ; or if $text is a string it should begin/end with @CRLF) ; FUNCTION AppendFile($fn,$text) IF Ubound($text) < 1 $text =Split($text,@CRLF) ENDIF DIM $oFSO,$oFile,$type $oFSO = CreateObject("Scripting.FileSystemObject") IF @Error EXIT @Error ENDIF $oFile = $oFSO.OpenTextFile($fn,1,0,0) ;Ascii IF @Error EXIT @Error ENDIF $type = $oFile.Read(2) $oFile.Close IF $type = CHR(255)+CHR(254) $type = -1 ELSE $type = 0 ENDIF $oFile = $oFSO.OpenTextFile($fn,8,0,$type) FOR $i = 0 TO UBound($text) -1 $oFile.WriteLine($text[$i]) NEXT $oFile.Write($text[UBound($text)]) $oFile.Close ENDFUNCTION ; ;======================================== ; Reads contents of file into an array. ; Ascii / Unicode is detected automatically. ; FUNCTION ReadFile($fn) DIM $oFSO,$oFile,$type $oFSO = CreateObject("Scripting.FileSystemObject") IF @Error EXIT @Error ENDIF $oFile = $oFSO.OpenTextFile($fn,1,0,0) ;Ascii IF @Error EXIT @Error ENDIF $type = $oFile.Read(2) $oFile.Close IF $type = CHR(255)+CHR(254) $type = -1 ELSE $type = 0 ENDIF $oFile = $oFSO.OpenTextFile($fn,1,0,$type) $ReadFile = Split($oFile.ReadAll,@CRLF) $oFile.Close ? "Type "+$type ENDFUNCTION
|
Top
|
|
|
|
#66363 - 2002-06-08 03:56 PM
Re: Unicode Converter UDF
|
Howard Bullock
KiX Supporter
Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
|
Thanks for setting us straight. That's what happens when you're tired and assume things should work the way you want them to.
Should have read the doc again... I hope that Ruud permits Nulls in strings because we have to sometimes use null delimited string in registry values.
Wait a minute... The docs say null (\0) can be in a string yet when I read the unicode file the null chartacters around chr(13) and chr(10) were included in the string and processed properly. That is how I determined the end of line. Why do some work? [ 08 June 2002, 15:57: Message edited by: Howard Bullock ]
|
Top
|
|
|
|
#66365 - 2002-06-10 03:53 PM
Re: Unicode Converter UDF
|
BrianTX
Korg Regular
Registered: 2002-04-01
Posts: 895
|
There is a utility called uconvert.exe included with Microsoft's Platform SDK.
It converts any code page to unicode...
As a side note.. would it be possible to call an object/method in internet explorer to convert to unicode? just a thought..
Brian
|
Top
|
|
|
|
#66366 - 2002-06-10 04:13 PM
Re: Unicode Converter UDF
|
Howard Bullock
KiX Supporter
Registered: 2000-09-15
Posts: 5809
Loc: Harrisburg, PA USA
|
Yeah, I saw that. This was just a KiXtart exercise.
I just wish (second one today) that all ASCII characters could be placed into strings like other languages. [ 14 July 2002, 16:04: Message edited by: Howard Bullock ]
|
Top
|
|
|
|
Moderator: Glenn Barnas, NTDOC, Arend_, Jochen, Radimus, Allen, ShaneEP, Ruud van Velsen, Mart
|
0 registered
and 447 anonymous users online.
|
|
|