AllenAdministrator
(KiX Supporter)
2016-05-09 04:13 PM
vbscript MD5 conversion issues

I found a vbscript that creates a MD5 hash but have not had luck converting it to kixtart.

Here is the original vbscript code that does work:
{edit: had to change code tag to quotes on the vbscript code... board didnt like it}

 Quote:


'MD5=151D5E94EE1197E3A3AD8E8089A73884
spath="C:\Downloads\Firefox Setup 46.0.1.exe"
wscript.echo bytesToHex(md5hashBytes(GetBytes(sPath)))

function md5hashBytes(aBytes)
Dim MD5
set MD5 = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")

MD5.Initialize()
'Note you MUST use computehash_2 to get the correct version of this method, and the bytes MUST be double wrapped in brackets to ensure they get passed in correctly.
md5hashBytes = MD5.ComputeHash_2( (aBytes) )
end function

function stringToUTFBytes(aString)
Dim UTF8
Set UTF8 = CreateObject("System.Text.UTF8Encoding")
stringToUTFBytes = UTF8.GetBytes_4(aString)
end function

function bytesToHex(aBytes)
dim hexStr, x
for x=1 to lenb(aBytes)
hexStr= hex(ascb(midb( (aBytes),x,1)))
if len(hexStr)=1 then hexStr="0" & hexStr
bytesToHex=bytesToHex & hexStr
next
end function

Function BytesToBase64(varBytes)
With CreateObject("MSXML2.DomDocument").CreateElement("b64")
.dataType = "bin.base64"
.nodeTypedValue = varBytes
BytesToBase64 = .Text
End With
End Function

Function GetBytes(sPath)
With CreateObject("Adodb.Stream")
.Type = 1 ' adTypeBinary
.Open
.LoadFromFile sPath
.Position = 0
GetBytes = .Read
.Close
End With
End Function


My goal was to create a all in one function to spit out the MD5 hash. I am getting results but they are not correct. What's interesting is, the first few digits are correct although there are 0's in between the values. Once the first few digits are reached, nothing else match the rest of the hash. There were a few vbscript specific functions that we do not have in kix that I recreated using the scriptcontrol. I was using dectohex but switched to the hex function in hopes it mattered, it didnt.

 Code:
  dim $sPath, $ado, $bytes, $MD5, $MD5hashbytes, $x, $hexstring, $bytestohex
  $Spath="C:\Downloads\Firefox Setup 46.0.1.exe"
  $ADO=createobject("Adodb.Stream")
  $ADO.Type=1 ; adTypeBindar
  $ADO.Open
  $ADO.LoadfromFile($Spath)
  $ADO.position=0
  $Bytes=$ADO.Read
  $ADO.close
  $MD5 = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")
  $MD5.Initialize()
  ;Note you MUST use computehash_2 to get the correct version of this method, and the bytes MUST be double wrapped in brackets to ensure they get passed in correctly.
  $md5hashBytes = $MD5.ComputeHash_2( ($Bytes) )
  for $x=1 to lenb($md5hashBytes)
    $hexStr= right("0" + hex(ascb(midb( ($md5hashBytes),$x,1))),2)
	? $hexstr
    $bytesToHex=$bytesToHex + $hexStr
  next
  
  ? $BytestoHex
  
Function ASCB($string)
  Dim $sc
  $sc = CreateObject("ScriptControl")
  $sc.Language = "VBScript"
  $ASCB = $sc.Eval('ASCB("'+ $String + '")')
EndFunction  

Function MidB($string, $start, $length)
  Dim $sc
  $sc = CreateObject("ScriptControl")
  $sc.Language = "VBScript"
  $MidB = $sc.Eval('MIDB("'+ $string + '",' + $Start + ',' + $length +  ')')
EndFunction  

Function LenB($string)
  Dim $sc
  $sc = CreateObject("ScriptControl")
  $sc.Language = "VBScript"
  $LenB = $sc.Eval('LenB("'+ $String + '")')
EndFunction

Function Hex($Number)
  Dim $sc
  $sc = CreateObject("ScriptControl")
  $sc.Language = "VBScript"
  $Hex = $sc.Eval('Hex('+ $Number + ')')
EndFunction




Any ideas/suggestions?


Glenn BarnasAdministrator
(KiX Supporter)
2016-05-09 05:12 PM
Re: vbscript MD5 conversion issues

I checked my lib - I just used md5.exe and called it a day! ;\)

Actually, I did recently update my internal code to call CertUtil, which is built-in on all current Windows platforms (at least XP). It can generate MD2 MD4 MD5 SHA1 SHA256 SHA384 & SHA512 Hash Algorithms.
 Code:
certUtil -hashfile pathToFileToCheck [HashAlgorithm]
I haven't released this update to prod, so my web site still references the md5.exe version, which required the externally obtained command. The code update simply changes CertUtil.exe for MD5.exe, so it's easy to hack if you want to go in that direction.

Glenn


AllenAdministrator
(KiX Supporter)
2016-05-09 06:20 PM
Re: vbscript MD5 conversion issues

CertUtil has this capability... I didn't know that. \:\)

If all else fails at least I have that as option. I would love to see this working though.


Glenn BarnasAdministrator
(KiX Supporter)
2016-05-09 07:47 PM
Re: vbscript MD5 conversion issues

I hacked around with this about a month ago without much success and finally gave up when I found that CertUtil did this and was on every system I needed to run it on. Wasn't worth my time/effort to pursue at that point.

Glenn


LonkeroAdministrator
(KiX Master Guru)
2016-05-09 09:17 PM
Re: vbscript MD5 conversion issues

as far as I can remember this totally correct:
kixtart does not support binary content types.
that's why inside kixforms.net all bytes are translated to a string-array.

but, even if it did, you force kixtart to translate the bytes to a string on this line:
 Code:
$MidB = $sc.Eval('MIDB("'+ $string + 


so if the byte-array contains a single zero or special character, you end up messing it up.


AllenAdministrator
(KiX Supporter)
2016-05-09 09:42 PM
Re: vbscript MD5 conversion issues

I might try to do all of this within scriptcontrol. I don't know why the exe is so unappealing to me

AllenAdministrator
(KiX Supporter)
2016-05-09 09:43 PM
Re: vbscript MD5 conversion issues

BTW, thanks for looking.

LonkeroAdministrator
(KiX Master Guru)
2016-05-09 10:07 PM
Re: vbscript MD5 conversion issues

oh, another BTW. you can use kixforms to overcome this issue.

AllenAdministrator
(KiX Supporter)
2016-05-09 10:23 PM
Re: vbscript MD5 conversion issues

Yeah... but I don't have kixforms installed on any of the pcs. I've just never invested the time in kixforms, I'm mostly a complete noob with it.

NTDOCAdministrator
(KiX Master)
2016-05-10 03:51 AM
Re: vbscript MD5 conversion issues

KiXforms is/was the thing of the future for us until Shawn got called away with other more pressing things in life :-(

I've made a few things and worked with Glenn and a few others over the years to make some things in KiXforms but have not used it much myself now in many years.


LonkeroAdministrator
(KiX Master Guru)
2016-05-10 03:55 AM
Re: vbscript MD5 conversion issues

I did re-target kf.net to newer .net version, but the development has completely stalled. Lack of users might be a good excuse.

Arend_
(MM club member)
2016-05-10 10:18 AM
Re: vbscript MD5 conversion issues

Seems interesting, I'm going to try it.
So far I get almost the correct value.
Above is the real, expected value, below is the output of my script.
 Code:
dfdc6c74 96	d9645953 89   ed48e229ee81 *KIX32.EXE
DFDC6C74 1320	D9645953 3020 ED48E229EE81


Arend_
(MM club member)
2016-05-10 11:01 AM
Re: vbscript MD5 conversion issues

Please be amazed ;-)
 Code:
$=SetOption('Explicit','On')
Function md5($filename)
  Dim $objXML, $objXEL, $objMD5, $objStream, $BinaryFile
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = 1
  $objStream.Open
  $objStream.LoadFromFile($filename)
  $BinaryFile = $objStream.Read
  $objStream.Close
  $objStream = ""
  $objMD5 = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")
  $=$objMD5.ComputeHash_2($BinaryFile)
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objMD5.Hash
  $md5 = $objXEL.Text
EndFunction

? md5("C:\KIX32.EXE")


Arend_
(MM club member)
2016-05-10 11:34 AM
Re: vbscript MD5 conversion issues

Only seems to work with files up to 250 MB. \:\(
Probably because of some variable size limit in Kix.


LonkeroAdministrator
(KiX Master Guru)
2016-05-10 02:24 PM
Re: vbscript MD5 conversion issues

Nice!!!!!!!!!!!

I might be mistaken, but your previous post seemed to me to have correct value as well


Arend_
(MM club member)
2016-05-10 02:30 PM
Re: vbscript MD5 conversion issues

For some reason 4 values were incorrectly identified in the string.
Hence why I dropped the whole loop converting the byte array.


AllenAdministrator
(KiX Supporter)
2016-05-10 03:26 PM
Re: vbscript MD5 conversion issues

Oh wow... \:\) I'll give it whirl.

Arend_
(MM club member)
2016-05-10 03:54 PM
Re: vbscript MD5 conversion issues

And just because it's possible:
 Code:
Function SHA1($strFileName)
  Dim $objXML, $objXEL, $objSHA1, $objStream, $BinaryFile
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = 1
  $objStream.Open
  $objStream.LoadFromFile($strFileName)
  $BinaryFile = $objStream.Read
  $objStream.Close
  $objStream = ""
  $objSHA1 = CreateObject("System.Security.Cryptography.SHA1CryptoServiceProvider")
  $=$objSHA1.ComputeHash_2($BinaryFile)
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objSHA1.Hash
  $SHA1 = $objXEL.Text
EndFunction

? SHA1("C:\KIX32.EXE")


AllenAdministrator
(KiX Supporter)
2016-05-10 06:31 PM
Re: vbscript MD5 conversion issues

 Quote:
Only seems to work with files up to 250 MB. \:\(
Probably because of some variable size limit in Kix.


It's failing here on the +200mb files:
$=$objSHA1.ComputeHash_2( $BinaryFile )
? @serror

Output is:
Not enough storage is available to complete this operation.

I read some articles on breaking the stream into smaller chunks, but it didn't really seem to apply to what we are doing.




LonkeroAdministrator
(KiX Master Guru)
2016-05-10 08:36 PM
Re: vbscript MD5 conversion issues

not so smart people of microsoft have destroyed all com documentation but with 99% certainty this is a limit of com/adodb.

Arend_
(MM club member)
2016-05-11 10:12 AM
Re: vbscript MD5 conversion issues

I've tried to get around the file size limit but to no avail.
Instead I've added SHA256, SHA384 en SHA512 functions :P
 Code:
Function SHA256($strFileName)
  Dim $objXML, $objXEL, $objSHA256, $objStream, $BinaryFile
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = 1
  $objStream.Open
  $objStream.LoadFromFile($strFileName)
  $BinaryFile = $objStream.Read
  $objStream.Close
  $objStream = ""
  $objSHA256 = CreateObject("System.Security.Cryptography.SHA256Managed")
  $=$objSHA256.ComputeHash_2($BinaryFile)
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objSHA256.Hash
  $SHA256 = $objXEL.Text
EndFunction

? "SHA256: "+SHA256("D:\Firefox Setup 46.0.1.exe")

 Code:
Function SHA384($strFileName)
  Dim $objXML, $objXEL, $objSHA384, $objStream, $BinaryFile
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = 1
  $objStream.Open
  $objStream.LoadFromFile($strFileName)
  $BinaryFile = $objStream.Read
  $objStream.Close
  $objStream = ""
  $objSHA384 = CreateObject("System.Security.Cryptography.SHA384Managed")
  $=$objSHA384.ComputeHash_2($BinaryFile)
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objSHA384.Hash
  $SHA384 = $objXEL.Text
EndFunction

? "SHA384: "+SHA384("D:\Firefox Setup 46.0.1.exe")

 Code:
Function SHA512($strFileName)
  Dim $objXML, $objXEL, $objSHA512, $objStream, $BinaryFile
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = 1
  $objStream.Open
  $objStream.LoadFromFile($strFileName)
  $BinaryFile = $objStream.Read
  $objStream.Close
  $objStream = ""
  $objSHA512 = CreateObject("System.Security.Cryptography.SHA512Managed")
  $=$objSHA512.ComputeHash_2($BinaryFile)
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objSHA512.Hash
  $SHA512 = $objXEL.Text
EndFunction

? "SHA512: "+SHA512("D:\Firefox Setup 46.0.1.exe")


Arend_
(MM club member)
2016-05-11 10:18 AM
Re: vbscript MD5 conversion issues

And I've just combined all in 1 function, enjoy \:\)
 Code:
Function Hash($strFileName, optional $strHashType)
  Dim $objXML, $objXEL, $objHash, $objStream, $BinaryFile
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = 1
  $objStream.Open
  $objStream.LoadFromFile($strFileName)
  $BinaryFile = $objStream.Read
  $objStream.Close
  $objStream = ""
  Select
    Case $strHashType = "MD5"
      $objHash = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")
    Case $strHashType = "SHA1"
      $objHash = CreateObject("System.Security.Cryptography.SHA1CryptoServiceProvider")
    Case $strHashType = "SHA256"
      $objHash = CreateObject("System.Security.Cryptography.SHA256Managed")
    Case $strHashType = "SHA384"
      $objHash = CreateObject("System.Security.Cryptography.SHA384Managed")
    Case $strHashType = "SHA512"
      $objHash = CreateObject("System.Security.Cryptography.SHA512Managed")
    Case 1
      $objHash = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")
  EndSelect
  $RC=$objHash.ComputeHash_2($BinaryFile)
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objHash.Hash
  $Hash = $objXEL.Text
EndFunction

? "Hash: "+Hash("D:\Firefox Setup 46.0.1.exe", "md5")


LonkeroAdministrator
(KiX Master Guru)
2016-05-11 04:01 PM
Re: vbscript MD5 conversion issues

nice. thanks arend.

Glenn BarnasAdministrator
(KiX Supporter)
2016-05-11 04:34 PM
Re: vbscript MD5 conversion issues

Nice job!!

If you're going to place this into the UDF forum, please consider a test for large files and exit with an error if a file size exceeds the allowed maximum size (and a null value returned). Also, a name other than "Hash" would be best as there is already a UDF with that name that mimics the Perl hash function, which is something else entirely. (maybe CryptoHash?)

Glenn


AllenAdministrator
(KiX Supporter)
2016-05-11 06:05 PM
Re: vbscript MD5 conversion issues

CheckSum() would also work as a the function name.

Arend_
(MM club member)
2016-05-12 10:52 AM
Re: vbscript MD5 conversion issues

Done.

Arend_
(MM club member)
2016-05-12 05:19 PM
Re: vbscript MD5 conversion issues

Well I have figured out why the size matters (Yes ladies, it does ;\) ).
Apparently it is a memory limit.
To fix this, you have to feed the bytes to the System.Security.Cryptography object in blocks using the .TransFormBlocks and eventually the .TransFormFinalBlocks property of the object before using the ComputeHash property.
The only problem I see is to get the stream reader to split it up in chucks of less then 200 MB.
If I get more time I'll try to sort it out.


Arend_
(MM club member)
2016-05-13 11:31 AM
Re: vbscript MD5 conversion issues

Well, even using TransFormBlock it still has the size problem.
Apparently the size problem lies within the System.Security.Cryptography.MD5CryptoServiceProvider.
 Code:
Function MD5($strFileName)
  Dim $adTypeBinary, $objStream, $objXML, $objXEL, $objMD5, $i, $intFinalBlock, $BinaryFile
  $adTypeBinary = 1
  $objStream = CreateObject("ADODB.Stream")
  $objStream.Type = $adTypeBinary
  $objStream.Open
  $objStream.LoadFromFile($strFileName)
  $objMD5 = CreateObject("System.Security.Cryptography.MD5CryptoServiceProvider")
  $objMD5.Initialize
  $BinaryFile = $objStream.Read
  For $i=0 to $objStream.Size Step 31744
    $=$objMD5.TransformBlock($BinaryFile, $i, 31744, $BinaryFile, $i)
    If ($objStream.Size -$i) < 31744
      $intFinalBlock = ($objStream.Size -$i)
      $=$objMD5.TransformFinalBlock($BinaryFile, $i, $intFinalBlock)
    EndIf
  Next
  $objXML = CreateObject("MSXML2.DOMDocument")
  $objXEL = $objXML.CreateElement("tmp")
  $objXEL.DataType = "bin.hex"
  $objXEL.NodeTypedValue = $objMD5.Hash
  $MD5 = $objXEL.Text
  $objStream.Close
  $objStream = ""
EndFunction


The last option is to read the Stream in chunks and feed the chunks to TransformBlock.


Arend_
(MM club member)
2016-05-13 12:52 PM
Re: vbscript MD5 conversion issues

I was right, I changed the code and updated the UDF, CheckSum now works for all sizes.
Have a nice day!


AllenAdministrator
(KiX Supporter)
2016-05-13 03:52 PM
Re: vbscript MD5 conversion issues

Nice work!

ShaneEP
(MM club member)
2016-05-13 03:53 PM
Re: vbscript MD5 conversion issues

Well done sir.

Arend_
(MM club member)
2016-05-13 04:10 PM
Re: vbscript MD5 conversion issues

Thanks \:\)
Couldn't be done without Allen's initial work, thanks Allen!


NTDOCAdministrator
(KiX Master)
2016-05-13 10:53 PM
Re: vbscript MD5 conversion issues

Very nice :thumbs up: