Page 1 of 1 1
Topic Options
#128985 - 2004-11-04 03:22 AM Detokenize as an option in new version
ciper Offline
Fresh Scripter

Registered: 2004-11-03
Posts: 36
When using kix32 with the /t argument to create a tokenized script an additional argument should be available to enable/disable extraction of the original script.

In other words, make it possible to create a file that cannot be extracted if thats needed (to hide contents) or a file that can be extracted at a later time (for faster/smaller scripts without a concern for hiding content)


Top
#128986 - 2004-11-04 10:06 AM Re: Detokenize as an option in new version
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
Tokenisation is a one-way process. More than simply replacing KiXtart keyword with tokens it compiles the code to KiXtart runtime byte-code. This will include any optimisations to the code structure that can be made at parse-time.

A stand-alone exe could reverse engineer the tokenised file and produce a working script, but it would not be the original script.

I'm struggling to think of an application - what would you use a reverse engineering facility for?

Top
#128987 - 2004-11-04 11:46 AM Re: Detokenize as an option in new version
Jochen Administrator Offline
KiX Supporter
*****

Registered: 2000-03-17
Posts: 6380
Loc: Stuttgart, Germany
Most probably the need to apply version changes after tokenization (to the tokenized script) ... because one accidentally deleted the source scripts... been there once, but luckily found one copy on one of my other boxes
_________________________



Top
#128988 - 2004-11-04 12:22 PM Re: Detokenize as an option in new version
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
That's not a good argument for reverse engineering. It's a good argument for proper source control or data archival.
Top
#128989 - 2004-11-04 02:11 PM Re: Detokenize as an option in new version
Jochen Administrator Offline
KiX Supporter
*****

Registered: 2000-03-17
Posts: 6380
Loc: Stuttgart, Germany
Sure ... didn't meant that as an argument pro reverse engineering. Just tried to think out loud on possible reasons.
IMO this option should never make it to the release. Sorry if you misunderstood
_________________________



Top
#128990 - 2004-11-04 02:12 PM Re: Detokenize as an option in new version
Shawn Administrator Offline
Administrator
*****

Registered: 1999-08-13
Posts: 8611
Richard/Cappy ... you guys got any comments to add on this thread, whats your views on "the size of kix32.exe" ?

http://www.kixtart.org/ubbthreads/showflat.php?Cat=&Board=UBB4&Number=128013


Top
#128991 - 2004-11-04 02:17 PM Re: Detokenize as an option in new version
Jochen Administrator Offline
KiX Supporter
*****

Registered: 2000-03-17
Posts: 6380
Loc: Stuttgart, Germany
Size in real world doesn't matter at all, it's all about whatcha do with it.
Go back to school if you disagree
_________________________



Top
#128992 - 2004-11-04 02:36 PM Re: Detokenize as an option in new version
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
Quote:

Sure ... didn't meant that as an argument pro reverse engineering. Just tried to think out loud on possible reasons.
IMO this option should never make it to the release. Sorry if you misunderstood




Not at all. I understand entirely. It was the only thing I could think of as well and (as we agree) it's a misuse.

I was just waiting for someone to suggest it so I could jump on it

Top
#128993 - 2004-11-04 10:25 PM Re: Detokenize as an option in new version
ciper Offline
Fresh Scripter

Registered: 2004-11-03
Posts: 36
It would help when you need to sit at the machine not working properly and you want to debug the script. If I want to run the script manually and step through the commands using the debugger how will this work with a tokenized file?

Sounds like many of you suggest using an archived copy (which should be the same as on the system but you cant be entirely sure) and running it on the machine in detokenized format. That means either the script must be modified to run from the users system (instead of the shared location it was originally on) or I have to store the detokenized version on the central location and point the machine to it. I wont have access to the same resources at a users desk as I would at my workstation so I have to go back to retrieve this archived copy.

What if the problem with the machine was that he didnt get the latest version of the script. Preventing me from detokenizing and comparing agaisnt the standard copy makes troubleshooting a little harder. I could do a CRC check against the two to verify this but this again takes another application I may not have access to at a users desk.

All of this adds up to more time troubleshooting. If I have access to the tokenized file and a kix executable I should be able to extract then debug the script at the location.

I do have a use for a script that is unreadable. Engineers love to circumvent our standard software installations, virus scanners for example. They will reverse engineer the method to detect virus scan installation and create dummy positives. If I could hide this script from them it would be very hard. This data would be stored in a second script with the "do not allow detokenize" enabled.

Top
#128994 - 2004-11-06 12:25 AM Re: Detokenize as an option in new version
Sealeopard Offline
KiX Master
*****

Registered: 2001-04-25
Posts: 11164
Loc: Boston, MA, USA
It's all about trade-offs. If you expect the need for troubleshooting your scripts then using the tokenizer might not be a good option. As Newton already discovered: "For every convenience there's also an inconvenience".
_________________________
There are two types of vessels, submarines and targets.

Top
#128995 - 2004-11-08 10:11 AM Re: Detokenize as an option in new version
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
Quote:

It would help when you need to sit at the machine not working properly and you want to debug the script. If I want to run the script manually and step through the commands using the debugger how will this work with a tokenized file?




Not very well

Quote:

Sounds like many of you suggest using an archived copy (which should be the same as on the system but you cant be entirely sure) and running it on the machine in detokenized format. That means either the script must be modified to run from the users system (instead of the shared location it was originally on) or I have to store the detokenized version on the central location and point the machine to it. I wont have access to the same resources at a users desk as I would at my workstation so I have to go back to retrieve this archived copy.




I don't understand this at all. You say you are going to run the original script manually - what does it matter where you get it from? You surely aren't suggesting that you want to leave a "detokenised" copy unprotected on the netlogon share? There is of course no reason not to have a restricted folder on the netlogon share which contains the original script so it is easy for you to retrieve. You will update the two in parallel when you release a new version.

Quote:

What if the problem with the machine was that he didnt get the latest version of the script. Preventing me from detokenizing and comparing agaisnt the standard copy makes troubleshooting a little harder.




Not at all. Whenever I make an amendment to my login script I update the version number and the last changed date. The script has an internal record of all changes. When a user logs on a hidden log file is written to their home directory containing the version number of all scripts, where they log in from, which logon server they are authenticating against, all the process stages that occur and and errors. An error causes the log to be mailed to me.

Identifying the script that has been run is therefore trivial.

If you don't want to go to this much effort, add a handler which on seeing the $VERSION flag on the command line just dumps the version info and exits.

Quote:

I could do a CRC check against the two to verify this but this again takes another application I may not have access to at a users desk.




Simple command line utilities like this do not require installation and can be executed from any available share.

Quote:

All of this adds up to more time troubleshooting. If I have access to the tokenized file and a kix executable I should be able to extract then debug the script at the location.



You have access to the original file. Running it at the user location is no more time consuming or complex than de-tokenising.

You certainly do not want the KiXtart executable to contain de-tokenising code, as that would uneccessarily bloat it so the de-tokeniser would be a stand-alone exe.

Quote:

I do have a use for a script that is unreadable. Engineers love to circumvent our standard software installations, virus scanners for example. They will reverse engineer the method to detect virus scan installation and create dummy positives. If I could hide this script from them it would be very hard. This data would be stored in a second script with the "do not allow detokenize" enabled.




This sort of situation must be covered by company policy. Any attempt to subvert security procedures is at the very least gross misconduct. Hiding scripts is not the right way to combat such blatant stupidity. If it were me I'd have an email off to company security, the engineers line manager and human resources as soon as I discovered this sort of nonsense, and leave them to sort it out.

The point of this long ramble is that detokenising does not add anything useful to these scenarios. As the detokenised code will not match the original code anyway it is likely to be less useful in your debugging tasks than the alternatives.

There are better tools and methods to achieve what you need.

While detokenising might at first seem to fit the bill it is the lure of the Dark Side - seemingly simple and easier at first you will find that it will only bring trouble, pain and undesirable results if you stray down that path.

Top
#199830 - 2010-09-13 07:53 PM Re: Detokenize as an option in new version [Re: Richard H.]
tremainv Offline
Just in Town

Registered: 2010-09-13
Posts: 1
Loc: Michigan, USA
This topic has been sitting out here for a while, but I have a situation that has arisen today where "detokenizing" would be of great value. I have just been given responsibility over a app we are running in Citrix and we use a tokenized Kix script. I have no idea of what all is included in the script and there is no source code for the tokenized script. There are no backups anywhere. The 2 people who new about the script and what it held have left the company in the past 2 weeks. I now need to try and find out what this script is doing. If detokenizing is still not an option, I would welcome suggestions on how to proceed in piecing this puzzle together.
Top
#199831 - 2010-09-13 08:35 PM Re: Detokenize as an option in new version [Re: tremainv]
Allen Administrator Online   shocked
KiX Supporter
*****

Registered: 2003-04-19
Posts: 4549
Loc: USA
Since you have no backups, a Phone call to the ex-employees and hope they are willing to help out... is about your only option.

The purpose of the tokenization is keep prying eyes out of the script... it would provide no "security" if everyone could untokenize it. Unfortunately, when situations like yours arises, it leaves little direction other than to completely trust what it is doing, remove it, or re-write it.

Top
#199833 - 2010-09-13 08:56 PM Re: Detokenize as an option in new version [Re: Allen]
Lonkero Administrator Offline
KiX Master Guru
*****

Registered: 2001-06-05
Posts: 22346
Loc: OK
there are also tools you can use that track changes to the computer.

sadly in this case the only secure option is to stop using the script.
_________________________
!

download KiXnet

Top
#199853 - 2010-09-14 02:31 PM Re: Detokenize as an option in new version [Re: tremainv]
Richard H. Administrator Offline
Administrator
*****

Registered: 2000-01-24
Posts: 4946
Loc: Leatherhead, Surrey, UK
There *was* source as the tokenisation process doesn't delete it, so it should be secured somewhere, not least because the coder will need access to it to engineer fixes and updates.

If it isn't available then the person responsible is utterly incompetent at best (good job they're gone) and criminally negligent at worst.

You can use monitoring tools as Jooel suggested to detect activity like registry writes and command execution, but this won't explain branching in the code based on groups or AD information or installed software.

Probably you are just going to have to remove the script, see what breaks and then re-engineer it.

Just make sure that you document the source location for the next poor devil that has to deal with it, as it might be you.

Top
#201938 - 2011-04-26 02:48 PM Re: Detokenize as an option in new version [Re: Richard H.]
Ruud van Velsen Moderator Offline
Developer
*****

Registered: 1999-05-06
Posts: 391
Loc: Amsterdam, The Netherlands
Just to say that if there is some dire need to detokenize a particular script, feel free to contact me directly.

Note that there is no way to detokenize password protected scripts if you have forgotten the password... Well, hypnotherapy, maybe.

Ruud

Top
Page 1 of 1 1


Moderator:  Lonkero, ShaneEP, Jochen, Radimus, Glenn Barnas, Allen, Ruud van Velsen, Mart 
Hop to:
Shout Box

Who's Online
1 registered (Allen) and 496 anonymous users online.
Newest Members
Raoul, Timothy, Jojo67, MaikSimon, kvn317
17875 Registered Users

Generated in 0.107 seconds in which 0.077 seconds were spent on a total of 13 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org