Page 1 of 1 1
Topic Options
#204412 - 2012-03-06 09:11 PM Pre-Tokenizing Performance Benefit
DanD9272 Offline
Just in Town

Registered: 2012-03-02
Posts: 2
Loc: Portland OR
Hello,

Does anyone know how pre-tokenizing will increase performance or can elaborate on how it increases performance? I see that it can reduce the size the of the kix script which I could see increasing performance if you were reading a smaller versus larger file. But are there performance benefits from pre-tokenizing that will increase the speed at which kix processes a given script?

We use kix in our environment for logon scripts which are 95% "if ingroup" commands used to map drives based on group membership. So the file itself is not particularly large to begin with, so I'm wondering if there is any benefit from pre-tokenizing if the tokenized file and the non-tokenized file are both 4kb or less...

Thanks!

Dan

Top
#204414 - 2012-03-07 10:05 AM Re: Pre-Tokenizing Performance Benefit [Re: DanD9272]
Mart Moderator Offline
KiX Supporter
*****

Registered: 2002-03-27
Posts: 4673
Loc: The Netherlands
Hi and welcome to the board.

Tokenizing a script adds an extra step to the execution process because the scrip needs to be de-tokenized before it can be executed. I never used a tokenized script but my guess is that it would not increase the speed because of the de-tokenizing step. Tokenizing a script is actually meant to protect the source code of the script and possible sensitive data that is placed in the script like passwords.

In my opinion passwords and other sensitive data does not belong in a script that is available to the end user but that is just my opinion.
_________________________
Mart

- Chuck Norris once sold ebay to ebay on ebay.

Top
#204415 - 2012-03-07 01:43 PM Re: Pre-Tokenizing Performance Benefit [Re: Mart]
Les Offline
KiX Master
*****

Registered: 2001-06-11
Posts: 12734
Loc: fortfrances.on.ca
Umm... Mart, are you sure about de-tokenizing? The way it was told to me, a clear text script is tokenized before it is executed. If it is already pretokeinized, it will save having to do when launching it. The savings don't add up to much if anything at all. Most of a script's preformance is tied to how it makes system calls, not how it parses the code.
_________________________
Give a man a fish and he will be back for more. Slap him with a fish and he will go away forever.

Top
#204416 - 2012-03-07 02:43 PM Re: Pre-Tokenizing Performance Benefit [Re: Mart]
Glenn Barnas Administrator Offline
KiX Supporter
*****

Registered: 2003-01-28
Posts: 4403
Loc: New Jersey
Gotta go with Les on this one, as that's my understanding as well. There were actually some discussions about de-tokenizing to recover the source and how even the script engine didn't do that, so it was near impossible.

Glenn
_________________________
Actually I am a Rocket Scientist! \:D

Top
#204417 - 2012-03-07 02:56 PM Re: Pre-Tokenizing Performance Benefit [Re: Les]
Mart Moderator Offline
KiX Supporter
*****

Registered: 2002-03-27
Posts: 4673
Loc: The Netherlands
 Originally Posted By: Les
Umm... Mart, are you sure about de-tokenizing? The way it was told to me, a clear text script is tokenized before it is executed. If it is already pretokeinized, it will save having to do when launching it. The savings don't add up to much if anything at all. Most of a script's preformance is tied to how it makes system calls, not how it parses the code.


Uhmm....sorry my bad got things mixed up a bit. Not such a great morning today and I was doing all kinds of things at the same time.
_________________________
Mart

- Chuck Norris once sold ebay to ebay on ebay.

Top
#204418 - 2012-03-07 06:51 PM Re: Pre-Tokenizing Performance Benefit [Re: Mart]
DanD9272 Offline
Just in Town

Registered: 2012-03-02
Posts: 2
Loc: Portland OR
Great, thanks for the info!
Top
#204419 - 2012-03-07 07:48 PM Re: Pre-Tokenizing Performance Benefit [Re: DanD9272]
Glenn Barnas Administrator Offline
KiX Supporter
*****

Registered: 2003-01-28
Posts: 4403
Loc: New Jersey
Dan,

Our commercial login script is about 3450 lines long, including comments but not including any external scripts or language files. It's tokenized, more to mask the code than anything. Since we provide commercial support, we can't have admins changing the code and then requesting support.

The script will usually run in 2-3 seconds in a typical environment and generally under 10 seconds in a fairly complex environment with a few dozen resources and several hundred lookups in a dozen or so tables. We test with an untokenized version internally and don't see much difference in observed performance.

We have a setting that we use for performance tuning that logs timestapms at each major decision point, and have a standard configuration file that exercises each capability of the script. In Debug mode, no network connections are actually made, so the timing is for the script decisions and processes only. In 100 iterations, the deviation of the running duration for our standard test is about 20 ms, so it's pretty repeatable.

When I run with the timing parameter enabled and debug disabled (network connections being made), the difference between tokenized and untokenized versions is usually under 100 ms for a script duration of under 2 seconds. Take the 20ms run variance into account and the typical difference is around 85-88ms. In my opinion, this isn't a significant difference in performance from a user's perspective.

The biggest time hog on most login scripts is overall network response. Our script in debug mode might complete in 0.38 seconds, while out of debug mode and actually making network connections it could take upwards of 5 seconds. Five seconds may not seem like a lot, but compare it to the three-eights of a second baseline and it's more than a 13-time increase! There isn't always much you can do about server response and network performance, but clearly you can optimize your decision logic related to authentication and authorization. We've been tweaking our script since its release in 1995 to minimize the logic overhead as much as possible.

Glenn
_________________________
Actually I am a Rocket Scientist! \:D

Top
Page 1 of 1 1


Moderator:  Jochen, Allen, Radimus, Glenn Barnas, ShaneEP, Ruud van Velsen, Arend_, Mart 
Hop to:
Shout Box

Who's Online
0 registered and 2312 anonymous users online.
Newest Members
fgarcia, Viginette, ManuvdWielNL, Sir_Barrington, batdk82
17889 Registered Users

Generated in 0.153 seconds in which 0.115 seconds were spent on a total of 12 queries. Zlib compression enabled.

Search the board with:
superb Board Search
or try with google:
Google
Web kixtart.org