#72878 - 2003-02-05 03:03 AM
HOWTO: Automated Log Maintenance - sample script
|
Glenn Barnas
KiX Supporter
Registered: 2003-01-28
Posts: 4402
Loc: New Jersey
|
"PerfMon logs need to be rotated every week, retaining the prior week's data. The batch file to do that is in D:\PerfMon, but it doesn't always work. The event logs are dumped and cleared every night and stored in D:\Logs. Most systems have a year's worth of files, but all we really need is 30 days worth. Oh, the event logs on the web servers in the DMZ aren't dumped by our program - firewall restrictions! They are probably full."
And so went my first day as the new network admin... Within a few short weeks, I found that the web servers needed to have their logs rotated on a weekly basis as well, and - while I was at it - could I manage to get the Cold Fusion logs under control as well? "Did I tell you we need 180 days of retention on the archived logs on all web servers? The script
that we wrote to do that fails more often than it works - it's that damn NT scheduler!"
OK, logs, logs everywhere, and few working tools to control them. After reviewing the tools that did exist, I got a feel for what applications needed log rotation, how long the retention periods should be, and which logs required long-term archival (by security mandate). Oh, did I tell you that there were four copies of the same script (well, not exactly, as each was just slightly different) doing the same thing in four different directories? How was that maintained? (answer: it wasn't!)
Well, let's come up with a solution by outlining the requirements:
1. Logs in a specific directory need to be moved/renamed on a periodic basis - the "Archiving" process. This requires knowlege of
a. the log source directory
b. the archive destination directory
2. The "Archived" logs need to be retained for some period, then deleted - the "Cleanup" process. This needs:
a. the retention period, in days
b. a method to quickly identify the oldest archived files
(I used A-! as the first chars, followed by the date, then original file name.)
3. Sometimes a service will need to be stopped prior to archiving, and then restarted. We need to know the service name.
4. Sometimes an external program must be run to prepare the logs for archiving.
5. Sometimes an external program must be run to properly restart the service.
6. The process should run without user intervention so it can be scheduled for off-hours execution.
7. The files that are archived or cleaned should be written to a log - a new log for every occurance. These logs should be removed using the same retention period.
8. A central log should be maintained (C:\) so an automated management tool can confirm the number of files archived and the date of the last run. This will be an INI style file.
Jeez - anything else?
Lets think about how we'll process this information. We can create a script and define each of the arguments on the command line. With Kix, that means referencing Kix32, the script, and lots of $VAR=val statements. This gets ugly for several reasons, but my favorite is CHANGE - the word that most administrators hate! If I create a task to run at midnight (with an Admin account) and load it up with complex parameters, I will be the one that has to change it (to protect the admin password). More work for me down the road. An alternative is putting the commands in a Batch file - now anyone could edit that. No, another security risk - who knows what extra commands could slip in there.
Kix has a wonderful capability that many of us might overlook - INI files! Now, I can create a scheduled task that only needs one parameter - the name of a section in the INI file. Any "power user" could edit that file without creating a huge security risk. I like that because it frees me up to write more Kix scripts!
Um - one more issue. Not everyone that will install or support this tool is an "administrator" by trade. So - I put together a batch file to "hide" the Kix32 command line, allowing the support staff to define a rather simple interface to the Kix script.
Sounds like a plan - lets look at the code. (Before we do, you'll notice references to \usr\local and \usr\local\bin. Because we support both Windows and Unix systems, and use a common scripting language - perl - we chose to make \usr\local the location to store all admin tool configuration files, and \usr\local\bin the location for scripts and stand-alone tools. You will need to mimic this, or change the parts of the script to fit your environment.)
We start all scripts by identifying the author, version, and a basic description of the script, along with a revision history.
code:
; Utility to archive and remove log files based on age
; Glenn Barnas / FRIT-EROC
; 3-4-2002
;
; VERSION 1.3
;
; Obtains settings from \usr\local\logmaint.ini, based on the SECTION argument
;
; Must be called as "kix32 \path\to\this\script\logcleanup $SECTION=section_name"
; See the LogMaint.BAT file for more information.
;
; see readme.txt or "Common Log Cleanup utility.doc" for more info
; 06/11/02 - Corrected error in debug code, added DEBUG option to the ini file,
; Added test code - MD command executes only if target doesn't exist
; Enclosed path parameters in quotes to support paths with spaces
; when used by shell commands (ugh!)
; 06/13/02 Added code to write errors to log file and log errors to LMSTATUS.ini
; to facilitate central log status collection.
BREAK on ; allow terminating the script without logging off
; this basically turns off "login script mode"
; NOTE - throughout the script, the $RTN variable is used to collect the return code
; of various functions. It is tested only when necessary. Its primary function
; is to prevent the return code from being displayed on the console (as a
; series of zeros or other numeric values)
Moving on, we do some preliminary setup - insure that the SECTION variable is defined, format the DATE value for later use, and read the parameters from the INI file section that was requested. Enveloping the ReadProfile function inside of the ExpandEnvironmentVars function allows environment variables to be used inside of the INI file. This is helpful when the only difference between a configuration on one machine or another is the %COMPUTERNAME% value.
You'll notice lots of comments. This is important, especially as you get older and begin suffering from CRAFT (Can't Remember A Flippin' Thing). It also helps when others need to maintain your code. Also note that during the evolution of this script that the process for "archive" was defined by the $BACKUP var, and "cleanup" by the $DELETE var.
code:
;===================================================================
; If the section argument isn't defined, carp and die
If $SECTION = ""
? "Required parameter to identify logs to archive was not specified."
? "Can't continue!" ?
Exit
EndIf
; format today's date for use in file names
$DATE = Substr(@DATE,1,4) + Substr(@DATE,6,2) +Substr(@DATE,9,2)
; Read the parameters from the LOGMAINT.INI file
; Wrapping the READ statement in the EXPANDENVIRONMENTVARS function allows
; environment variables to be defined in the INI file for the path values
; (ie SRCDIR=%SYSTEMROOT%\logs)
$INIFILE = "%SYSTEMDRIVE%\usr\local\logmaint.ini"
$SRCDIR = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "SRCDIR"))
$DSTDIR = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "DSTDIR"))
$DEBUG = ReadProfileString($INIFILE, "COMMON", "DEBUG")
$SFLIST = ReadProfileString($INIFILE, $SECTION, "FILES")
$BACKUP = ReadProfileString($INIFILE, $SECTION, "BACKUP")
$DELETE = ReadProfileString($INIFILE, $SECTION, "DELETE")
$MAXAGE = ReadProfileString($INIFILE, $SECTION, "AGE")
$SERVICE = ReadProfileString($INIFILE, $SECTION, "SERVICE")
$PREARCH = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "PREARCH"))
$POSTARCH = ExpandEnvironmentVars(ReadProfileString($INIFILE, $SECTION, "POSTARCH"))
; define the archive log name and location
$AFILE = "_" + $DATE + "_archive.wri"
$ARCHIVELOG = $DSTDIR + "\" + $AFILE
$FIRSTYEAR = 2001 ; anything older than this year is not calculated, just deleted
;====================================================================
; Make sure that the DSTDIR exists before we start
; future versions may support making multiple dirs directly by the MD command!
$P = $DSTDIR ; use a working VAR
$MP = "" ; Make this Path
If InStr($P, ":") = 2 ; If the second char is a ":"
$P = SubStr($P, 3, 252) ; drop the leading drive definition
$MP = Left($DSTDIR, 2)
EndIf
if Left($P, 1) = "\"
$P = Substr($P,2,252) ; remove the leading dir delimiter for now
EndIf
$DIRS = split($P,"\",-1) ; put the individual dirs of the path into an array
for each $D in $DIRS
$MP = $MP + "\" + $D ; add the next dir to the root path
If Not Exist($MP) ; make sure each dir in the path exists
md $MP ; create it if necessary
EndIf
If $DEBUG = "T"
? "processing $D"
? "$MP"
? "@SERROR"
EndIf
Next
; Open the archiving log file
$RTN = Open(1,$ARCHIVELOG,5)
If @ERROR
? "Error opening archive log file - exiting @CRLF FILE: $ARCHIVELOG" ?
exit
EndIf
; Append today's date and a message to the log file
$RTN = WriteLine(1,"Log cleanup executed on @DATE using [" + $SECTION + "] parameters." + @CRLF)
; Update the LMStatus file - central file used by network monitor software (ie: Tivoli)
$TMP = ReadProfileString("c:\LMStatus.ini", $SECTION, "LastRunD")
If $TMP = ""
$TMP = "Never" ; LastRunD doesn't exist, set PriorRun to "Never"
EndIf
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "PriorRun", $TMP)
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "LastRunD", @DATE)
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "LastRunT", @TIME)
;======================================================================
; If BACKUP <> 0, then MOVE the file(s) from the SRCDIR to the DSTDIR, adding the
; current date to the front of the file name (test.log becomes 20020301_test.log)
; Using MOVE in the same directory results in a simple RENAME operation!
; The A-! at the beginning of the new name allows quick identification of archived files,
; especially when the archiving occurs in the same folder as the original logs.
If $BACKUP <> 0
$RTN = WriteLine(1,"Archive process starting." + @CRLF)
$CNT = 0
$ECNT = 0
The code here stops a service, if defined, to allow the log files to be managed. We prefer XNET for this task, and have deployed a copy to \usr\local\bin on every server during the final build process. For a more portable solution, simply use the NET.EXE command instead. Also see the section later where the service is started.
code:
; The service start/stop function relies on XNET.EXE being available in the system PATH
If $SERVICE <> "" ; Stop any service defined prior to archiving
If $DEBUG = "T" ; to insure the files can be moved/renamed
? "xnet stop " + Chr(34) + $SERVICE + Chr(34)
Else
Shell "xnet stop " + Chr(34) + $SERVICE + Chr(34)
EndIf
EndIf
; PREARCH/POSTARCH - commands to run prior to archiving. An alternative method
; for stopping but not starting services that may require manual password entry, or
; procedures that require multiple steps to prepare the files for archiving.
; It is assumed that the creator of the config file will include any quotes or
; other escape sequences to handle proper argument passing!!!
If $PREARCH <> "" ; If a PREARCH command is defined
If $DEBUG = "T"
? "running $PREARCH"
Else
Shell "$PREARCH" ; run it!
EndIf
EndIf
; Loop through all files in the SRCDIR to see what should be MOVED
$FILES = Split($SFLIST, ",", -1) ; split the list into an array
For Each $FILE in $FILES ; process each arg in the array
$SPATH = $SRCDIR + "\" + $FILE
$FNAME = Dir($SPATH) ; find the matching files in the SRC directory
While $FNAME <> "" And @ERROR = 0 ; process all non-null names
; skip parent paths ("." & "..") and previously archived files
If Left($FNAME,3) <> "A-!" And $FNAME <> "." AND $FNAME <> ".."
; The following line was shortened to fit the KBBS display
$MSG = " " + $SRCDIR + "\" + $FNAME + " -> "
$MSG = $MSG + $DSTDIR + "\A-!" + $DATE + "_" + $FNAME + @CRLF
$RTN = WriteLine(1,$MSG)
; The following line was shortened to fit the KBBS display
$CMD = "cmd.exe /c move " + CHR(34) + $SRCDIR + "\" + $FNAME + CHR(34) + " "
$CMD = $CMD + CHR(34) + $DSTDIR + "\A-!" + $DATE + "_" + $FNAME + CHR(34)
If $DEBUG = "T"
? "$CMD"
Else
shell $CMD ; move/rename the file
If @ERROR <> 0 ; if an error occured,
? ? "@SERROR" ; display the error and write it to the log
$RTN = WriteLine(1," @SERROR")
$ECNT = $ECNT + 1 ; increase the error count
EndIf
EndIf
$CNT = $CNT + 1
EndIf
$FNAME = Dir() ; get the next file
Loop ; While $FNAME <> "" ...
Next ; $FILES
$RTN = WriteLine(1,"Archive process complete, " + $CNT + " files archived." + @CRLF + @CRLF)
; write to the common log showing # of files archived and number of errors encountered
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "Archive", $CNT)
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "ArchErr", $ECNT)
If $POSTARCH <> "" ; run a post-acrhiving command if defined
If $DEBUG = "T"
? "$POSTARCH"
Else
Shell "$POSTARCH"
EndIf
EndIf
If $SERVICE <> "" ; start a service if defined
If $DEBUG = "T"
? "xnet start " + Chr(34) + $SERVICE + Chr(34)
Else
Shell "xnet start " + Chr(34) + $SERVICE + Chr(34)
EndIf
EndIf
Else
; rather than write nothing (and leave the viewer guessing), say that we did nothing!
$RTN = WriteLine(1,"Archive process not performed, 0 files archived." + @CRLF + @CRLF)
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "Archive", 0)
EndIf ; BACKUP PROCESS
;========================================================================
; If DELETE <> 0 then DELETE all files in DSTDIR older than MAXAGE days that have
; the auto-archive filename prefix of "A-!"
; Include "*_archive.wri" (our own log file) in the list of files that we check
If $DELETE <> 0
$RTN = WriteLine(1,"Cleanup process starting." + @CRLF)
$CNT = 0
; Create an array of monthly day values (cumulative days, 13 values including the ZERO array cell)
; TO keep the logic simple, the array starts with a ZERO postion, but we use only positions 1-12
; which correspond to the month integer values. Position ZERO holds "0" as a placeholder only.
$MV = split("0,0,31,59,90,120,151,181,212,243,273,304,334", ",", -1)
; calculate today's date in days from 1/1/2001 (first day of $FIRSTYEAR)
$CDATE = split(@DATE, "/", -1)
$X = Val($CDATE[1])
$CDV = ((Val($CDATE[0]) - $FIRSTYEAR) * 365) + $MV[$X] + $CDATE[2]
; archived files either begin with A-! or are our own log files
; if we don't do archiving (BACKUP=0) then archived files are defined
; in the SFLIST variable. The SFLIST value is included when we DO archiving
; ONLY when the SRCDIR and DSTDIR values are different.
If $BACKUP = 0
$ALLFILES = $SFLIST + ",_????????_archive.wri"
Else
$ALLFILES = "A-!*,_????????_archive.wri"
If $SRCDIR <> $DSTDIR ; Add SFLIST if SRC and DST dirs are different
$ALLFILES = $SFLIST + "," + $ALLFILES
EndIf
EndIf
; Loop through all files in the DSTDIR to see what should be DELETED
$FILES = Split($ALLFILES, ",", -1) ; split file list into array
For Each $FILE in $FILES ; process the files in the array
$DPATH = $DSTDIR + "\" + $FILE
$FNAME = Dir($DPATH) ; read the directory
While $FNAME <> "" And @ERROR = 0
If $FNAME <> $AFILE
; get timestamp
$FD = Left(GetFileTime($DSTDIR + "\" + $FNAME),10)
$FDATE = split($FD, "/", -1)
If Val($FDATE[0]) < $FIRSTYEAR ; prior to FIRSTYEAR? ANCIENT FILE!!
If $DEBUG = "T"
? "Del " + $DSTDIR + "\" + $FNAME
Else
Del $DSTDIR + "\" + $FNAME
EndIf
$CNT = $CNT + 1
; Append the deleted file to the log file
$RTN = WriteLine(1," " + $DSTDIR + "\" + $FNAME + " - " + $FD + " - ANCIENT!" + @CRLF)
Else
; convert to Days since 1/1/2001 if year is 2001 or later
$X = Val($FDATE[1])
$FDV = ((Val($FDATE[0]) - $FIRSTYEAR) * 365) + $MV[$X] + $FDATE[2]
; find number of days between file date and today
; delete $FNAME if difference is greater than $MAXAGE
If ($CDV - $FDV) > $MAXAGE
If $DEBUG = "T"
? "Del " + $DSTDIR + "\" + $FNAME
Else
Del $DSTDIR + "\" + $FNAME
EndIf
$CNT = $CNT + 1
; Append the deleted file to the log file
$RTN = WriteLine(1," " + $DSTDIR + "\" + $FNAME + " - " + $FD + @CRLF)
EndIf
EndIf
EndIf
$FNAME = Dir()
Loop
Next
$RTN = WriteLine(1,"Cleanup process complete, " + $CNT + " files deleted." + @CRLF + @CRLF)
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "Cleanup", $CNT)
Else
$RTN = WriteLine(1,"Cleanup process not performed, 0 files deleted." + @CRLF + @CRLF)
$RTN = WriteProfileString("c:\LMStatus.ini", $SECTION, "Cleanup", 0)
EndIf ; DELETE
; Close the log file
$RTN = Close(1)
That's the end of the script. Here's the BAT file that I use to call it from the scheduler.
code:
@echo off
REM Front-end batch file used to pass arguments to the LogCleanup script
REM Glenn Barnas / FRIT - 6/2002
REM One parameter is required - if it's blank, go complain
If "%1"=="" goto USAGE
REM run the kix script, passing the SECTION value
kix32 %SYSTEMDRIVE%\usr\local\bin\logcleanup.kix $SECTION=%1
goto DONE
:USAGE
echo.
echo Must specify SECTION name from \usr\local\logmaint.ini file
echo.
:DONE
And finally, a sample of the INI file that controls the process.
code:
[COMMON]
DEBUG=
[EVENTLOG]
# delete all event logs older than 30 days
# this is for TRUSTED network where the event logs are exported nightly by the security group
# we don't archive them, just clean up the .EVT files archived by another process
SRCDIR=
DSTDIR=D:\EventLogs
FILES=*.evt
BACKUP=0
DELETE=1
AGE=30
[EVENTLOGDMZ]
# DMZ Server event log management
# Export / Clear the event logs, move them to the EventLogs folder,
# delete all archived event logs older than 30 days
SRCDIR=C:\TEMP
DSTDIR=D:\EventLogs
FILES=APP.evt,SEC.evt,SYS.evt
BACKUP=1
DELETE=1
AGE=30
# dump the event logs to local files in C:\TEMP
PREARCH=kix32 c:\usr\local\bin\eldump.kix
[PERFMON]
# NT ONLY!!! 2K systems automatically perform this function!
# Rename the current week's perfmon log, then delete any older than 2 weeks
# only the PERFMON.LOG file is backed up
# the MONITOR service is stopped prior to running, then restarted
SRCDIR=D:\PerfMon
DSTDIR=D:\PerfMon
FILES=PerfMon.log
BACKUP=1
DELETE=1
AGE=12
SERVICE=MONITOR
[WEBTMP]
# A web app generates TMP files for ad-hoc reports during the day. All the .TMP files
# should be deleted every night - no archiving is necessary
SRCDIR=
DSTDIR=D:\WebRoot\app3\tmp
FILES=*.tmp
BACKUP=0
DELETE=1
AGE=-1
SERVICE=
Whew!
I have a script that installs this utility onto our servers and creates a scheduled task. It doesn't - however - apply the proper user ID and password to the task. My next step is to create a kix script that uses the ScheduleTask UDF to correct this deficiency. ![[Wink]](images/icons/wink.gif)
Well, I hope that the script, samples, and explainations of the process proved helpful. Like any code, there may be better, faster, or just plain different ways to do it.
Glenn
_________________________
Actually I am a Rocket Scientist! 
|
|
Top
|
|
|
|
Moderator: Glenn Barnas, NTDOC, Arend_, Jochen, Radimus, Allen, ShaneEP, Ruud van Velsen, Mart
|
0 registered
and 756 anonymous users online.
|
|
|
