There are many a time you wonder how exactly LEM would handle certain Syslog messages. Typically, these situations arise when you are transitioning from log aggregation tools to SIEM. So,you may have a Syslog archive and you are interested in knowing what would be the corresponding LEM normalized events for specific log messages in that archive. This could be to ascertain device support, to decide how your correlation rules must be defined, etc. Moreover, you may not be in a position to re-create the event readily (maybe a specific Virus Attack message).
Engineer's Toolset Syslog is an easy, handy and quick way to answer these questions.
Pre-requisites:
- You have LEM installed
- You have Engineer's Toolset installed
- You have a Syslog sample that you want to test
Just earlier today, I was sent a couple of logs from a PowerTech Agent for iSeries and if LEM would handle that or would it require a different iSeries agent. ( I have changed identifiable information like IPs and user names below. Clearly, src IP of 10.10.10.10 going to a dest IP of 20.20.20.20 would be a Halloween special)
CEF:0|PowerTech|Interact|3.0|TPW0016|An invalid password was entered for user profile B1C1D1E1F1.|2|src=10.10.10.10 dst=20.20.20.20 msg=TYPE:JRN CLS:IDS JJOB:QINTER JUSER:QSYS JNBR:632886 PGM:QLESPI OBJECT: LIBRARY: MEMBER: DETAIL:P B1C1D1E1F1 QPADEV001R
CEF:0|PowerTech|Interact|3.0|TCA0001|The data queue SWIPRODA/TERM263441 authority has been changed for user profile *PUBLIC.|2|src=30.30.30.30 dst=40.40.40.40 msg=TYPE:JRN CLS:AUD JJOB:BFDW774201 JUSER:EWAUECG100 JNBR:263441 PGM:SCA0100X OBJECT: LIBRARY: MEMBER: DETAIL:A TERM263441 SWIPRODA *DTAQ *PUBLIC Y Y Y Y Y Y Y Y Y Y GRT 0000 00000 * *
So, these are the steps I follow to perform this test
Step 1: Identify an unused Syslog facility in LEM
This can be done using the checklogs command. In my case, I determined local2 was empty
Step 2: Send a test Syslog message from Toolset to this facility.
This is done from the 'Send Syslog Test Message' button of the 'Syslog Server' tool in Toolset. The link is highlighted below
And verify that the message has reached LEM using the checklogs command mentioned in Step 1. The actual message you send here could be any gibberish. This step is simply to ensure there isn't anything blocking Syslog traffic between the Toolset and LEM
Step 3: Turn on a LEM connector to read from this facility's log
In this case, I searched from LEM's connector list and created an instance of the PowerTech connector to read from /var/log/local2.log
(assumption: you are familiar with LEM connnectors. If not, please refer to the LEM evaluation guide)
Step 4: Send a Syslog message from Toolset to LEM
A very important thing to note here is that you must insert a leading space to the message you copy. Also, note that I have chose Facility Code as 'local use - level 2' which is same as saying ''Facility local2'
Step 5: See the normalized events in LEM (or not)
If the connector is able to handle this message and your Toolset IP isn't a managed node in LEM, you should see a message pop-up in the LEM console that 'New nodes were found' which you can click through (but not essential). You can then use your MONITOR view or nDepth search to see the normalized events
For the 2 messages, the corresponding normalized events look like below
Conclusion: LEM's connector for PowerTech iSeries Agent has been verified completely offline!
Similarly, I was sent the following Fortigate log lines (again, I have changed identifiable information below like IPs, users, computer names, etc). Don't forget the leading space as mentioned in Step 4 above
2014-10-08 11:32:42,local7.notice,x.x.x.x, date=2014-10-08 time=11:32:42 devname=BLAHBLAH devid=BLAHBLAH logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=192.168.x.x srcport=51923 srcintf="VLAN108" dstip=202.x.x.x dstport=53 dstintf="vlan815" sessionid=50229650 status=accept policyid=7 dstcountry="Country123" srccountry="Reserved" trandisp=snat transip=203.x.x.x transport=51923 service=DNS proto=17 duration=180 sentbyte=77 rcvdbyte=93 sentpkt=1 rcvdpkt=1
And, it resulted in a UDPTrafficAudit event like below
One thing to note is that the timestamp in millseconds and the IP are prefixed by LEM's Syslog Server while saving the message into the /var/log partition (which is obvious when you see the output of the checklogs command as shown below).
So, if you have a log sample that has timestamp and IP as the first 2 fields of your sample, remove those 2 fields and send the rest (and not forgetting the leading space) as the test message. This is so that the format is identical to what would have been the format if the device were to send it directly to LEM