Can anyone recommend a log search tool / application / script
Question asked by Nathalie V - 4/1/2015 at 6:53 AM
Answered
Unfortunately the way the Smartermail logs are output is that there are multiple lines of logs for each email transaction (the biggest problem is that the recipient and sender email address are on two different lines).
 
This makes it very difficult to automate searching the logs for specific scenarios.  For example right now we have a user that somehow their email forwarder setting is blank (no idea how as Smartermail won't let you even create an email forwarder with no forwarding address). This was working fine for a long time but somehow it got wiped out.
 
The user wants to know who emailed their address during the time this was broken, so we need to search the logs for their email address and then find the corresponding sender email address.
 
 
I need to search a lot of email transactions so trying to find a way to automate this.
 
So there are 2 search operations that have to happen (first finding all lines that contain a particular recipient email address), and then parsing the ID code from each and searching on that to find the sender email address.

At this point I don't care if this is a WIndows application, Perl Script, Bash Script, or Powershell script, or whatever but pretty desperate to find something that can do this.  Even if the output isn't pretty and contains more info that I need that is fine.
 
Something that runs on WIndows would be ideal but if I have to copy the logs to a linux box to get this done that's fine too.
 
 
Can anyone recommend a tool or script that can search this way?
 
 
Thank you
 

3 Replies

Reply to Thread
2
Nathalie V Replied
Marked As Answer
Awesome I found this thread:
 
This tool is perfect and exactly what I needed.
 
1
Scarab Replied
I used to use the utility that Nathalie V suggested. It's a very good utility but when your Log Files get several hundred MBs to over a GB in size it takes forever (sometimes a half-hour to an hour) to parse...and you have to manually run your queries.
 
Personally I use Grep (via Cygwin for Windows) and have never looked back. Even complex queries are almost instantaneous and you can use a Bash Script to consolidate the result of one query into a second query.
0
Nathalie V Replied
Scarab would you mind sharing any of your grep or bash script search examples. If you don't mind sharing some examples of commands or your bash script but can't paste the code here perhaps post it on pastebin for example and post the link here? Would be much appreciated.

Reply to Thread