MS LogParser 2.2

I am IMPRESSED!

I've been wanting to parse log files or any types of text delimitered files for a long time. Been thinking about the various types of techniques using StreamReaeder and what not, but I still have to parse them myself. Mainly, I want to create a Log statistics program myself.

I ran into MS LogParser 2.2 the other day and on first impression, it was okay. I thought it's cool that you can use SQL to parse the text file, but what else could it do? How is the performance?

I ran a quick test and in about less than a second or mabye less? cuz I thought I just blinked my eyes,  it parsed 22138 rows of IIS logs. You heard me? 22138 rows of data and roughly around 11mb of data.

I tested again, running multiple queries and genreate different views of the same 22138 rows, it processes the page as if it's a normal page. No obviouis performance bottleneck. VERY IMPRESSED.

Although, when I tested a log file around 103mb (219,322 rows)  it took 10 seconds to process two different queries(a select all query and a more complicated aggregation query) and 4 seconds to process a simple select all query.

Really Interesting indeed. I doubt that I will ever get 103mb of hits in a day.  Even if i do,  all I gotta do is  work out the  page logic and make sure I don't  put too many queries on a single  page.  Definitly doable. Time to have some fun in my spare time. :)

The next step is to simulate the same display using the traditional .net programming and benchmark against the LogParser. I doubt that I can do better though.

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS