Even though we could manage to store the data for several months with no problem, the other consideration is what happens when we come to use it. With over a million records per month being added to the IISLog table, any query which sorts and summarizes the data is going to get progressively slower as time goes on.
More to the point, do we actually need all this data? As far as the IISLog table goes, much of it is relatively useless. We probably aren't interested in the number of hits on each navigation bar button image, for example. We also have one record per hit, so if 100 visitors load the Home page in one hour, we get around 2,500 records in IISLog (including graphics and other items). Maybe we only want to know how many hits the page itself gets per hour, per day, or even per week.
So it makes sense to summarize the data on a regular basis, and archive the old data that we no longer need. How you do this depends on what information you want to keep. The big question is how do you know what to keep? It would certainly be unfortunate if the first question that your finance director asks, regarding traffic on the Web site, can't be answered because you dropped the data required that would provide the result.
After a great deal of consideration, we chose a staged summary and archive process for our site, and implemented it using a set of separate tables and stored procedures in SQL Server. We've provided the entire database, containing all these tables and stored procedures, with the samples available for this book on our Web site. You can download them and set up the complete process on your server from http://webdev.wrox.co.uk/books/1797.
Bear in mind that this is only one possible solution. You may have different requirements, and the sample code and database structure can easily be adapted to your specific situation.