We have probably all come across John Naisbett's famous phrase "We are drowning in information, but starving for knowledge." This phrase takes on more truth with each passing day. Most corporate legacy systems have years of data on purchases, clients, products, etc. But it is just that - so much data. On its own it is meaningless; it is we and our users who have to make sense of it, and turn it into meaningful and useful information. So ironically, as more and more data piles up around us, we are confronted with the paradox of "the more data there is, the less information we have."
So the real question for us in this chapter is how can we distill that mountain of data into some useable knowledge? For example, how can a product manager at a company take all of the information that is in the various databases and use it to market products smarter? Or how can they see subtle trends that will permit a better use of limited advertising funds?
To take this to an extreme, consider the American Stock Exchange. With all of the data and all of the investment companies employing sophisticated algorithms to gain an edge, to a large extent securities trading has become a game of computers against computers. Humans are there to advise only on a meta level.
In this chapter we are going to go beyond the opening and closing of recordsets. Here we are actually going to create our own simple data mining program. Using the data in the
table, we will apply our programs to the data to see if there are any underlying trends. We will aggregate all of the detail from the orders and products table and distill it into a single table that we can apply our algorithms to. Yes, there is a bit of coding, but you will learn quite a few things along the way in this chapter.Nwind.mdb