George Papayiannis » Data Mining http://www.sematopia.com Sat, 01 May 2010 04:50:01 +0000 http://wordpress.org/?v=2.8.4 en hourly 1 k-means and EM clustering algorithms http://www.sematopia.com/2006/04/k-means-and-em-clustering-algorithms/ http://www.sematopia.com/2006/04/k-means-and-em-clustering-algorithms/#comments Sun, 16 Apr 2006 20:32:33 +0000 George A. Papayiannis http://www.sematopia.com/?p=72 The final data mining assignment (which was due in the middle of exams) was to implement the k-means and EM algorithms in C. These are two pretty simple clustering algorithms, with k-means (or renditions of k-means) used in industry all the time. Both algorithms work in 2-dimensions and take an input file of data points separated by a space.

K-means works as follow:

  • Place K points into the space represented by the objects that are being clustered. These points represent initial group centroids.
  • Assign each object to the group that has the closest centroid.
  • When all objects have been assigned, recalculate the positions of the K centroids.
  • Repeat Steps 2 and 3 until the centroids no longer move. This produces a separation of the objects into groups from which the metric to be minimized can be calculated.

      The EM algorithm is a bit more complicated, but has generally the same idea:

      em

      Since this was in the middle of exams, I wrote both solutions (code) in one night (7pm to 3am), so no guarantee whatsoever. Take a look; if you’d change anything let me know. The code is licensed under GPL v2.

      ]]> http://www.sematopia.com/2006/04/k-means-and-em-clustering-algorithms/feed/ 5 Apriori Online – Mining for Associations http://www.sematopia.com/2006/04/apriori-online/ http://www.sematopia.com/2006/04/apriori-online/#comments Thu, 06 Apr 2006 01:24:40 +0000 George A. Papayiannis http://www.sematopia.com/?p=70 Why are associations interesting? For the same reason data mining in general is interesting – you can learn things you would have never thought of. Take Shoppers Drug Mart as an example, don’t quote me on this, but I bet when you buy a bulk of goods (and swipe you Shoppers Optimum Card) some database is registering your personal information (age, sex, location, etc.) and everything you purchased (Tylenol, Kleenex, gum, etc..).

      With high confidence (no Apriori pun indented) I can say that periodically Shoppers Drug Mart mines that data for interesting association, like perhaps:

      - Males between the ages of 25 and 45, between the hours of 7:00pm to 12:00am purchase toilet paper and Mach 3 razors.

      What would Shoppers do? In the evenings they would set up and isle display of expensive toilet paper. At the front of that isle (as the customer is walking to the cashier) they would set up another display of Mach 3 razors…10 packs. They’ll do this because they’ve found associations, which tell them with high confidence; males will purchase these two items together.

      For a data mining project this past semester, one of the deliverables we created was a web based version the Apriori algorithm. The algorithm was initially developed by Agrawal et al (Agrawal 93, Agrawal 94) and is used to find association rules from a dataset. This online version of the Apriori algorithm is based off an implementation by Christian Borgelt’s.

      The URL is http://apriori.sematopia.com. Aside from simply executing the Apriori implementation by Christian, the web-based application does a series of pre-processing on the data. To begin, all missing values in a given numeric column are filled in with the average of that attribute. Then categorization occurs on all numeric attributes to place each column value in a bin. This is based off of the Category Granularity (how many categories to make in a given column) and the Categorization Threshold (if the number of unique values in the column are < this number, then categorization will not be performed). There are many options (besides Category Granularity & Categorization Threshold) available that can be set to adjust the results Apriori returns; these include the max/min support, the min confidence, min sets per rules, etc.

      The file uploaded must be in CSV format with the first row as headings, and each subsequent row being observations. The max file size that can be uploaded is 200,000 bytes. After the algorithm finishes (which should take less than 3 seconds) the first 100 association rules will be shown. You can download the full output from the website.

      The associations produced will look like this:

      verbal_sat=c3_g7(442,469) <- math_sat=c2_g7(500,530) act=c4_g6(21,23) (36.1, 93.0)

      c3 means column 3
      g7(442,469) means group (bin) 7 were this specific bin was between 442 and 469
      (36.1, 93.0) at the end of each rule, the support and confidence is given

      Each IP address will be allowed to run 5 datasets per day.
      Good luck, happy mining.
      http://apriori.sematopia.com

      ]]>       http://www.sematopia.com/2006/04/apriori-online/feed/ 6