Compression and knowledge discovery in ecology

Abstract

Knowledge discovery is the non-trivial process of identifying valid, novel, interesting, potentially useful and ultimately understandable patterns in data. It encompasses a wide range of techniques ranging from data cleaning to finding manifolds and separating mixtures. Starting in the early 50's, ecologists contributed greatly to the development of these methods and applied them to a large number of problems. However, underlying the methodology are some fundamental questions bearing on their choice and function. In addition, other fields, from sociology to quantum mechanics, have developed alternatives or solutions to various problems. In this paper, I want to look at some of the general questions underlying the processes. I shall then briefly examine aspects of 3 areas, manifolds, clustering and networks, specifically for choosing between them using the concept of compression. Finally, I shall briefly examine some of the future possibilities which remain to be examined. These provide methods of possibly improving the results of clustering analysis in vegetation studies.