The use of data topology in unsupervised clustering of high-dimensional data with self -organizing maps

Date
2008
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

High-dimensional data is increasingly becoming common because of its rich information content that can provide comprehensive characterization of objects (patterns) in real world situations. Unsupervised clustering aims to utilize this rich information content for detailed discovery of distinct patterns. However, conventional clustering methods may be inadequate for capturing intricate structure in high-dimensional and large data, such as hyperspectral images or genetic microarray data. These data usually have many meaningful clusters, including interesting rare ones, whose discovery may be of great importance. Yet, faithful delineation of clusters may be impossible and rare clusters may be undiscovered due to limitations of clustering methods. A powerful method in high-dimensional data analysis is the Self-Organizing Map (SOM) [1]. An SOM is a neural learning algorithm that quantizes data spaces and spatially orders the quantization prototypes on a rigid lattice. The information learned by the SOM can be exploited to extract detailed cluster structure either by explanatory visualization or by clustering the SOM prototypes. Available SOM visualization or clustering schemes that are successful for relatively simple data often miss the finer structure in high-dimensional and large data. Our goal is to provide advanced visualization and clustering schemes for SOMs for detailed cluster extraction. The main contribution is the exploitation of the data topology inherent in the SOM's knowledge but largely underutilized in existing approaches. We achieve this by proposing a “connectivity matrix” CONN , which is a weighted Delaunay triangulation. CONN and its specific rendering on the SOM (CONNvis) help detailed delineation of clusters which can be obscure in existing schemes. The capability of CONNvis in cluster extraction inspires a new index for the evaluation of cluster validity. The proposed index, Conn_Index , is shown to be effective in various applications of synthetic and real data sets. Based on our experiences, we expect CONN and Conn_Index to help produce an automated clustering of the SOM which may be as detailed as can be achieved with the interactive methods including our CONNvis clustering. This will be a significant achievement for structure discovery given that automated schemes in previous works produce results inferior to results from semi-manual procedures.

Description
Degree
Doctor of Philosophy
Type
Thesis
Keywords
Electrical engineering, Artificial intelligence, Computer science, Applied sciences, Clustering Data mining, Data topology, Knowledge discovery, Self-organizing maps, Visualization
Citation

Tasdemir, Kadim. "The use of data topology in unsupervised clustering of high-dimensional data with self -organizing maps." (2008) Diss., Rice University. https://hdl.handle.net/1911/103530.

Has part(s)
Forms part of
Published Version
Rights
Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
Link to license
Citable link to this page