Cluster Analysis for Big-K Data: Models and Algorithms based on K-indicators
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Cluster analysis is a fundamental unsupervised machine learning strategy with wide-ranging applications. When clustering big data, existing methods of choices increasingly encounter performance bottlenecks that limit solution quality and efficiency. To address such emerging bottlenecks, we propose a new clustering model, called K-indicators, based on a ``subspace matching" viewpoint. This non-convex optimization model allows an effective semi-convexification scheme, leading to an essentially deterministic, two-layered alternating projection algorithm called KindAP that requires neither random initialization nor parameter-tuning, while maintaining a complexity linear in the number of data points. We establish global convergence for the inner iterations and an exact recovery result for data sets with tight clusters. Built on the basic K-indicators model, a more advanced model is constructed to perform simultaneous outlier detection and cluster analysis. Under the spectral clustering framework, extensive experimental results on both synthetic datasets and real datasets show that the proposed methods exhibit improved scalability in terms of both solution quality and time compared to K-means and other baseline methods. An open-source software package in Python has been developed and released online that implements the algorithms studied in this thesis.
Description
Advisor
Degree
Type
Keywords
Citation
Yang, Yuchen. "Cluster Analysis for Big-K Data: Models and Algorithms based on K-indicators." (2021) Diss., Rice University. https://hdl.handle.net/1911/111176.