Efficient clustering of web-derived data sets

• Author(s): Luís António Diniz Fernandes de Morais Sarmento ; Eugénio da Costa Oliveira ; Alexander P. Kehlenbeck ; Lyle Ungar
• Date: 2009
• Persistent ID: http://hdl.handle.net/10216/15175
• Origin: Repositório Aberto da Universidade do Porto
• Subject(s): Ciências Físicas; Ciência de computadores; Informática
• Document Type: Conference Object
• Language: Portuguese

Description

Many data sets derived from the web are large, high-dimensional, sparse and have a Zipfian distribution of both classes and features. On such data sets, current scalable clustering methods such as streaming clustering suffer from fragmentation, where large classes are incorrectly divided into many smaller clusters, and computational efficiency drops significantly. We present a new clustering algorithm based on connected components that addresses these issues and so works well on web-type data.