- Open Open Peer Review
Partitioning clustering algorithms for protein sequence data sets
BioData Miningvolume 2, Article number: 3 (2009)
Genome-sequencing projects are currently producing an enormous amount of new sequences and cause the rapid increasing of protein sequence databases. The unsupervised classification of these data into functional groups or families, clustering, has become one of the principal research objectives in structural and functional genomics. Computer programs to automatically and accurately classify sequences into families become a necessity. A significant number of methods have addressed the clustering of protein sequences and most of them can be categorized in three major groups: hierarchical, graph-based and partitioning methods. Among the various sequence clustering methods in literature, hierarchical and graph-based approaches have been widely used. Although partitioning clustering techniques are extremely used in other fields, few applications have been found in the field of protein sequence clustering. It is not fully demonstrated if partitioning methods can be applied to protein sequence data and if these methods can be efficient compared to the published clustering methods.
We developed four partitioning clustering approaches using Smith-Waterman local-alignment algorithm to determine pair-wise similarities of sequences. Four different sets of protein sequences were used as evaluation data sets for the proposed methods.
We show that these methods outperform several other published clustering methods in terms of correctly predicting a classifier and especially in terms of the correctness of the provided prediction. The software is available to academic users from the authors upon request.
In bioinformatics, the number of protein sequences is more than half a million, and it is necessary to find meaningful partitions of them in order to detect their functions. Early approaches of comparing and grouping protein sequences are alignment methods. In fact, pair-wise alignment is used to compare and to cluster sequences. There are two types of pair-wise sequence alignments, local and global [1, 2]. Smith and Waterman local alignment algorithm  helps in finding conserved amino acid patterns in protein sequences. Needleman and Wunsch global alignment algorithm  attempts are made to align the entire sequence using as many characters as possible, up to both ends of each sequence. In order to cluster a large data set of proteins into meaningful clusters, the pair-wise alignment is computationally expensive because of the large number of comparisons carried out. In fact, each protein of the data set should be compared to all others of the data set.
For this reason the pair-wise alignment methods are not efficient to cluster a large set of data. These approaches do not consider the fact that the data set can be too large and may not fit into the main memory of some computers.
The main objective of the unsupervised learning technique is to find a natural grouping or meaningful partition using a distance function [5, 6]. Clustering is a technique which has been extensively applied in a variety of fields related to life science and biology. In sequence analysis, clustering is used to group homologous sequences into gene or protein families.
Many methods are currently available for the clustering of protein sequences into families and most of them can be categorized in three major groups: hierarchical, graph-based and partitioning methods. Among these various methods, most are based on hierarchical or graph-based techniques and they were successfully established. In fact, COG  uses a hierarchical merging of clusters and a manual validation to prevent chaining of multi-domain families. ProtoNet  uses a special metric as described by Sasson et al.  to merge clusters. Picasso  uses multiple alignments as profiles which are then merged hierarchically. ClusTr  and GeneRage  use standard single linkage clustering approaches. SYSTERS  combines hierarchical clustering with graph-based clustering. ProtoMap  and N-cut [15, 16] methods use graph-based clustering approaches. ProClust  uses an extension of the graph-based clustering approach proposed by . ProClust algorithm is based on transitivity criterion and it is capable of handling multi-domain proteins. TribeMCL  applies the Markov clustering approach (MCL) described by Van Dongen . This method operates on a graph that contains similarity information obtained by pair-wise alignment of sequences.
A small amount of partitioning techniques is used in the protein sequence clustering field. Guralnik and Karypis  have proposed one method based on a standard k-means approach where proteins are represented by vectors. However, no tool or database resulting from this interesting work has been made available to the scientific community. JACOP  uses the partitioning algorithm implemented under the name PAM (Partitioning Around Medoids) in the R statistical package . JACOP is based on a random sampling of sequences into groups. It is available on the MyHits platform  where user can submit his own data set. Methods presented bellow are not generally tools since they cannot be applied to cluster a user-provided data set. In fact, several of these methods have been applied to large known data sets and user can only consult the resulting classifications stored in databases. Among the protein sequence clustering methods defined bellow only ProClust, TribeMCL and JACOP are accessed by the community and user can classify his own sequence set.
The main idea here is to design and develop efficient clustering algorithms based on partitioning techniques, which are not very investigated in protein sequence clustering field, in order to cluster large sets of protein sequences. In fact, the number of protein sequences available now is very important (in the order of millions) and hierarchical methods are computationally very expensive so they cannot be extended to cluster large protein sets. However, partitioning methods are very simple and more appropriate to cluster large data sets . For these reasons, we propose here new clustering algorithms based on partitioning techniques which aim to find meaningful partitions, to improve the classification's quality and to reduce the computation time compared to the published clustering tools, ProClust, TribeMCL and JACOP, on different data sets.
Several partitioning clustering algorithms have been proposed in literature. K-means [23, 25–27] is a standard partitioning clustering method based on K centroids of a random initial partition which is iteratively improved. LEADER [28, 29] is an incremental partitioning clustering algorithm in which each of the K clusters is represented by a leader. CLARA (Clustering LARge Applications)  is a partitioning algorithm based on a combination of a sampling approach and the PAM algorithm. CLARANS (Clustering Large Applications based on RANdomized Search)  algorithm views the process of finding optimal medoids as searching through a certain graph, in which each node represents a set of medoids.
We adapted the partitioning algorithms cited bellows to protein sequence data sets. These proposed algorithms are named: Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS. Performance measures are used to evaluate the proposed methods and to compare them with ProClust, TribeMCL and JACOP results.
To facilitate subsequent discussion, the main symbols used through the paper and their definitions are summarized in Table 1.
The main objective in the proposed algorithms Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS is to produce K clusters from a data set D of n protein sequences, so that the objective function f(V) is maximized.
f(V) is the global score function that evaluates the clustering quality and it is as follows
Where Ri is the centroid of the group i for which belong the object Oj and Score (Oj, Ri) is the alignment score of the protein sequences Oj and Ri, calculated as follows
Where S (A i , B j ) is the substitution score of the amino acid A i by B j as determined from a scoring matrix and g(n) is the total cost of penalties for a gap length n. The gap is defined as follows
Where Po is the gap opening penalty and Pe is the gap extension penalty.
We chose Smith and Waterman local alignment algorithm for computing alignment score. The choice of this algorithm was motivated by the sensitivity for low-scoring alignments  compared to heuristic algorithms such as FASTA  and BLAST , and by execution time  compared to Needleman and Wunsch global alignment algorithm.
We present here Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS partitioning clustering algorithms for protein sequence sets.
The Pro-Kmeans algorithm proposed here, starts by a random partition of the data set D into K clusters and then uses the Smith Waterman algorithm to compare proteins of each cluster Si (i ∈ [1..K]) and to compute SumScore(Si, Oj) of each protein j in Si as follows
Where m is the size of the subset S i , for which belongs the object O j .
The sequence O j in each cluster S i which has the maximum SumScore(S i , O j ) is considered as the centroid R i of the cluster. The Smith Waterman algorithm is used here also to compare each protein O h of the data set D with centroids and to assign the object to the nearest cluster where the R i have the maximum score of similarity with the object O h . Pro-Kmeans proceeds to this procedure for a number of times, q, in order to maximize the f(V) function. Input parameters are the number of clusters, K, and of iterations, q, and as outputs the algorithm returns the best partition of the training base D and the center, or mean, of each cluster S i . Pro-Kmeans algorithm is illustrated in Figure 1.
Pro-LEADER is an incremental algorithm which selects the first sequence of the data set D as the first leader, and use the Smith Waterman algorithm to compute the similarity score of each sequence in D with all leaders. The algorithm detects the nearest leader R i to each sequence O j and compares the score, Score(R i , O j ), with a pre-fixed Threshold. If the similarity score of R i and O j , is more than the Threshold, O j is considered as a new leader and if not, the sequence O j is assigned to the cluster defined by the leader R i . Pro-LEADER is thus an incremental algorithm in which each of the K clusters is represented by a leader. The K clusters are generated using a suitable Threshold value. Pro-LEADER aims also to maximize the f(V) function. Input parameter is the similarity score Threshold to consider an object O j as a new leader, and as outputs the algorithm returns the best partition of the training base D and the K leaders of the obtained clusters. The Pro-LEADER algorithm is fast, requiring only one pass through the data set D. Pro-LEADER algorithm is depicted in Figure 2.
Pro-CLARA relies on the sampling approach to handle large data sets . Instead of finding medoids for the entire data set, Pro-CLARA algorithm draws a small sample S of 40 + 2K sequences from the data set D. To generate an optimal set of medoids for this sample, Pro-CLARA applies the proposed PAM algorithm for protein sequence data sets, Pro-PAM algorithm,
The Pro-PAM algorithm proposed here, selects randomly K sequences from the data set as clusters, and then use the Smith Waterman algorithm to compute the total score TSih of each pair of selected sequence Ri and non selected sequence Oh. TSih is as follows
Where S jih is the differential score of each pair of non-selected object O h in D and selected object R i(i ∈ [1..K])with all non-selected objects O j in D. S jih is as follows
Pro-PAM selects the maximal TS ih , MaxTS ih . If MaxTS ih is positive, the corresponding non selected sequence O h will be selected, otherwise Smith Waterman algorithm is used to compare each protein O h of the data set with all medoids R i(i ∈ [1..K]), and to assign the sequence O h to the nearest cluster. Input parameter of Pro-PAM is the number of clusters, K, and as outputs the algorithm returns the best partition of the protein sequence base and the medoid of each cluster. Pro-PAM algorithm is depicted in Figure 3.
Pro-CLARA uses the optimal set of medoids R i (i ∈ [1..K])obtained by Pro-PAM and the Smith Waterman algorithm to compare each protein O h of the data set D with all medoids R i(i ∈ [1..K]), and to assign the sequence O h to the nearest cluster. In order to alleviate sampling bias, Pro-CLARA repeats the sampling and the clustering process a pre-defined number of times, q, and subsequently selects as the final clustering result the set of medoids with the maximal f (V). Input parameters of Pro-CLARA algorithm are the number of clusters, K, and of iterations, q, and as outputs the algorithm returns the best partition of the training base D and the K medoids of the obtained clusters. Pro-CLARA algorithm is detailed in Figure 4.
Pro-CLARANS algorithm starts from an arbitrary node C in the graph, C = [R 1, R 2,..., R k ], which represents an initial set of medoids. Pro-CLARANS randomly selects one of C neighbors, C*, which differs by only one sequence. If the total score of the selected neighbour, TS ih (Equation (5)), is higher than that of the current node TS' ih , Pro-CLARANS proceeds to this neighbor and continues the neighbor selection and comparison process. Otherwise, Pro-CLARANS randomly checks another neighbor until a better neighbor is found or the pre-determined maximal number of neighbours to check, Maxneighbor, has been reached. In this study Maxneighbor is defined as proposed by 
Where the maximal number of neighbours must be at least a threshold value 250 or obtained using the number of clusters K and the number of sequences, n, in the data set as: 1.25%*K*(n-K).
Pro-CLARANS algorithm aims to maximize the total score, TS ih
Pro-CLARANS algorithm use then the Smith Waterman algorithm to compute the similarity score of each sequence O h in D with each medoid R i(i ∈ [1..K])and to assign it to the nearest cluster. The algorithm repeats the clustering process a pre-defined number of times, q, and selects as the final clustering result the set of medoids with the maximal f (V). Input parameters of Pro-CLARANS algorithm are the number of clusters, K, and of iterations, q, and as outputs the algorithm returns the best partition of the training base D and the K medoids of the obtained clusters. Pro-CLARANS algorithm is detailed in Figure 5.
The proposed algorithms presented here have been implemented in Java package. All of these algorithms used the EMBOSS ftp://emboss.open-bio.org/pub/EMBOSS/ implementation of the Smith and Waterman local alignment algorithm for computing alignment score.
BLOSUM62 (Blocks Substitution Matrix)  was chosen to compute amino acids substitution scores . We chose the default penalties proposed by Smith and Waterman EMBOSS implementation as gap opening (P o ) and gap extension penalties (P e ) (P o = 10 and P e = 2).
To evaluate the Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS clustering algorithms, a large data set, Training data set, is used. We obtained from the training phase K clusters and each cluster is defined by a medoid (centroid or leader). The training phase results are used to cluster a different data set named, Test data set. Smith Waterman algorithm is used to compare each protein sequence on the test data set with all medoids R i(i ∈ [1..K])obtained from the training phase, and to assign each sequence to the nearest cluster. The predicted family group of each sequence is which of the nearest medoid.
The results obtained from the test phase are used to calculate the Sensitivity and the Specificity of each algorithm and to compare them with results of the published clustering tools, ProClust, TribeMCL and JACOP, tested on the same set: "Test data set".
Sensitivity specifies the probability of correctly predicting a classifier and it is defined as
and Specificity the probability that the provided prediction is correct and it is defined as
where TP (True Positives) is the number of correctly identified true homologues pairs, FN (False Negatives) is the number of not identified true homologues pairs and FP (False Positives) is the number of non-homologue pairs predicted to be homologue. A pair of sequences is considered truly homologous, if both are in the same family group.
Protein sequence data sets
To evaluate the performance of the proposed clustering algorithms Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS, and to compare their results with the available graph-based clustering tools ProClust and TribeMCL and the only available partitioning clustering tool JACOP, protein sequence families with known subfamilies/groups are considered. Protein sequences of HLA protein family have been collected from ftp://ftp.ebi.ac.uk/pub/databases/imgt/mhc/hla. From this set, we have randomly selected 893 sequences named DS1 and grouped into 12 classes. Protein sequences of Hydrolases protein family have been collected from http://www.brenda-enzymes.org/. Hydrolases protein family sequences are categorized into 8 classes according to their function and 3737 sequences, named DS2, have been considered from this family. From Globins protein family , sequences have been collected randomly from 8 different classes and 292 sequences, named DS3, have been selected from the data set IPR000971 in http://srs.ebi.ac.uk. Thus, totally 28 different classes containing sequences are considered as they have been classified by scientists/experts.
The data set considered has a total of 4922 sequences, named DS4, out of which 3500 sequences (practically 70% of the dataset DS4) are randomly for training, and 1422 for testing (practically 30% of the dataset DS4). The same method to obtain the training and the test sets are used on DS1, DS2 and DS3: randomly 70% of the set is selected for the training set and 30% for the test set [see Additional file 1].
Results and Discussion
Experiments are conducted on Intel Pentium4 processor based machine, having a clock frequency of 2.4 GHZ and 512 MB of RAM. Experimental results are obtained using default values as follows. In Pro-Kmeans, Pro-CLARA and Pro-CLARANS algorithms, the number of iterations q is fixed to 5  and the number of clusters K is fixed to 28. After a number of simulations, we find that the best clustering results are obtained when the parameter K = 28 . In Pro-LEADER algorithm, the Threshold value is fixed to 350 after a number of simulations .
Experimental results of Pro-Kmeans, Pro-LEADER, Pro-CLARA, Pro-CLARANS, ProClust, TribeMCL and JACOP algorithms on DS1, DS2, DS3 and DS4 are summarized in Table 2.
In our experiments, the use of partitioning clustering methods Pro-Kmeans, Pro-LEADER, Pro-CLARA, Pro-CLARANS and JACOP have improved sensitivity and specificity of hierarchical methods, ProClust and TribeMCL.
We have demonstrated the performance of the proposed Pro-Kmeans, Pro-LEADER, Pro-CLARA, Pro-CLARANS algorithms for the clustering of protein sequences using similarity.
Experiments show also that on the considered data sets DS1, DS2, DS3 and DS4, the higher probability of correctly predicting a classifier (Sensitivity) is obtained using JACOP method. Pro-CLARA method gives the higher probability that the provided prediction is correct (Specificity) although that, Pro-Kmeans, Pro-LEADER, and, Pro-CLARANS obtain also good results. The use of Pro-LEADER method on very large and heterogeneous set, DS4, is not very valuable. In fact the number of not identified true homologues pairs (False Negatives) is very important for that, the obtained Sensitivity is limited to 23.34.
Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS result confirm that those proposed partitioning methods are valuable, reliable tools for the automatic functional clustering of protein sequences. The use of these methods instead of alignment methods or the classic known clustering methods by biologists can improve the clustering sensitivity and specificity and reduce significantly the computational time. The proposed methods can be used by new biologists especially to cluster a large data set of proteins into meaningful clusters in order to detect their functions.
Similar protein sequences probably have similar biochemical function and three dimensional structures. If two sequences from different organisms are similar, they may have a common ancestor sequence and hence they are said to be homologous. Protein sequence clustering, using Pro-Kmeans, Pro-LEADER, Pro-CLARA and Pro-CLARANS methods helps in classifying a new sequence, retrieve a set of similar sequences for a given query sequence and predict the protein structure of an unknown sequence. We noticed that the classification of large protein sequence data sets using clustering techniques instead of only alignment methods reduce extremely the execution time and improve the efficiency of this important task in molecular biology.
Clote P, Backofen R: Computational Molecular Biology – An Introduction. 2000, John Wiley & Sons, Ltd
Mount DW: Bioinformatics – Sequence and Genome Analysis. 2002, Cold Spring Harbor Laboratory Press, New York
Smith TF, Waterman MS: Identification of common molecular subsequences. J Mol Biol. 1981, 147: 195-197. 10.1016/0022-2836(81)90087-5.
Needleman SB, Wunsch CD: A general method applicable to the search for similarities in the amino acid sequence of the proteins. J Mol Biol. 1970, 48: 443-453. 10.1016/0022-2836(70)90057-4.
Cabena P: Discovering Data Mining: From Concept to Implementation. 1998, Prentice Hall PTR, Upper Saddle River, NJ
Fayyad UM: Data mining and knowledge discovery: Making sense out of data. IEEE Expert. 1996, 11: 20-25. 10.1109/64.539013.
Tatusov R, Fedorova N, Jackson J, Jacobs A, Kiryutin B, Koonin E, Krylov D, Mazumder R, Mekhedov S, Nikolskaya A, Rao B, Smirnov S, Sverdlov A, Vasudevan S, Wolf Y, Yin J, Natale D: The COG database: an updated version includes eukaryotes. BMC Bioinformatics. 2003, 4: 41-10.1186/1471-2105-4-41.
Kaplan N, Sasson O, Inbar U, Friedlich M, Fromer M, Fleischer H, Portugaly E, Linial N, Linial M: ProtoNet 4.0: a hierarchical classification of one million protein sequences. Nucleic Acids Res. 2005, D216-8. 33 Database
Sasson O, Linial N, Linial M: The metric space of proteins-comparative study of clustering algorithms. Bioinformatics. 2002, 18 (Suppl 1): S14-21.
Herger A, Holm L: Picasso: generating a covering set of protein family profiles. Bioinformatics. 2001, 17 (3): 272-9. 10.1093/bioinformatics/17.3.272.
Kriventseva E, Servant F, Apweiler R: Improvements to CluSTr: the database of SWISS-PROT + TrEMBL protein clusters. Nucleic Acids Res. 2003, 31 (1): 388-9. 10.1093/nar/gkg035.
Enright A, Ouzounis C: GeneRAGE: a robust algorithm for sequence clustering and domain detection. Bioinformatics. 2000, 16 (5): 451-7. 10.1093/bioinformatics/16.5.451.
Krause A, Stoye J, Vingron M: Large scale hierarchical clustering of protein sequences. BMC Bioinformatics. 2005, 6: 15-10.1186/1471-2105-6-15.
Yona G, Linial N, Linial M: ProtoMap: automatic classification of protein sequences and hierarchy of protein families. Nucleic Acids Res. 2000, 28 (1): 49-55. 10.1093/nar/28.1.49.
Shi J, Malik J: Normalized cuts and image segmentation. Proceedings of the IEEE conference on Computer Vision Pattern Recognition. 1997, 731-737.
Wu Z, Leahy R: An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. PAMI. 1993, 11: 1101-1113.
Pipenbacher P, Schliep A, Schneckener S, Schönhuth A, Schomburg D, Schrader R: ProClust: improved clustering of protein sequences with an extended graph-based approach. Bioinformatics. 2002, 18 (Suppl 2): S182-91.
Bolten E, Schliep A, Schneckener S, Schomburg D, Schrader R: Clustering protein sequences-structure prediction by transitive homology. Bioinformatics. 2001, 17 (10): 935-41. 10.1093/bioinformatics/17.10.935.
Enright A, Van Dongen S, Ouzounis C: An efficient algorithm for large-scale detection of protein familes. Nucleic Acids Res. 2002, 30 (7): 1575-84. 10.1093/nar/30.7.1575.
Van Dongen S: Graph clustering by flow simulation. Phd Thesis. 2000, University of Utrecht, The Netherlands
Guralnik V, Karypis G: A scalable algorithm for clustering sequential data. SIGKDD Workshop on Bioinformatics, BIOKDD. 2001
Sperisen P, Pagni M: JACOP: a simple and robust method for the automated classification of protein sequences with modular architecture. BMC Bioinformatics. 2005, 6: 216-10.1186/1471-2105-6-216.
Kaufman L, Rousseeuw P: Finding Groups in Data: An Introduction to Cluster Analysis. 1990, John Wiley & Sons, Inc., New York
Pagni M, Ioannidis V, Cerutti L, Zahn-Zabal M, Jongeneel C, Hau J, Martin O, Kuznetsov D, Falquet L: MyHits: improvements to an interactive resource for analyzing protein sequences. Nucleic Acids Res. 2007, W433-37. 10.1093/nar/gkm352. 35 Web server
Anil KJ, Richard CD: Algorithms for Clustering Data. 1988, Prentice-Hall
Faber V: Clustering and the continuous k-means algorithm. Los Alamos Science. 1994, 22: 138-144.
Hartigan J, Wong M: Algorithm AS136: A k-means clustering algorithm. Applied Statistics. 1979, 28: 100-108. 10.2307/2346830.
Can F: Incremental clustering for dynamic information processing. ACM Trans Inf Syst. 1993, 11 (2): 143-164. 10.1145/130226.134466.
Spath H: Cluster analysis algorithms. 1980, Ellis Horwood, Chichester, UK
Ng R, Han J: Efficient and Effective Clustering Methods for Spatial Data Mining. Proceedings of International Conference on Very Large Data Bases. 1994, Santiago, Chile, 144-155.
Brenner SE, Chothia C, Hubbard TJ: Assessing sequence comparison methods with reliable structurally identified distant evolutionary relationships. Proc Natl Acad Sci USA. 1998, 95: 6073-6078. 10.1073/pnas.95.11.6073.
Pearson WR, Lipman DJ: Improved tools for biological sequence comparison. Proc Natl Acad Sci USA. 1988, 85: 2444-2448. 10.1073/pnas.85.8.2444.
Altschul S, Gish W, Miller W, Myers E, Lipman D: Basic local alignment search tool. J Mol Biol. 1990, 215: 403-410.
Essoussi N, Fayech S: A comparison of four pair-wise sequence alignment methods. Bioinformation. 2007, 2: 166-168.
Henikoff S, Henikoff J: Performance evaluation of amino acid substitution matrices. Proteins. 1993, 17: 49-61. 10.1002/prot.340170108.
Schneckener S: Positionsgenaues Alignment von Proteinse-quenzen. PhD Thesis. 1998, Universität zu köln
Cathy H: The Universal Protein Ressource (UniProt): an expanding universe of protein information. Nucleic Acids Res. 2006, 34: 87-191. 10.1093/nar/gkl485.
Dubes RC: How many clusters are best?. Pattern Recogn. 1987, 20 (6): 645-663. 10.1016/0031-3203(87)90034-3.
The authors declare that they have no competing interests.
SF incepted and directed the research and wrote the manuscript. NE and ML participated in the coordination and the direction of the whole study. All authors read and approved the final manuscript.