Using graph theory to analyze biological networks

The theory of complex networks plays an important role in a wide variety of disciplines, ranging from computer science, sociology, engineering and physics, to molecular and population biology. Within the fields of biology and medicine, potential applications of network analysis include for example drug target identification, determining a protein's or gene's function, designing effective strategies for treating various diseases or providing early diagnosis of disorders. Protein-protein interaction (PPI) networks, biochemical networks, transcriptional regulation networks, signal transduction or metabolic networks are the highlighted network categories in systems biology often sharing characteristics and properties.

Protein-protein interaction (PPI) networks[1] mainly hold information of how different proteins operate in coordination with others to enable the biological processes within the cell. Despite the fact that for the majority of proteins the complete sequence is already known, their molecular function is not yet fully determined. Predicting protein function is still a bottleneck in computational biology research and many experimental and computational techniques have been developed in order to infer protein function from interactions with other biomolecules. Large-scale and high-throughput techniques can detect proteins that interact within an organism. Among them, the most well-known are the pull down assays [2], tandem affinity purification (TAP) [3], yeast two-hybrid (Y2H) [4], mass spectrometry [5], microarrays [6] and phage display [7]. Some very well-known datasets that have been recently produced by employing the aforementioned techniques and that are widely used are the Tong [8], Krogan [9], DIP [10], MIPS [11], Gavin 2002 [5] and Gavin 2006 [12] datasets. Besides the various experimental methods, a variety of large biological databases that contain information concerning PPI data is already available and most of them are organism specific. Some well-known databases are the Yeast Proteome Database (YPD) [13], the Munich Information Center for Protein Sequences (MIPS) [14], the Molecular Interactions (MINT) database [15], the IntAct database [16], the Database of Interacting Proteins (DIP) [10], the Biomolecular Interaction Network Database (BIND) [17], the BioGRID database [18], the Human Protein Reference Database (HPRD) [19], the HPID [20] or the DroID [21] for Drosophila. Two additional well-documented services based on text mining analysis are the Stitch [22] and String [23] databases.

Regulatory networks (GRNs) contain information concerning the control of gene expression in cells. This process is modulated by many variables, such as transcription factors [24], their post-translational modifications or association with other biomolecules [25]. Usually, these networks use a directed graph representation in an effort to model the way that proteins and other biological molecules are involved in gene expression and try to imitate the series of events that take place in different stages of the process. They often exhibit specific motifs and patterns concerning their topology. Data collection, data integration and analysis techniques give now the possibility to study gene regulatory networks in a larger scale [26]. Protein-DNA interaction data is collected in databases like JASPAR [27], TRANSFAC [28, 29] or B-cell interactome (BCI) [30], while post-translational modification can be found in databases like Phospho.ELM [31], NetPhorest [32] or PHOSIDA [33].

Signal transduction networks often use multi-edged directed graphs to represent a series of interactions between different bioentities such as proteins, chemicals or macromolecules and to investigate how signal transmission is performed either from the outside to the inside of the cell, or within the cell. Environmental parameters change the homeostasis of the cell and, depending on the circumstances, different responses can be triggered. Similarly to GRNs, these networks also exhibit common patterns and motifs concerning their topology [34]. Databases that store information about signal transduction pathways are MiST [35], TRANSPATH [36], etc.

Metabolic and biochemical networks[37] are powerful tools for studying and modelling metabolism in various organisms. As metabolic pathways, we consider a series of chemical reactions occurring within a cell at different time points. The main role within a metabolic network is played by the enzymes, since they are the main determinants in catalyzing biochemical reactions. Often, enzymes are dependent on other cofactors such as vitamins for proper functioning. The collection of pathways, holding information about a series of biochemical events and the way they are correlated, is called a metabolic network. Modern sequencing techniques allow the reconstruction of the network of biochemical reactions in many organisms, from bacteria to human [38, 39]. Among the several databases holding information about biochemical networks some of the most popular are the Kyoto Encyclopedia of Genes and Genomes (KEGG) [40], EcoCyc [41], BioCyc [42] and metaTIGER [43]. Several methods have also been discovered to analyze the pathway structure of metabolic networks [44–48].

Many computer readable formats are available to describe biological networks. The Systems Biology Markup Language (SBML) [49] is an XML-like machine-readable language, that is able to represent models to be analyzed by a computer. SBML can represent metabolic networks, cell signaling pathways, regulatory networks, and many other kinds of systems [50]. Other file formats that can represent biological networks are the Proteomics Standards Initiative Interaction (PSI-MI)[51], Chemical Markup Language (CML) [52, 53] for chemicals or BioPAX[54] for pathways. Secondary formats that can also be used in similar ways are the Cell Markup Language[55] which is an XML-like machine-readable language mainly developed for the exchange of computer-based mathematical models or the Resource Description Framework, RDF which is a language for the representation of information about resources on the World Wide Web [56, 57].

After having given a short overview of how data can be produced either experimentally or retrieved from various databases and which formats are available for each type of network, we further emphasize on the computational analysis as defined in graph theory. We finally conclude by describing which properties of the ones discussed below characterize the various networks.

To introduce the basic concepts of graph theory, we give both the empirical and the mathematical description of graphs that represent networks as they are originally defined in the literature [58, 59].

Weighted graph

A weighted graph is defined as a graph G = (V, E) where V is a set of vertices and E is a set of edges between the vertices E = {(u, v) | u, v ∈ V} associated with it a weight function w: E→R, where R denotes the set of all real numbers. Most of the times, the weight w _ij of the edge between nodes i and j represents the relevance of the connection. Usually, a larger weight corresponds to higher reliability of a connection. Weighted graphs are currently the most widely used networks throughout the field of bioinformatics. As an example, relations whose importance varies are frequently assigned to biological data to capture the relevance of co-occurrences identified by text mining, sequence or structural similarities between proteins or co-expression of genes [23, 60].

Bipartite graph is an undirected graph G = (V, E) in which V can be partitioned into 2 sets V ₁ and V ₂ such that (u,v) ∈ E implies either u ∈ V ₁ and v ∈ V ₂ OR v ∈ V ₁ and u ∈ V ₂. Applications of this type of graph to visualization or modeling of biological networks range from representation of enzyme-reaction links in metabolic pathways to ontologies or ecological connections, as discussed in [61] or [62].

If G = (V, E) is a graph, then G ₁ = (V ₁ , E ₁ ) is called a subgraph or if V ₁ ⊆ V and E ₁ ⊆ E, where each edge in E ₁ is incident with vertices in V ₁ .

Examples and shapes describing the aforementioned graph types can be found in Figure 1. The most common data structures that are used to make these networks computer readable are adjacency matrices or adjacency lists. The following section provides a short mathematical description of these data structures.

The degree of a node in an undirected graph is the number of connections or edges the node has to other nodes and is defined as deg(i) = k(i) = |N(i)| where N(i) is the number of the neighbors of node i. If a network is directed, then each node has two different degrees, the in-degree deg _in (i) which is the number of incoming edges to node i, and the out-degree deg _out (i) which is the number of outgoing edges from node i. The total connectivity of a network is defined as where E is the number of edges and N the total number of nodes. The connectivity structure of biological networks is often informative with respect to reaction interplay and reversibility, compounds that structure the network, like in metabolism, or trophic relationships, like in food-web networks. Such connectivity profiles can be detected based on mixture models using software like MixNet [63].

The two main data structures used to store network graph representations are described below.

Adjacency list

Given a graph G = (V, E) the adjacency list representation consists of an array Adj of |E| elements where for each e∈E Adj(0, e) = i ∈V. Adjacency lists require space Θ (|V|+|E|) and are preferable for sparse graphs with a low density of connections. An example of how these data structures represent a graph is given in Figure 2.

Looking at different network properties can provide valuable insight into the internal organization of a biological network, the repartition of molecules among cellular processes, as well as the evolutionary constraints that have shaped an organism's protein, metabolic or regulatory network into a functional, feasible structure. In the following, we give a short description of the main properties that are commonly analyzed in networks.

The graph density shows how sparse or dense a graph is according to the number of connections per node set and is defined as . A sparse graph is a graph where |E| = O(|V| ^k ) and 2 > k > 1 or otherwise when |E| " |V|. Dense is a graph where |E| " |V| ² . It has been argued that biological networks are generally sparsely connected, as this confers an evolutionary advantage for preserving robustness. This has been observed for a series of organisms: the transcriptional regulatory networks of S. cerevisiae, E. coli, D. melanogaster all have connectivity densities lower than 0.1 [64].

In the mathematical field of graph theory, a complete graph is a simple graph in which every pair of distinct vertices is connected by a unique edge. The complete graph on n vertices has , n = |V| number of edges and it is a regular graph of degree |V| - 1 .

Graph Isomorphism

Let G ₁ = (V ₁ , E ₁ ) and G ₂ = (V ₂ , E ₂ ) be two undirected graphs. A function f: V ₁ -> V ₂ is called isomorphism if f is an edge-preserving bisection, such that for all a, b∈V ₁ , (a, b)∈E ₁ if and only if (f(a), f(b)) ∈ E ₂ . When such function exists, then G ₁ and G ₂ are called isomorphic. An example is shown in Figure 3.

A walk is a pass through a specific sequence of nodes (v ₁ , v ₂ ,..., v _L ) such that {(v ₁ , v ₂ ), (v ₂ , v ₃ ),..., (v _L-1 , v _L )} ⊆ E. A simple path is a walk with no repeated nodes. A cycle is a walk (v ₁ , v ₂ ,..., v _L ) where v ₁ = v _L with no other nodes repeated and L > 3, such that the last node is the same with the first one. A trail is a path where no edge can be repeated. A graph is called cyclic if it contains a cycle. In any other case it is called acyclic. All of the aforementioned can be found as an example in Figure 4. A complete graph is a graph in which every pair of nodes is adjacent. If (i, j) is an edge in a graph G between nodes i and j, we say that the vertex i is adjacent to the vertex j. An undirected graph is connected if one can get from any node to any other node by following a sequence of edges. A directed graph is strongly connected if there is a directed path from any node to any other node. This does not require an all-against combination. The distance δ(i, j) from i to j is the length of the shortest path from i to j in G. If no such path exists, then we set δ(i, j) = ∞ assuming that the nodes are so far between each other so they are not connected. Practically, for the distance δ(i, j) = ∞ we can use the maximum weight of the graph by adding one. Thus δ(i, j) = ∞ = (max _{d(i, j)} +1). To define the shortest path problem we can briefly say that it is the methodology of finding a path between two nodes such that the sum of the weights of its constituent edges is minimized. The average path length and the diameter of a graph G are defined to be the average and maximum value of δ(i, j) taken over all pairs of distinct nodes, i, j ∈V(G) which are connected by at least one path. More specifically, the average path length of a network is the average number of edges or connections between nodes, which must be crossed in the shortest path between any two nodes. It is calculated as where δ _min (i, j) is the minimum distance between nodes i and j. The diameter of a network is the longest shortest path within a network. The diameter is defined as . The most common algorithms for calculating the shortest paths are Dijkstra's greedy algorithm [65] and Floyd's dynamic algorithm [66]. Dijkstra's algorithm has running time complexity O(N ² ) where N is the number of vertices and returns the shortest path between a source vertex i and all other vertices in the network. Floyd's algorithm has running time complexity O(N ³ ) and requires an all-against-all matrix that contains the distances of every node in the network to every other node in the network.

A clique in an undirected graph G is a subgraph G' which is complete. An independent set in a graph is a subset of the vertices such that no pair of vertices is an edge in the graph. The size of a clique comes from the number of vertices it contains. A maximal clique is a clique that cannot be extended by including one more adjacent vertex, i.e. a clique which does not exist exclusively within the vertex set of a larger clique. A maximum clique is a clique of the largest possible size in a given graph. The clique problem refers to the problem of finding the largest clique in any graph G. This problem is NP-complete, and as such, many consider that it is unlikely that an efficient algorithm for finding the largest clique of a graph exists. Figure 3b shows a clique. A very famous method to find maximal cliques in a graph is the so-called Bron-Kerbosch algorithm [67]. Detection and analysis of these structures has found many biological applications: identifying groups of consistently co-expressed genes in microarray datasets, finding cis regulatory motifs or matching three-dimensional structures of molecules [68, 69]. Several tools have been developed for clique identification, like Clique Finder within the Arabidopsis Co-expression Tool server [70] or MIClique [68]. Bioconductor [71] provides a large collection of software for clique analysis.

Clustering Coefficient is the measurement that shows the tendency of a graph to be divided into clusters. A cluster is a subset of vertices that contains lots of edges connecting these vertices to each other. Assuming that i is a vertex with degree deg(i) = k in an undirected graph G and that there are e edges between the k neighbors of i in G, then the Local Clustering Coefficient of i in G is given by . Thus, C _i measures the ratio of the number of edges between the neighbors of i to the total possible number of such edges, which is k(k- 1)/ 2. It takes values as 0 ≤ C _i ≤ 1. The average Clustering Coefficient of the whole network C _average is given by where N=|V| is the number of vertices. The closer the local clustering coefficient is to 1, the more likely it is for the network to form clusters. Obviously, a clique would come with local clustering coefficient equal to 1. An example showing how local clustering coefficient is calculated is shown in Figure 5.

Biological networks have a significantly higher average clustering coefficient compared to random networks, which proves their modular nature. Indeed, many cellular processes are governed by subsets of biomolecules that form an interaction module. Since cellular processes are linked, the modules tend to be linked as well, but the linking molecules are often few, such that the module overlap is quite low [72, 73].

Centralization is the measurement that shows whether a network has a star-like topology or whether the nodes of the network have on average the same connectivity. The closer the centralization is to 1, the more likely is the network to have a star-like topology. The closer to 0, the more likely it is that the nodes of the network have on average the same connectivity (for example a square, where every node is connected with 2 neighbors). It is calculated as

Network Motifs represent patterns in complex networks occurring significantly more often than in randomized networks [74]. They consist of subgraphs of local interconnections between network elements. A motif is a small connected graph G'. A match G' of a motif in graph G is a graph G'' which is isomorphic to G' and a subgraph of G. Signal transduction and gene regulatory networks tend to be described by various motifs [72, 75]. Although motif determination gives lots of information concerning the properties and the characteristics of a network, it does not necessarily reveal evidence about its function and the function of its components [76]. However, some motifs have been found to be associated with optimized biological functions, like in the case of positive and negative feedback loops, oscillators or bifans [73]. Figure 6 shows the most common motifs that are found in various networks.

This section shows how nodes can be ranked or sorted according to their properties, depending on the question asked. In biological networks, it is important for example to detect central nodes or intermediate nodes that affect the topology of the network, depending of course on the biological question. Such a question would be to find the molecules in a biological pathway that are not necessarily central but have a crucial biological role in signal transduction or in PPI networks, to detect such nodes that interact with many other proteins or find molecules that are crucial for stimulating the expression of genes.

Degree Centrality shows that an important node is involved in a large number of interactions. For a node i, the degree centrality is calculated as C _d (i) = deg(i). For directed graphs, each node is obviously characterized by two degree centralities. These are C _{d in} (i) = deg _in (i) and C _{d out} (i) = deg _out (i). Nodes with very high degree centrality are called hubs since they are connected to many neighbors (see Figure 5). Scale-free networks tend to contain hubs. The removal of such central nodes has great impact on the topology of the network. It has been shown that biological networks tend to be robust against random perturbations, but disruption of hubs often leads to system failure [77, 78].

Closeness Centrality indicates important nodes that can communicate quickly with other nodes of the network. Let G = (V, E) be an undirected graph. Then, the centrality is defined as where dist(i, j) denotes the distance or else the shortest path p between the nodes i and j. An example is shown in Figure 7. Closeness centrality has been used to identify the top central metabolites in genome-based large-scale metabolic networks [79], to compare unicellular and multicellular eukarya, to rank pathways and obtain a perspective on the evolution of metabolic organization [80]. A decrease in closeness centrality of components has been observed as a consequence of increased distance between pathways throughout evolution [80]. It has been chosen as the best centrality measure that can be used extract the metabolic core of a network [81].

Betweenness Centrality shows that nodes which are intermediate between neighbors rank higher. Without these nodes, there would be no way for two neighbors to communicate with each other. Thus, betweenness centrality shows important nodes that lie on a high proportion of paths between other nodes in the network. For distinct nodes i, j, w ∈ V(G), let σ _ij be the total number of shortest paths between i and j and σ _ij (w) be the number of shortest paths from i to j that pass through w. Moreover, for w ∈ V(G), let V (i) denote the set of all ordered pairs, (i, j) in V(G) × V(G) such that i, j, w are all distinct. Then, the Betweenness Centrality is calculated as . An example is shown in Figure 7. Proteins with high betweenness centralities have been termed "bottlenecks", for their role as key connector proteins with essential functional and dynamic properties [73], for example metabolites that control the flux between two big metabolic modules. Calculation of this centrality measure is discussed in [82] and [83] and their properties within the PPI network of yeast are detailed in [84].

Eigenvector Centrality ranks higher the nodes that are connected to important neighbors. Let G = (V, E) be an undirected graph and A the adjacency matrix of network G. The eigenvector centrality is the eigenvector C _eiv- of the largest eigenvalue λ _max in absolute value such that λC _eiv = AC _eiv . Formally, if A is the adjacency matrix of a network G with V(G) = {v ₁,..., v _n }, and , then the eigenvector centrality C _eiv (v _i ) of the node v_i is given by the i ^th coordinate x _i of a normalized eigenvector that satisfies the condition Ax=ρ(A)x. Such algorithms can be used for efficient page ranking on the web. In biology this centrality measurement has been used, among others, to identify synthetic genetic interactions [85], gene-disease associations [86] or network hubs [77].

Eccentricity Centrality is the measure that shows how easily accessible a node is from other nodes. Let G = (V, E) be an undirected graph. The eccentricity centrality is calculated as where dist(i, j) is the shortest path between nodes i and j. The eccentricity C _ecc of a vertex V is the greatest distance between v and any other vertex. An example is shown in Figure 8. In biological networks, proteins or other bioentities with high eccentricity are easily functionally reachable by other components of the network, and thus can readily perceive changes in concentration of other enzymes or molecules they are linked to. In contrast, those proteins that have lower eccentricities will often play a marginal functional role in the system [87].

Subgraph Centrality is the measure that ranks nodes according to the number of subgraphs of the overall network in which the node participates, with more weight given to small subgraphs. Let G = (V, E) be an undirected graph and A the adjacency matrix of network G. The subgraph centrality of a node is calculated as . Subgraph centrality analysis has been used to study essential proteins in proteomic maps [77], to compute the degree of folding of protein chains [88], to understand the molecular structure of drug-like compounds [89] or to zoom into the topological environment of certain nodes in PPI networks of several organisms [90].

Matching Index is the measure that shows how similar two nodes are within the network. Two vertices that are functionally similar do not always have to be connected. The matching index M _ij measures the "similarity" of two nodes and is based on the number of common neighbors shared by nodes i and j. It is calculated as

or . An example is shown in Figure 9. The matching index is often used to cluster different components of a biological network according to some property. For instance, it has been used to describe spatial growth in brain networks during development [91] or to predict the connectivity of primate cortical networks [92].

Further centrality measurements and their application to the study of PPIs in yeast are introduced in [85]. A discussion about how centrality correlates with lethality in biological networks can be found in [93]. The coupling between centrality and essentiality has also been investigated in several eukaryotic protein networks [94]. It is very often the case that studies of a particular network involve the analysis and comparison of several centrality measures, for instance to study pleiotropy in human genetic diseases [87], to compare PPI and transcriptional regulation networks [95] or to test hub essentiality [77]. Tools that have implemented functionality for exploring the different types of centralities previously mentioned in biological networks and not only are CentiBiN [96], Visone [97], Pajek [98], VisANT [99]. In most of the cases, however, only a limited selection of centrality measures is available.

The topology of the network often reveals information about its biological significance. Often, networks follow patterns and rules and have a specific topology that allows scientists to go through a deeper investigation towards knowledge extraction.

Scale-free or otherwise real world networks describe natural networks like online communities (i.e Facebook) where the nodes are the people and the edges the connection between them, or networks such as the World Wide Web (www) where the nodes are individual web pages and the links are hyperlinks. Many biological networks also have scale-free properties, with nodes representing bioentities and edges the interactions between them (like proteins that interact physically or metabolites that take part in the same reaction) [73, 93, 100]. Assuming that k is the number of links originating from a given node and P(k) the probability that the degree of a randomly chosen vertex equals k, a scale-free network exhibits a power law distribution P(k) ~ k ^-γ where γ denotes the degree exponent. A scale-free network can be constructed by progressively adding nodes to an existing network and introducing links to existing nodes with preferential attachment so that the probability of linking to a given node i is proportional to the number of existing links k _i that the node has. Thus the connectivity of one node i to any other node j should approximately follow the rule: .

The degree distribution P(k) has become one of the most prominent characteristics in network topology. In terms of numerical estimation, a more reliable property, very similar to the previous, is the cumulative degree distribution P _c (k). For a power law distribution P(k) ~ k ^-γ the cumulative degree distribution is of the form P(k) ~ k ^(-γ-1) and describes the probability of a random chosen node in the network to have a degree greater than k. Even though lots of research has been done on power law analysis in biological networks, it is still not an established approach widely accepted by the scientific community [101].

To visually represent the properties of the network we usually rank the vertices according to their degree and then plot the degree versus the rank of each vertex. Another representation is to create a histogram by plotting the vertices of the graph sorted according to their degree using a logarithmic scale. A third and very popular representation is to plot the degrees of the nodes sorted versus either their degree distribution P(k) or their cumulative degree distribution P _c (k). An interesting analysis of most of these properties in various PPI, metabolic or transcriptional networks of several organisms (S. cerevisiae, H. pylori, C. elegans) can be found in [100].

A network is called assortative if the vertices with higher degree have the tendency to connect with other vertices that also have high degree of connectivity; one such category is social networks [102]. If the vertices with higher degree have the tendency to connect with other vertices with low degree then the network is called disassortative. This is characteristic to most molecular interaction networks, where hubs have the tendency to link to nodes with fewer interaction partners rather than to other hubs [103, 104]. Newman [102] discusses this property for protein interaction networks, neural networks and food webs.

To correlate the degrees of two nodes i and j we use a joint probability distribution P(k _i , k _j ) = P(k _i )P(k _j ). A more straightforward way is to use the Pearson's Correlation Coefficient (PCC), which quantifies the correlation or linear dependence between two variables (in this case, the degrees of two nodes). In other words, it measures to which extent one variable increases/decreases as the other increases. PCC (r-value) between two nodes is defined as the covariance of the two nodes divided by the product of their standard deviations. For the entire network, the assortativity coefficient is the measure of how assortative or disassortative a network is overall. If M is the number of edges, and x _i and y _i the degrees of the vertices at either ends of edge i, the assortativity coefficient r is calculated as follows [102].:

This is equivalent to the Pearson correlation coefficient of the degrees at either ends of an edge. The range of the r- values is between +1 and -1, r < 0 corresponding to a disassortative network whereas r > 0 to an assortative one. Another way to correlate degrees is to calculate the average neighbor degree. For each vertex i, the average degree of its neighbor is calculated as . The values are then averaged for all vertices with the same degree k, showing the average neighbor degree k _nn (k).

Several topological models have been built to describe the global structure of a network, as introduced below.

Cluster analysis[110] aims at classifying a set of observations into two or more mutually exclusive unknown groups based on combinations of variables. Thus, cluster analysis is usually presented in the context of unsupervised classification [111]. It can be applied to a wide range of biological study cases, such as microarray, sequence and phylogenetic analysis [112]. The purpose of clustering is to group different objects together by observing common properties of elements in a system. In biological networks, this can help identify similar biological entities, like proteins that are homologous in different organisms or that belong to the same complex and genes that are co-expressed [113, 114].

It is generally difficult to predict behavior and properties based on observations of behaviors or properties of other elements in the same system, therefore various approaches for cluster analysis emerge. Clustering algorithms may be Exclusive, Overlapping, Hierarchical or Probabilistic. In the first case, data are grouped in an exclusive way, so that a certain element can be assigned to only one group (exclusively). On the other hand, the overlapping clustering uses fuzzy sets to cluster data, so that each point may belong to two or more clusters with different degrees of membership. A hierarchical clustering algorithm organizes data in hierarchies and is based on the union between the two nearest clusters; it is commonly used for microarray and sequence analysis [115]. A more analytical categorization of clustering algorithms can be found at [110, 116].

An important component of a clustering algorithm is the distance measure between data points. If all the components of the data instance vectors have the same physical units, it is then possible that the simple Euclidean distance metric is sufficient to successfully group similar data instances. One example is to cluster cities on a map, since in this case Euclidean distance represents real natural distances. However, for higher dimensional data the Euclidean distance can sometimes be misleading. In that case, a popular measure is the Minkowski metric and is calculated as where D is the dimensionality of the data. The Euclidean can be calculated if we set p = 2, while Manhattan metric has p = 1. There are no general theoretical guidelines for selecting a measure for a given application.

Hierarchical clustering is a method of cluster analysis which seeks to build a hierarchy of clusters. There are two different strategies to organize data. These are the agglomerative and the divisive: Agglomerative: It is a "bottom-up" approach. Each observation starts in its own cluster, and pairs of clusters are merged as one moves up the hierarchy. Divisive: This is a "top-down" approach. In this case, all of the observations start by forming one cluster, and then split recursively as one moves down the hierarchy. Some of the most common tree based clustering algorithms that organize data in hierarchies are the Unweighted Pair Group Method with Arithmetic Mean (UPGMA) [117, 118], Neighbor Joining [112, 119] and Hierarchical Clustering [120, 121], all of which represent their clusters as tree structures. The results of hierarchical clustering are usually presented in a dendrogram. Figure 10 shows an example of how genes can be clustered.

Let n _r be the number of clusters and x _ri is the i th object in cluster r and cluster r is formed from clusters p and q. In the following, we describe the different methods used to calculate distances between clusters in hierarchical clustering.

Single linkage calculates the smallest distance between objects in the two clusters to merge them: d(r, s) = min(dist(x _ri , x _sj )), i ∈ (i,..., n _r ), j ∈ (1,....n _s ).

Complete linkage calculates the largest distance between objects in the two clusters to merge them: d(r, s) = max(dist(x _ri , x _sj )), i ∈ (i,..., n _r ), j ∈ (1,....n _s ).

Average linkage uses the average distance between all pairs of objects in any two clusters: . This algorithm is also known as Unweighted Pair Group Method with Arithmetic Mean ( UPGMA)[117, 118].

Centroid linkage finds the Euclidean distance between the centroids of the two clusters: , is the Euclidean distance.

Median linkage uses the Euclidean distance between weighted centroids of the two clusters, are weighted centroids for the clusters r and s. If cluster r was created by combining clusters p and q, x_r is defined recursively as .

Single or complete linkages are the fastest of the linkage methods. However, single linkage tends to produce stringy clusters, which is not always preferable. The centroid or average linkage produce better results regarding the accordance between the produced clusters and the structure present in the data. These methods require much more computations. Average linkage and complete linkage may be the preferred methods for microarray data analysis [115].

Ward's linkage finds the incremental sum of squares; that is, the increase in the total within-cluster sum of squares as a result of joining two clusters. The within-cluster sum of squares is defined as the sum of the squares of the distances between all objects in the cluster and the centroid of the cluster. The sum of squares measure is equivalent to the following distance measure ,

where || ||₂ is the Euclidean distance and are the centroids of clusters r and s and n _r and n _s are the number of elements in clusters r and s.

Weighted average linkage uses a recursive definition for the distance between two clusters. If cluster r was created by combining clusters p and q, the distance between r and another cluster s is defined as the average of the distance between p and s and the distance between q and s: .

Neighbor Joining[112, 119] was initially proposed for finding pairs of operational taxonomic units (OTUs) that minimize the total branch length at each stage of clustering of OTUs starting with a star-like tree. The branch lengths as well as the topology of a parsimonious tree can quickly be obtained by using this method [112].

Known platforms that already share the tree-based algorithms described above are the Hierarchical Clustering Explorer (HCE) [122, 123], MEGA [124–127] or TM4 [128]. A recent review article shows which file formats, visualization techniques and algorithms can be used for tree analysis [129].

Another category of clustering algorithms tries to cluster data in separate groups by identifying common properties that the nodes of a network share. Different strategies exist, like for example trying to find dense areas in a graph or areas where message exchange between nodes is easier or to identify strongly connected components or clique-like areas etc. Many of such algorithms have been used in different case studies like for example to identify protein families [130], to detect protein complexes in PPI networks [131, 132], or for finding patterns and motifs in a sequence [133]. Even though many more exist, some of the most famous algorithms are given below.

Markov Clustering[134] (MCL) algorithm is a fast and scalable unsupervised clustering algorithm based on simulation of stochastic flow in graphs. The MCL algorithm can detect cluster structures in graphs by a mathematical bootstrapping procedure which takes into account the connectivity properties of the underlying network. The process deterministically computes the probabilities of random walks through a graph by alternating two operations: expansion, and inflation of the underlying matrix. The principle behind it is that random walks on a graph are likely to get locked within dense subgraphs rather than move between dense subgraphs via sparse connections. In other words, higher length paths are more often encountered between nodes in the same cluster than between nodes within different clusters, such that the probabilities between nodes in the same complex will typically be higher in expanded matrices. Clusters are identified by alternating expansion and inflation until the graph is partitioned into subsets so that there are no longer paths between these subsets [135, 136].

k -Means[137] is a method of cluster analysis which aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean. K-means and its modifications are widely used for gene expression data analysis [138]. It is a supervised method and users need to predefine the number of clusters. Its complexity is O(nlk) where k is the number of clusters, n the size of the dataset and l the loops of the algorithm. The k-means algorithm is one of the simplest and fastest clustering algorithms. However, it has a major drawback: the results of the k-means algorithm may change in successive runs because the initial clusters are chosen randomly.

Affinity Propagation[139] takes as input measures of similarity between pairs of data points and simultaneously considers all data points as potential candidates. Real-valued messages are exchanged between data points until a high-quality set of exemplars and corresponding clusters gradually emerges.

Restricted Neighborhood Search Cluster Algorithm[140]: It tries to find low cost clustering by composing first an initial random clustering. Later it iteratively moves one node from one cluster to another in a random way trying to improve the clustering cost.

Spectral clustering[141]: This algorithm tries to find clusters in the graph such that the nodes within a cluster are connected with highly-similar edges and the connections between such areas are weak, constituted by edges with low similarity. The aim is to identify these tightly coupled clusters, and cut the inter-cluster edges. Figure 11 shows an example of protein complex prediction from PPI yeast dataset [12].

Despite the great variety of clustering techniques, many articles directly compare the various clustering methodologies like [135] and [142]. Very often we encounter articles that compare similar algorithms using different datasets and come to very diverse conclusions and results i.e [142, 143].

Concerning the visualization of networks, the availability of clustering techniques and their complex configuration/combination, today to a large extent, there is a lack of visualization platforms or tools that are able to integrate a variety of more advanced algorithms and the development implementation of such implementations emerges [144]. Platforms that share clustering algorithms are the Network Analysis Tool (NEAT) [145] and jClust [146] but they are still poor in the variety of methods they offer. Software like ArrayCluster [147] and MCODE [60] is often used in analysis of gene expression profiles and coexpression detection. Many visualization tools [144] such as Medusa [148], Cytoscape [149], Pajek [98] and many others [144] visualize networks in both 2D and 3D, but very few of them like Arena3D [150] try to bridge the gap between clustering analysis and visualization.

Protein-protein interaction (PPI) networks[1] are very diverse and it is difficult to come to general conclusions about their properties, mainly because data are generated from different sources both computationally and experimentally as described in a previous section. In most of the cases, PPI networks follow the laws of scale-free networks [93]. In such networks there are always proteins with higher degree of connectivity that appear to be of higher biological significance. Such proteins are the most important for the survival of the cell [93]. Large-scale maps of protein interaction networks have been constructed recently using high-throughput approaches to identify protein interactions [151–155]. It has been shown that these networks are highly dynamic, both during common cellular processes and on the evolutionary scale [109]. Further details on PPI network construction and analysis are given in [156].

Regulatory networks (GRNs) are usually sparsely connected. More specifically, the average number of upstream-regulators per gene is less than two [64]. Theoretical results show that the selection for robust gene networks will form minimal complexes even more sparsely connected [64], thus a fundamental design constraint could shape the evolution of gene network complexity. Network maps have been constructed for the transcriptional regulatory networks of E. coli and S. cerevisiae and are maintained in databases [26, 157, 158]. They are very sensitive and flexible to evolution [159] since their dynamics changes continuously over time and since transcription factors evolve faster than their target genes [160]. The number of regulators N _reg grows faster than the number of genes N _tot they regulate and it has been shown that for prokaryotes and for eukaryotes, where N is the network size [161, 162]. Mostly they follow the power-law distributions and thus belong to the scale-free network category, even though some of them, like the transcriptional regulatory networks of E. coli and S. cerevisiae have been shown to possess mixed scale-free and exponential properties [75].

Signal transduction networks are characterized by several patterns and motifs like self-sustaining feedback loops. These patterns appear at every time point during the signal transduction in the network and they reveal information about the topology of the network, therefore are important for biological functionality [163]. The nodes with the highest centralities in such networks correspond to domains involved in signal transduction and cell-cell contacts [164]. Signal transduction networks are sparse and they follow the scale-free properties. In E. coli and S. sereviase, the degree distribution is P(k) = k ^-γ, γ ≈ 2 and most of the molecules are involved into few interactions and only few of them have higher connectivity [8, 165].

Metabolic and biochemical networks are scale-free networks indicating a small-world structure considering the topology of the network based on its metabolites [166, 167], where all of the nodes in such networks are connected through a short path to any other. One example is presented in [167] for E. coli. The probability that a substrate participates as input in k metabolic reactions follows the power-law distribution P(k) = k ^-γin , γ _in ≈ 2.2 whereas the probability of a substrate to be produced by k metabolic reactions equals similarly to P(k) = k ^-γout , γ _out ≈ 2.2. Metabolic networks are extremely heterogeneous and vary from organism to organism. The scale-free structure remains robust even after removal of some central nodes [166] and despite the fact that the architecture of the metabolic networks rests on highly connected substrates [167]. A characteristic feature of these networks is the apparent conservation of network diameter even in distantly related organisms [167]. It has been shown that metabolic networks can form hierarchical structures [168, 169] where specific patterns and motifs are overrepresented. Methods to detect such motifs have been applied on network pathways analysis [44, 45, 47, 48], one example being flux mode analysis [48].

The mathematical discipline which underpins the study of complex networks in biological and other applications is graph theory. It has been successfully applied to the study of biological network topology, from the global perspective of their scale-free, small world, hierarchical nature, to the zoomed-in view of interaction motifs, clusters and modules and the specific interactions between different biomolecules. The structure of biological networks proves to be far away from randomness but rather linked to function. Furthermore, the power of network topology analysis is limited, as it provides a static perspective of what is otherwise a highly dynamic system, such that additional tools should be combined with this approach in order to obtain a deeper understanding of cellular processes.

The complexity of biological networks increases as data are accumulated. The inherent variability of biological data, data inaccuracy and noise, the overload of information and the need to study the dynamics and network topology over time, are currently the bottlenecks in systems biology. Improved techniques for integration of data arising from different sources, as well as for visualization, will be crucial for understanding the functionality of complex networks. Moreover, new mathematical developments in the field and discovery of new areas of applications should be pursued in the near future.