The UAHDataScienceUC package provides a robust collection of clustering algorithms implemented in R. This package, developed at the Universidad de Alcalá de Henares, offers both traditional and advanced clustering methods, making it a valuable tool for data scientists and researchers. In this vignette, we’ll explore the various clustering algorithms available in the package and learn how to use them effectively.
The package implements several clustering algorithms, each with its own strengths and use cases:
It partitions n observations into k clusters where each observation belongs to the cluster with the nearest mean.
# Perform k-means clustering
result <- kmeans_(data, centers = 3, max_iterations = 10)
# Plot results
plot(data, col = result$cluster, pch = 20)
points(result$centers, col = 1:3, pch = 8, cex = 2)
This algorithm builds a hierarchy of clusters from bottom-up, starting with individual observations and progressively merging them into clusters.
# Perform hierarchical clustering
result <- agglomerative_clustering(
data,
proximity = "single",
distance_method = "euclidean",
learn = TRUE
)## ________________________________________________________________________________## EXPLANATION:## ## The Agglomerative Hierarchical Clustering algorithm defines a clustering hierarc## hy for a dataset following a `n` step process, which repeats until a single clus## ter remains:## ## 1. Initially, each object is assigned to its own cluster. The matrix of dist## ances between clusters is computed.## 2. The two clusters with closest proximity will be joined together and the p## roximity matrix updated. This is done according to the specified proximity.## This step is repeated until a single cluster remains.## ## The definitions of proximity considered by this function are:## ## 1. `single`. Defines the proximity between two clusters as the distance betw## een the closest objects among the two clusters. It produces clusters where e## ach object is closest to at least one other object in the same cluster. It i## s known as SLINK, single-link or minimum-link.## 2. `complete`. Defines the proximity between two clusters as the distance be## tween the furthest objects among the two clusters. It is known as CLINK, com## plete-link or maximum-link.## 3. `average`. Defines the proximity between two clusters as the average dist## ance between every pair of objects, one from each cluster. It is also known## as UPGMA or average-link.## ## Euclidean Distance Formula:## d(x,y) = √(∑1ⁿ (xᵢ - yᵢ)²)## Press [enter] to continue## ## ________________________________________________________________________________## STEP 1:## ## Initially, each object is assigned to its own cluster. This leaves us with the f## ollowing clusters:## CLUSTER #-1 (size: 1)
## x y
## 1 -1.578117 -1.292868
## CLUSTER #-2 (size: 1)
## x y
## 2 0.7027994 1.193823
## CLUSTER #-3 (size: 1)
## x y
## 3 0.7854535 1.191428
## CLUSTER #-4 (size: 1)
## x y
## 4 0.6757613 -0.04002442
## CLUSTER #-5 (size: 1)
## x y
## 5 0.8484305 0.2230609
## CLUSTER #-6 (size: 1)
## x y
## 6 0.515489 0.3014147
## CLUSTER #-7 (size: 1)
## x y
## 7 0.9187371 1.347416
## CLUSTER #-8 (size: 1)
## x y
## 8 0.9062708 -0.01894187
## CLUSTER #-9 (size: 1)
## x y
## 9 0.7017478 1.37873
## CLUSTER #-10 (size: 1)
## x y
## 10 0.4289005 1.109321## ## Press [enter] to continue## ## The matrix of distances between clusters is computed:## Distances:
## -1 -2 -3 -4 -5 -6 -7 -8 -9
## -2 3.374
## -3 3.429 0.083
## -4 2.579 1.234 1.236
## -5 2.861 0.982 0.970 0.315
## -6 2.632 0.912 0.930 0.377 0.342
## -7 3.634 0.265 0.205 1.409 1.127 1.121
## -8 2.792 1.230 1.216 0.231 0.249 0.505 1.366
## -9 3.512 0.185 0.205 1.419 1.165 1.093 0.219 1.413
## -10 3.130 0.287 0.366 1.176 0.981 0.813 0.545 1.225 0.383## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 2:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-2 (size: 1)
## CLUSTER #-3 (size: 1)
## Proximity:
## [1] 0.08268877## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #1 (size: 2) [CLUSTER #-2 + CLUSTER #-3]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 -4 -5 -6 -7 -8 -9 -10
## -4 2.579
## -5 2.861 0.315
## -6 2.632 0.377 0.342
## -7 3.634 1.409 1.127 1.121
## -8 2.792 0.231 0.249 0.505 1.366
## -9 3.512 1.419 1.165 1.093 0.219 1.413
## -10 3.130 1.176 0.981 0.813 0.545 1.225 0.383
## 1 3.374 1.234 0.970 0.912 0.205 1.216 0.185 0.287## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 3:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-9 (size: 1)
## CLUSTER #1 (size: 2)
## Proximity:
## [1] 0.1849095## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #2 (size: 3) [CLUSTER #-9 + CLUSTER #1]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 -4 -5 -6 -7 -8 -10
## -4 2.579
## -5 2.861 0.315
## -6 2.632 0.377 0.342
## -7 3.634 1.409 1.127 1.121
## -8 2.792 0.231 0.249 0.505 1.366
## -10 3.130 1.176 0.981 0.813 0.545 1.225
## 2 3.374 1.234 0.970 0.912 0.205 1.216 0.287## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 4:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-7 (size: 1)
## CLUSTER #2 (size: 3)
## Proximity:
## [1] 0.2051747## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #3 (size: 4) [CLUSTER #-7 + CLUSTER #2]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 -4 -5 -6 -8 -10
## -4 2.579
## -5 2.861 0.315
## -6 2.632 0.377 0.342
## -8 2.792 0.231 0.249 0.505
## -10 3.130 1.176 0.981 0.813 1.225
## 3 3.374 1.234 0.970 0.912 1.216 0.287## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 5:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-4 (size: 1)
## CLUSTER #-8 (size: 1)
## Proximity:
## [1] 0.2314716## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #4 (size: 2) [CLUSTER #-4 + CLUSTER #-8]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 -5 -6 -10 3
## -5 2.861
## -6 2.632 0.342
## -10 3.130 0.981 0.813
## 3 3.374 0.970 0.912 0.287
## 4 2.579 0.249 0.377 1.176 1.216## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 6:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-5 (size: 1)
## CLUSTER #4 (size: 2)
## Proximity:
## [1] 0.2488189## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #5 (size: 3) [CLUSTER #-5 + CLUSTER #4]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 -6 -10 3
## -6 2.632
## -10 3.130 0.813
## 3 3.374 0.912 0.287
## 5 2.579 0.342 0.981 0.970## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 7:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-10 (size: 1)
## CLUSTER #3 (size: 4)
## Proximity:
## [1] 0.2866377## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #6 (size: 5) [CLUSTER #-10 + CLUSTER #3]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 -6 5
## -6 2.632
## 5 2.579 0.342
## 6 3.130 0.813 0.970## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 8:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-6 (size: 1)
## CLUSTER #5 (size: 3)
## Proximity:
## [1] 0.342037## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #7 (size: 4) [CLUSTER #-6 + CLUSTER #5]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1 6
## 6 3.130
## 7 2.579 0.813## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 9:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #6 (size: 5)
## CLUSTER #7 (size: 4)
## Proximity:
## [1] 0.8125336## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #8 (size: 9) [CLUSTER #6 + CLUSTER #7]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## -1
## 8 2.579## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 10:## ## The two clusters with closest proximity are identified:## Clusters:
## CLUSTER #-1 (size: 1)
## CLUSTER #8 (size: 9)
## Proximity:
## [1] 2.578678## ## Press [enter] to continue## ## They are merged into a new cluster:## CLUSTER #9 (size: 10) [CLUSTER #-1 + CLUSTER #8]## ## Press [enter] to continue## ## The proximity matrix is updated. To do so the rows/columns of the merged cluster## s are removed, and the rows/columns of the new cluster are added:## Distances:
## dist(0)## ## Press [enter] to continue## ## ________________________________________________________________________________## RESULTS:## ## Since all clusters have been merged together, the final clustering hierarchy is:## (Check the plot for the dendrogram representation of the hierarchy)## ## Press [enter] to continue## ## ________________________________________________________________________________
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is particularly effective at finding clusters of arbitrary shape and identifying noise points.
## ________________________________________________________________________________## EXPLANATION:## ## The data given by data is clustered by the DBSCAN method, which aims to partitio## n the points into clusters such that the points in a cluster are close to each o## ther and the points in different clusters are far away from each other. The clus## ters are defined as dense regions of points separated by regions of low density.## ## The DBSCAN method follows a 2 step process:## ## 1. For each point, the neighborhood of radius epsilon is computed. If the ne## ighborhood contains at least min_pts points, then the point is considered a## core point. Otherwise, the point is considered an outlier.## 2. For each core point, if the core point is not already assigned to a clust## er, a new cluster is created and the core point is assigned to it. Then, the## neighborhood of the core point is explored. If a point in the neighborhood## is a core point, then the neighborhood of that point is also explored. This## process is repeated until all points in the neighborhood have been explored.## If a point in the neighborhood is not already assigned to a cluster, then i## t is assigned to the cluster of the core point.## ## Whatever points are not assigned to a cluster are considered outliers.## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 1:## ## The pairwise distances between observations are precomputed in order to later de## termine which of them are core observations. The distance matrix is:## Distances:
## 1 2 3 4 5 6 7 8 9 10
## 1 0.000 3.374 3.429 2.579 2.861 2.632 3.634 2.792 3.512 3.130
## 2 3.374 0.000 0.083 1.234 0.982 0.912 0.265 1.230 0.185 0.287
## 3 3.429 0.083 0.000 1.236 0.970 0.930 0.205 1.216 0.205 0.366
## 4 2.579 1.234 1.236 0.000 0.315 0.377 1.409 0.231 1.419 1.176
## 5 2.861 0.982 0.970 0.315 0.000 0.342 1.127 0.249 1.165 0.981
## 6 2.632 0.912 0.930 0.377 0.342 0.000 1.121 0.505 1.093 0.813
## 7 3.634 0.265 0.205 1.409 1.127 1.121 0.000 1.366 0.219 0.545
## 8 2.792 1.230 1.216 0.231 0.249 0.505 1.366 0.000 1.413 1.225
## 9 3.512 0.185 0.205 1.419 1.165 1.093 0.219 1.413 0.000 0.383
## 10 3.130 0.287 0.366 1.176 0.981 0.813 0.545 1.225 0.383 0.000## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP 2:## ## Every observation is labeled as UNVISITED. We are now going to loop over every o## bservation and, if it is not already assigned to a cluster, we will try to expan## d a new cluster around it...## ## Press [enter] to continue## ## ________________________________________________________________________________## NOISE:## ## An UNVISITED observation is labeled as NOISE:## Observation #1 [UNVISITED -> NOISE]## ## Press [enter] to continue## ## ________________________________________________________________________________## CLUSTER #1:## ## A new cluster is going to be expanded around an UNVISITED core observation:## Observation #2 [UNVISITED -> CLUSTER #1]## ## Press [enter] to continue## ## The cluster is also expanded around the neighbors of the core observation:## Observation #3 [UNVISITED -> CLUSTER #1]
## Observation #7 [UNVISITED -> CLUSTER #1]
## Observation #9 [UNVISITED -> CLUSTER #1]
## Observation #10 [UNVISITED -> CLUSTER #1]## ## All of these observations are added to the cluster.## ## Press [enter] to continue## ## ***## ## The following core observation is expanded:## Observation #3 [CLUSTER #1]## ## It's neighborhood is:## Observation #2 [CLUSTER #1]
## Observation #3 [CLUSTER #1]
## Observation #7 [CLUSTER #1]
## Observation #9 [CLUSTER #1]## ## Upon doing it, no observations are added to the cluster...## ## Additionally, no other observations are expanded...## ## Press [enter] to continue## ## ***## ## The following core observation is expanded:## Observation #7 [CLUSTER #1]## ## It's neighborhood is:## Observation #2 [CLUSTER #1]
## Observation #3 [CLUSTER #1]
## Observation #7 [CLUSTER #1]
## Observation #9 [CLUSTER #1]## ## Upon doing it, no observations are added to the cluster...## ## Additionally, no other observations are expanded...## ## Press [enter] to continue## ## ***## ## The following core observation is expanded:## Observation #9 [CLUSTER #1]## ## It's neighborhood is:## Observation #2 [CLUSTER #1]
## Observation #3 [CLUSTER #1]
## Observation #7 [CLUSTER #1]
## Observation #9 [CLUSTER #1]## ## Upon doing it, no observations are added to the cluster...## ## Additionally, no other observations are expanded...## ## Press [enter] to continue## ## ________________________________________________________________________________## NOISE:## ## An UNVISITED observation is labeled as NOISE:## Observation #4 [UNVISITED -> NOISE]## ## Press [enter] to continue## ## ________________________________________________________________________________## NOISE:## ## An UNVISITED observation is labeled as NOISE:## Observation #5 [UNVISITED -> NOISE]## ## Press [enter] to continue## ## ________________________________________________________________________________## NOISE:## ## An UNVISITED observation is labeled as NOISE:## Observation #6 [UNVISITED -> NOISE]## ## Press [enter] to continue## ## ________________________________________________________________________________## NOISE:## ## An UNVISITED observation is labeled as NOISE:## Observation #8 [UNVISITED -> NOISE]## ## Press [enter] to continue## ## ________________________________________________________________________________## RESULTS:## ## Having gone through every observation the following clusters have been found:## CLUSTER #0 (NOISE):
## x y
## 1 -1.5781168 -1.29286766
## 4 0.6757613 -0.04002442
## 5 0.8484305 0.22306094
## 6 0.5154890 0.30141469
## 8 0.9062708 -0.01894187## ## CLUSTER #1:
## x y
## 2 0.7027994 1.193823
## 3 0.7854535 1.191428
## 7 0.9187371 1.347416
## 9 0.7017478 1.378730
## 10 0.4289005 1.109321## ## Press [enter] to continue## ## ________________________________________________________________________________
This probabilistic model assumes that the data points are generated from a mixture of several Gaussian distributions.
## ________________________________________________________________________________## EXPLANATION:## ## The Gaussian Mixture Model with Expectation Maximization (GMM with EM) algorithm## aims to model the data as a Gaussian Mixture Model i.e. the weighted sum of sev## eral Gaussian distributions, where each component i.e. each Gaussian distributio## n, represents a cluster.## ## The Gaussian distributions are parameterized by their mean vector (mu), covarian## ce matrix (sigma) and mixing proportion (lambda). Initially, the mean vector is## set to the cluster centers obtained by performing a k-means clustering on the da## ta, the covariance matrix is set to the covariance matrix of the data points bel## onging to each cluster and the mixing proportion is set to the proportion of dat## a points belonging to each cluster. The algorithm then optimizes the GMM using t## he EM algorithm.## ## The EM algorithm is an iterative algorithm that alternates between two steps:## ## 1. Expectation step. Compute how much is each observation expected to belong## to each component of the GMM.## 2. Maximization step. Recompute the GMM according to the expectations from t## he E-step in order to maximize them.## ## The algorithm stops when the changes in the expectations are sufficiently small## or when a maximum number of iterations is reached.## ## Press [enter] to continue## ## ________________________________________________________________________________## INITIALIZATION:## ## The GMM is initialized by calling kmeans. The initial components are:## *** Component #1 ***## mu:
## [1] 0.7364879 0.1163773
## sigma:
## [,1] [,2]
## [1,] 0.03129520 -0.01414258
## [2,] -0.01414258 0.02946434
## lambda:
## [1] 0.4## ## *** Component #2 ***## mu:
## [1] -1.578117 -1.292868
## sigma:
## [,1] [,2]
## [1,] NA NA
## [2,] NA NA
## lambda:
## [1] 0.1## ## *** Component #3 ***## mu:
## [1] 0.7075276 1.2441437
## sigma:
## [,1] [,2]
## [1,] 0.03209268 0.01368235
## [2,] 0.01368235 0.01306666
## lambda:
## [1] 0.5## ## These initial components are then optimized using the EM algorithm.## ## Press [enter] to continue## ## ________________________________________________________________________________## EM ALGORITHM:## ## To measure how much the expectations change at each step we will use the log lik## elihood. The log likelihood is the sum of the logarithm of the probability of th## e data given the model. The higher the log likelihood, the better the model.## ## The current log likelihood is:## loglik:
## [1] -958.4645## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP #0:## ## E-STEP:## ## The expectation of each observation to belong to each component of the GMM is th## e following:## Expectation:
## [,1] [,2] [,3]
## 1 1 NA 0
## 2 0 NA 1
## 3 0 NA 1
## 4 1 NA 0
## 5 1 NA 0
## 6 1 NA 0
## 7 0 NA 1
## 8 1 NA 0
## 9 0 NA 1
## 10 0 NA 1## ## Press [enter] to continue## ## M-STEP:## ## The new components are:## *** Component #1 ***## mu:
## [1] 0.2735670 -0.1654717
## sigma:
## [,1] [,2]
## [1,] 1.0949504 0.6417621
## [2,] 0.6417621 0.4192926
## lambda:
## [1] 0.5## ## *** Component #2 ***## mu:
## [1] 1 1
## sigma:
## [,1] [,2]
## [1,] 0.8415997 0.7219930
## [2,] 0.7219930 0.9798987
## lambda:
## [1] 1e-300## ## *** Component #3 ***## mu:
## [1] 0.7075276 1.2441437
## sigma:
## [,1] [,2]
## [1,] 0.03209268 0.01368235
## [2,] 0.01368235 0.01306666
## lambda:
## [1] 0.5## ## Press [enter] to continue## ## The new log likelihood is:## loglik:
## [1] -210.6484## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP #1:## ## E-STEP:## ## The expectation of each observation to belong to each component of the GMM is th## e following:## Expectation:
## [,1] [,2] [,3]
## 1 1 0 0
## 2 0 0 1
## 3 0 0 1
## 4 1 0 0
## 5 1 0 0
## 6 1 0 0
## 7 0 0 1
## 8 1 0 0
## 9 0 0 1
## 10 0 0 1## ## Press [enter] to continue## ## M-STEP:## ## The new components are:## *** Component #1 ***## mu:
## [1] 0.2735670 -0.1654716
## sigma:
## [,1] [,2]
## [1,] 1.0949503 0.6417621
## [2,] 0.6417621 0.4192927
## lambda:
## [1] 0.5## ## *** Component #2 ***## mu:
## [1] 0.6035213 0.2872708
## sigma:
## [,1] [,2]
## [1,] 0.1674271 0.0841496
## [2,] 0.0841496 0.2447075
## lambda:
## [1] 4.072114e-301## ## *** Component #3 ***## mu:
## [1] 0.7075276 1.2441437
## sigma:
## [,1] [,2]
## [1,] 0.03209268 0.01368235
## [2,] 0.01368235 0.01306666
## lambda:
## [1] 0.5## ## Press [enter] to continue## ## The new log likelihood is:## loglik:
## [1] -4.75882## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP #2:## ## E-STEP:## ## The expectation of each observation to belong to each component of the GMM is th## e following:## Expectation:
## [,1] [,2] [,3]
## 1 1 0 0
## 2 0 0 1
## 3 0 0 1
## 4 1 0 0
## 5 1 0 0
## 6 1 0 0
## 7 0 0 1
## 8 1 0 0
## 9 0 0 1
## 10 0 0 1## ## Press [enter] to continue## ## M-STEP:## ## The new components are:## *** Component #1 ***## mu:
## [1] 0.2735670 -0.1654716
## sigma:
## [,1] [,2]
## [1,] 1.0949503 0.6417621
## [2,] 0.6417621 0.4192927
## lambda:
## [1] 0.5## ## *** Component #2 ***## mu:
## [1] 0.6502022 0.2060456
## sigma:
## [,1] [,2]
## [1,] 0.04111510 -0.02457912
## [2,] -0.02457912 0.05821502
## lambda:
## [1] 6.015025e-301## ## *** Component #3 ***## mu:
## [1] 0.7075276 1.2441437
## sigma:
## [,1] [,2]
## [1,] 0.03209268 0.01368235
## [2,] 0.01368235 0.01306666
## lambda:
## [1] 0.5## ## Press [enter] to continue## ## The new log likelihood is:## loglik:
## [1] -4.758821## ## Press [enter] to continue## ## ________________________________________________________________________________## STEP #3:## ## E-STEP:## ## The expectation of each observation to belong to each component of the GMM is th## e following:## Expectation:
## [,1] [,2] [,3]
## 1 1 0 0
## 2 0 0 1
## 3 0 0 1
## 4 1 0 0
## 5 1 0 0
## 6 1 0 0
## 7 0 0 1
## 8 1 0 0
## 9 0 0 1
## 10 0 0 1## ## Press [enter] to continue## ## M-STEP:## ## The new components are:## *** Component #1 ***## mu:
## [1] 0.2735670 -0.1654716
## sigma:
## [,1] [,2]
## [1,] 1.0949503 0.6417621
## [2,] 0.6417621 0.4192927
## lambda:
## [1] 0.5## ## *** Component #2 ***## mu:
## [1] 0.6489054 0.1874634
## sigma:
## [,1] [,2]
## [1,] 0.04375938 -0.02950721
## [2,] -0.02950721 0.03592890
## lambda:
## [1] 3.077336e-300## ## *** Component #3 ***## mu:
## [1] 0.7075276 1.2441437
## sigma:
## [,1] [,2]
## [1,] 0.03209268 0.01368235
## [2,] 0.01368235 0.01306666
## lambda:
## [1] 0.5## ## Press [enter] to continue## ## The new log likelihood is:## loglik:
## [1] -4.758821## ## Press [enter] to continue## ## ________________________________________________________________________________## FINAL RESULTS:## ## The algorithm stopped because the change in the log likelihood was smaller than## 1e-6.## ## With the current GMM every observation is assigned to the cluster it is most lik## ely to belong to. The final clusters are:## Cluster assignments:
## 1 2 3 4 5 6 7 8 9 10
## 1 3 3 1 1 1 3 1 3 3## ## ________________________________________________________________________________## 
This algorithm combines traditional k-means with genetic algorithm concepts for potentially better cluster optimization.
result <- genetic_kmeans(
data,
k = 3,
population_size = 10,
mut_probability = 0.5,
max_generations = 10,
learn = TRUE
)## ________________________________________________________________________________## EXPLANATION:## ## The Genetic K-Means algorithm combines the K-Means clustering method with geneti## c algorithm concepts.## It follows these main steps:## 1. Initialize a population of random cluster assignments## 2. Evaluate the fitness of each individual based on within-cluster variation## 3. Select parents for the next generation based on fitness## 4. Apply mutation and crossover to create new individuals## 5. Repeat steps 2-4 for a specified number of generations## ## Press [enter] to continue## ________________________________________________________________________________## INITIALIZATION:## ## A population of 10 individuals has been randomly initialized.## Each individual represents a possible clustering solution.## Here's a sample of the initial population (first 5 individuals, first 10 data po## ints):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 3 3 2 1 3 2 3 1 2 1
## [2,] 1 1 3 3 2 3 2 3 1 2
## [3,] 3 1 2 1 2 1 3 2 1 3
## [4,] 1 1 3 1 2 2 3 2 2 3
## [5,] 1 3 3 2 1 3 2 3 1 2## ## Press [enter] to continue## INITIAL FITNESS:## Best fitness: 4.74## Average fitness: 2.39## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 1## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 1 3 2 1 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 1 2 3 3 3 3
## [2,] 1 2 2 2 1 1 2 2 3 3
## [3,] 1 1 3 2 2 1 2 2 3 3
## [4,] 1 2 2 2 1 2 3 2 3 2
## [5,] 1 2 3 2 1 2 3 2 3 3## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 1 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 2.41## Average fitness: 2.17## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 2## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 3 2 2 2 2 3 3
## [2,] 1 3 3 2 2 2 1 2 3 3
## [3,] 1 3 2 3 2 2 3 2 3 3
## [4,] 1 3 3 2 1 2 3 2 2 2
## [5,] 1 3 3 2 2 2 3 3 3 2## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 0.00## Average fitness: 0.00## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 3## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 3 3 2 2 3
## [2,] 1 2 3 2 2 3 1 3 3 1
## [3,] 1 2 3 2 2 2 3 2 3 3
## [4,] 1 2 3 3 2 2 3 2 3 1
## [5,] 1 3 3 2 2 2 3 3 3 3## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 3.43## Average fitness: 3.09## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 4## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 1 2 2 2 3 3 3 3
## [2,] 1 3 2 2 1 1 3 2 3 3
## [3,] 3 2 3 2 1 2 2 2 3 3
## [4,] 1 2 2 2 2 2 3 2 3 3
## [5,] 3 3 3 2 2 2 3 2 3 1## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 3 2 2 1 1 1 2 1 2 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 3 1 3 2 2 2 1 3 1 1## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 7.46## Average fitness: 5.72## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 5## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 2 2 2 2 3 2 3 3
## [2,] 1 2 3 1 3 2 3 2 3 2
## [3,] 1 3 3 2 2 3 1 3 3 3
## [4,] 3 3 1 1 2 1 2 2 3 2
## [5,] 1 3 3 1 2 3 1 2 3 3## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 3 2 2 1 1 1 2 1 2 2
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 3.45## Average fitness: 3.10## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 6## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 2 3 2 2 2 2 1 2 3 3
## [2,] 1 3 3 2 2 2 1 2 3 1
## [3,] 1 2 3 1 2 2 1 2 3 2
## [4,] 1 3 2 2 2 1 3 2 1 3
## [5,] 1 3 3 2 2 3 3 2 2 3## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 2 3 3 2 2 2 1 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 4.25## Average fitness: 3.83## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 7## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 2 3 3 2 3 1 1 2 1 3
## [2,] 1 3 3 1 2 2 3 2 3 1
## [3,] 1 1 3 2 2 2 2 2 1 3
## [4,] 1 3 3 2 3 3 1 2 3 3
## [5,] 3 3 2 2 2 1 1 2 2 2## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 2 1 1 2 3 3 1 3 1 1
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 3 1 1 2 2 2 1 2 1 1## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 3.43## Average fitness: 2.77## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 8## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 2
## [2,] 1 2 2 2 2 2 3 2 3 3
## [3,] 1 2 3 2 3 2 3 1 2 3
## [4,] 1 3 3 1 2 2 2 2 3 1
## [5,] 1 3 2 2 2 3 3 3 2 3## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 0.00## Average fitness: 0.00## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 9## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 2 3 3 2 3 1 3 2 1 1
## [2,] 1 3 2 2 2 3 3 2 3 3
## [3,] 1 3 3 2 3 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 1 3 3
## [5,] 1 3 1 2 2 3 3 2 3 3## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 2 3 3 2 3 1 3 2 3 1
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 1 3 3 2 2 2 3 2 3 3
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 5.35## Average fitness: 4.38## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## GENERATION: 10## ## SELECTION:## Parents for the next generation are selected based on their fitness.## Selected parent (first 10 data points):## [1] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## MUTATION:## Random mutations are applied to the chromosomes with probability0.50## Sample of mutated population (first 5 individuals, first 10 data points):## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 2 3 2 1 2 3 2 3 1
## [2,] 1 3 2 2 2 2 3 2 3 3
## [3,] 3 1 2 3 1 2 3 2 1 3
## [4,] 1 3 3 2 3 2 3 2 2 2
## [5,] 1 2 3 2 2 2 3 2 2 2## ## Press [enter] to continue## CROSSOVER:## K-Means Operator (KMO) is applied as a form of crossover.## This reassigns each point to its nearest center.## Sample of population after crossover (first 5 individuals, first 10 data points)## :## [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,] 1 3 3 2 2 2 3 2 3 3
## [2,] 1 3 3 2 2 2 3 2 3 3
## [3,] 3 1 1 2 2 2 1 2 1 1
## [4,] 1 3 3 2 2 2 3 2 3 3
## [5,] 1 3 3 2 2 2 3 2 3 3## ## Press [enter] to continue## FITNESS EVALUATION:## The fitness of each individual is calculated based on within-cluster variation.## Best fitness in this generation: 2.45## Average fitness: 2.21## Total Within-Cluster Variation of best solution: 0.36## ## Press [enter] to continue## ________________________________________________________________________________## FINAL RESULTS:## ## Number of clusters: 3## Total sum of squares: 11.68## Total within-cluster sum of squares: 0.36## Between-cluster sum of squares: 11.31## Cluster sizes:## [1] 1 4 5## Final cluster centers:## [,1] [,2]
## 1 "-1.58" "-1.29"
## 2 "0.74" "0.12"
## 3 "0.71" "1.24"## ## Generating plot of clustering results...## Plot generated. Check the graphics device.## 
Correlation clustering performs hierarchical clustering by analyzing relationships between data points and a target, with support for weighted features.
# Create sample data
data <- matrix(c(1,2,1,4,5,1,8,2,9,6,3,5,8,5,4), ncol=3)
dataFrame <- data.frame(data)
target <- c(1,2,3)
weights <- c(0.1, 0.6, 0.3)
# Perform correlation clustering
result <- correlation_clustering(
dataFrame,
target = target,
weight = weights,
distance_method = "euclidean",
normalize = TRUE,
learn = TRUE
)## EXPLANATION:## ## The Correlation Hierarchical Clustering algorithm is a classification technique## that:## ## 1. Initializes a cluster for each data point## 2. Calculates distances between clusters and a given target## 3. Applies weights to achieve weighted results## ## Due to normalize = TRUE, weights will be normalized to [0,1] range## ## ________________________________________________________________________________## WEIGHT NORMALIZATION:## ## Initial weights:## Weight 1: 0.100000## Weight 2: 0.600000## Weight 3: 0.300000## No weights provided## Initializing 3 weights with value 1## ## Normalizing weights:## Total sum: 1.000000## Formula: weight[i] = weight[i] / total## ##
## These are the new weights:## Weight 1: 0.100000## Weight 2: 0.600000## Weight 3: 0.300000## ## ________________________________________________________________________________## WEIGHTS:## ## The following weights will be used:## [1] 0.1 0.6 0.3## ## Euclidean Distance Formula:## d(x,y) = √(∑1ⁿ (xᵢ - yᵢ)²)## This distance metric will be used to:## - Calculate distances between each cluster and the target## - Sort clusters by their similarity to the target## ## DATA INITIALIZATION:## ## Input data:## X1 X2 X3
## 1 1 1 3
## 2 2 8 5
## 3 1 2 8
## 4 4 9 5
## 5 5 6 4## ## Each cluster is initialized as a matrix with one row and the same columns as the## input data## ## Initialized clusters:## [[1]]
## [,1] [,2] [,3]
## [1,] 1 1 3
##
## [[2]]
## [,1] [,2] [,3]
## [1,] 2 8 5
##
## [[3]]
## [,1] [,2] [,3]
## [1,] 1 2 8
##
## [[4]]
## [,1] [,2] [,3]
## [1,] 4 9 5
##
## [[5]]
## [,1] [,2] [,3]
## [1,] 5 6 4## ## ________________________________________________________________________________## TARGET INITIALIZATION:## ## Input target:## [1] 1 2 3## ## [1] 1 2 3## Converting vector target to matrix format## ## Validating target dimensions:## Target has one row## Target has same number of columns as input data## ## Final target:## [,1] [,2] [,3]
## [1,] 1 2 3## ## ________________________________________________________________________________## INITIALIZATION:## ## Initialized data:## [[1]]
## [,1] [,2] [,3]
## [1,] 1 1 3
##
## [[2]]
## [,1] [,2] [,3]
## [1,] 2 8 5
##
## [[3]]
## [,1] [,2] [,3]
## [1,] 1 2 8
##
## [[4]]
## [,1] [,2] [,3]
## [1,] 4 9 5
##
## [[5]]
## [,1] [,2] [,3]
## [1,] 5 6 4## ## Initialized target:## [,1] [,2] [,3]
## [1,] 1 2 3## ## ________________________________________________________________________________## DISTANCE CALCULATION:## ## Calculating distances between clusters and target using euclidean distance## ## ________________________________________________________________________________## DISTANCES:## ## Calculated distances:## [1] 1.000000 6.403124 5.000000 7.874008 5.744563## ## Sorted distances:## [1] 1.000000 5.000000 5.744563 6.403124 7.874008## ##
## Then, using sorted distances, the function order the clusters.
## ## ________________________________________________________________________________## RESULTS:## ## Final sorted distances:## cluster sortedDistances
## 1 1 1.000000
## 2 3 5.000000
## 3 5 5.744563
## 4 2 6.403124
## 5 4 7.874008## ## Final sorted clusters:## cluster X1 X2 X3
## 1 1 1 1 3
## 2 3 1 2 8
## 3 5 5 6 4
## 4 2 2 8 5
## 5 4 4 9 5## ## Dendrogram visualization:## ## ________________________________________________________________________________
The package supports various distance metrics for algorithms like agglomerative clustering and correlation clustering, the available metrics are: euclidean distance, manhattan distance, canberra distance and chebyshev distance.
You can specify these in algorithms that accept a distance parameter:
## Cluster method : single
## Distance : euclidean
## Number of objects: 5
## Cluster method : single
## Distance : manhattan
## Number of objects: 5
## Cluster method : single
## Distance : canberra
## Number of objects: 5
## Cluster method : single
## Distance : maximum
## Number of objects: 5