~/Machine Learning Algorithms

1.1 Machine Learning in a Nutshell

• ML algorithms learn from various types of data — images, text, sensor readings, historical records.
• Common algorithms: decision trees, neural networks, support vector machines.
• Wide applications: Netflix recommendations, speech recognition, medical diagnosis, autonomous vehicles, chatbots, personalised ads, fraud detection.
• Challenges: overfitting (too specialised on training data), bias in training data, lack of interpretability (black boxes).

1.2 Three Types of Machine Learning

1.3 Supervised Learning — Two Types

Supervised learning splits into two families depending on what kind of output you are predicting:

2.1 Correlation — Foundation of Regression

Three Types of Correlation

Pearson's Correlation Coefficient (r)

Requirements for using Pearson's r

1. Scale of measurement must be interval or ratio.
2. Variables should be approximately normally distributed.
3. The association must be linear.
4. There should be no outliers in the data.

2.2 When NOT to Use Regression

2.3 Regression — Predicting a Continuous Variable

Variables in Regression

Linear Regression Equation

• a — intercept of the regression line with the y-axis.
• b — slope; rate of change in y per unit change in x.
• e — error / residual (difference between observed y and predicted y).

2.4 Finding the Best-Fit Line

The least-squares method minimises the squared differences between observed values of y and values predicted by the regression equation. This gives the best-fit line that captures the overall trend of the data.

Properties of the Regression Line

• Minimises the sum of squared differences between observed (y) and predicted (ŷ) values.
• Passes through the mean of independent and dependent features.

2.5 Linear Regression — Two Types

Applications of Linear Regression

Python — Linear Regression (Advanced)

import numpy as np
import matplotlib.pyplot as plt

# Sample data
x = np.array([1, 2, 3, 4, 5])
y = np.array([2, 4, 5, 4, 5])

# Calculate means and std devs
x_mean = np.mean(x)
y_mean = np.mean(y)
x_std = np.std(x)

# Covariance and slope
covariance = np.sum((x - x_mean) * (y - y_mean)) / (len(x) - 1)
slope = covariance / (x_std**2)
intercept = y_mean - slope * x_mean

# Predict and plot
y_pred = slope * x + intercept
plt.scatter(x, y)
plt.plot(x, y_pred, color="red")
plt.title("Simple Linear Regression")
plt.show()

print(f"Slope: {slope:.2f}")
print(f"Intercept: {intercept:.2f}")

3.1 What is Classification?

3.2 How Classification Works — 5 Steps

3.3 Four Types of Classification

3.4 K-Nearest Neighbours (k-NN)

6 Steps in k-NN

Applications, Pros & Cons of k-NN

Python — k-NN (Advanced)

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

# Load dataset
data_set = pd.read_csv("user_data.csv")

# Feature extraction
x = data_set.iloc[:, [2, 3]].values
y = data_set.iloc[:, 4].values

# Split train / test
x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.25, random_state=0)

# Feature scaling
st_x = StandardScaler()
x_train = st_x.fit_transform(x_train)
x_test  = st_x.transform(x_test)

# Train and predict
knn = KNeighborsClassifier(n_neighbors=5)
knn.fit(x_train, y_train)
y_pred = knn.predict(x_test)

4.1 What is Clustering?

4.2 How Clustering Works — 4 Key Steps

4.3 Four Types of Clustering Methods

4.4 K-Means Clustering Algorithm

7 Steps in K-Means

1. Select K — decide the number of clusters.
2. Initial centroids — pick K random points (can be from the dataset).
3. Assign each data point to its closest centroid → forms K clusters.
4. Re-compute centroids — calculate the new mean (centroid) of each cluster.
5. Re-assign every data point to the new closest centroid.
6. Repeat steps 4-5 until no point changes cluster (convergence).
7. Model ready — the clusters are final.

K-Means — Applications

Python — K-Means (Advanced)

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.cluster  import KMeans

# Generate synthetic data
X, _ = make_blobs(n_samples=300, centers=4,
                  cluster_std=0.60, random_state=0)

# Apply K-means
kmeans = KMeans(n_clusters=4)
kmeans.fit(X)
y_kmeans = kmeans.predict(X)

# Plot points coloured by cluster
plt.scatter(X[:, 0], X[:, 1], c=y_kmeans, s=50, cmap="viridis")

# Mark centroids in red
centers = kmeans.cluster_centers_
plt.scatter(centers[:, 0], centers[:, 1], c="red", s=200, alpha=0.75)

plt.title("K-means Clustering")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.show()

5.1 Summary Comparison — Regression vs Classification vs Clustering

5.2 Practical Programs & Certification (Syllabus)