PART B ▪ UNIT 2 · Advanced Concepts of Modeling in AI

  root@vm-learning
  ~
  $
  open
  ch-b2
  

PART B ▪ UNIT 2

Advanced Concepts of Modeling in AI

AI · ML · DL · Rule-based vs Learning · Neural Networks

AI Modelling is the process of developing algorithms (also called models) that can be trained to give intelligent outputs. In simple terms, it is writing code to make a machine "artificially intelligent". There are two broad ways to build a model — Rule-Based or Learning-Based.

This unit has three sub-units: (2.1) Revisiting AI, ML & DL and common data terms · (2.2) Modelling — Rule-based vs Learning-based, plus three categories of ML (Supervised, Unsupervised, Reinforcement) with all sub-categories · (2.3) Artificial Neural Networks — how neurons work and how AI makes a decision.

Learning Outcome 1: Differentiate between AI, ML and DL + common data terms

2.1 Revisiting AI, ML & DL

To build an AI project, we work with AI models / algorithms. Before jumping into modelling let us clarify the three related terms:

🤖 AI (Artificial Intelligence)

Any technique that enables computers to mimic human intelligence. An AI machine works on algorithms and data fed to it, producing the desired output.

🧠 ML (Machine Learning)

Enables machines to improve at tasks with experience. The machine learns from new data, takes mistakes into account, and considers exceptions in the next iteration.

🕳️ DL (Deep Learning)

Enables software to train itself using vast amounts of data. Multiple ML algorithms work together inside DL. The most advanced form of AI.

🔹 Relationship between AI, ML and DL — Funnel / Venn Model

🔄 AI ⊇ ML ⊇ DL

AI — all techniques that mimic human intelligence

ML — improves with experience

DL — self-trains with deep neural nets

Funnel approach: AI is the umbrella → ML is a subset of AI → DL is a subset of ML. A lot of AI applications exist; fewer use ML; very few use DL.

🧠 Machine Learning (ML) — Examples

ML model is given past / historical data as input; the model learns from it and generates output.

Object Classification — identifies and labels objects in an image (e.g., apple vs strawberry).
Anomaly Detection — finds unexpected things in data. Example: tracking heart rate and flagging a sudden spike as a potential issue.

🕳️ Deep Learning (DL) — Examples

Input is given to an ANN; after processing, the output is generated by the DL block. The machine can develop algorithms for itself.

Object Identification — identifies and labels objects within an image using powerful DL algorithms.
Digit Recognition — training computers to identify handwritten digits (0-9) within images.

2.2 Common Terminologies Used with Data

📊 DataInformation in any form. Example: a table with details of fruits — each row has information about different fruits.

🏷️ FeaturesColumns of the table — the attributes. For the fruit dataset: name, colour, size.

🎯 LabelsA special feature used to attach meaning to data. Depends on the problem you are solving. If we predict the fruit based on colour, then colour = feature, fruit name = label.

🗂️ Labeled DataData that has tags/labels attached — e.g., name, type, number.

📦 Unlabeled DataRaw data without any tags.

📚 Training DataExamples given to the model to analyse and learn from — like a teacher teaching through solved examples.

🧪 Testing DataUsed to test the accuracy of the model — like a surprise class test. Done without labels; results are then verified with labels.

📉 DatasetA collection of data used for training or testing.

Learning Outcome 2: Familiarise with Supervised, Unsupervised & Reinforcement Learning

2.3 Modelling — Types of AI Models

Generally, AI models are classified into two main approaches:

🗂️ CLASSIFICATION OF AI MODELS

AI Models

Rule-Based Learning-Based

ML: Supervised ML: Unsupervised ML: Reinforcement DL: ANN / CNN

📘 1. Rule-Based Approach

Rule-Based Approach refers to AI modelling where the relationships or patterns in data are defined by the developer. The machine follows the rules mentioned by the developer and performs its task accordingly.

Example — Rule-Based Chatbot on a clothing website:
Rule 1: If user message contains "track order", "shipment status" or "delivery" → show order-tracking options.
Rule 2: If user asks for their order number → prompt to enter it. If entered correctly → show tracking info. If invalid → show error and ask again.
Rule 3: If no defined rule matches → "Sorry, I can't help. Please try our FAQs or contact support."

Limitation of Rule-Based: Learning is static — once trained, the machine cannot improvise on feedback. It fails on any dataset different from the rules and data it was given. That's why Machine Learning was introduced.

📗 2. Learning-Based Approach

A Learning-Based Approach is a method where a computer learns how to do something by looking at examples or getting feedback — similar to how humans learn from experience. Instead of being explicitly programmed, the computer analyses data and finds patterns or rules on its own.

Example — Learning-Based Spam Email Filter: Instead of being programmed with rules to identify spam, the filter learns from thousands of labelled emails (spam / not spam). It analyses words used, sender info and attachments, then classifies new emails. It continuously adapts and improves as it encounters new examples.

Example — 1000 Stray Dog Images: You have 1000 images but no info about breed or colour. Put this into a learning-based model — the machine finds its own patterns. It might cluster by colour, size, fur style, or even a clustering you hadn't thought of!

2.4 Categories of Machine Learning Models

Machine Learning can be further divided into three main categories:

1️⃣ Supervised Learning

Machine learns from labelled data — like a teacher teaching a class.

2️⃣ Unsupervised Learning

Machine finds its own patterns in unlabelled data — like a child learning to swim alone.

3️⃣ Reinforcement Learning

Machine learns by trial and error using rewards and penalties.

📗 1. Supervised Learning

In a Supervised Learning model, the dataset fed to the machine is labelled. A label is some information used as a tag. Example: student grades are labels based on marks.

Analogy: "Supervised" = having a supervisor. A maths teacher teaches using many solved examples (training), then tests the class with new problems (testing).

Currency Coin Example:
Problem: Predict the coin based on its weight.
Dataset: 1 Euro = 5g · 1 Dirham = 7g · 1 Dollar = 3g · 1 Rupee = 4g …
Feature = Weight · Label = Currency
The trained model can then identify a new coin just from its weight.

📘 2. Unsupervised Learning

An Unsupervised Learning model works on unlabelled datasets. The person training the model does not have information about it. It is used to identify relationships, patterns and trends in the data.

Analogy: A child learning to swim on his own without any supervision. The child is the model; the swimming pool is the unknown data.

Supermarket Customer Example: You have purchase records with no labels of "grocery shopper" or "non-grocery shopper". The model discovers the pattern on its own and creates these two clusters — useful for sending a grocery-offer message to regular grocery buyers.

Cats & Dogs: Input data has no labels. The model analyses characteristics and forms 2 clusters — all cats in Cluster 1, all dogs in Cluster 2, based on similarity.

🔹 Supervised vs Unsupervised — Quick Comparison

Aspect	Supervised Learning	Unsupervised Learning
Input Data	Labelled (tagged)	Unlabelled (raw)
Goal	Predict / classify using known labels	Find hidden patterns & groupings
Analogy	Student learning with teacher	Child discovering on its own
Problem Types	Classification, Regression	Clustering, Association
Example	Currency prediction, spam detection	Customer segmentation, Netflix recommendations

🔹 Identify the Model — Quick Test

Case	Answer
Social media identifies your friend in a picture from tagged photos.	Supervised — uses tagged photos as labels.
OTT platform recommends movies based on watch history.	Unsupervised — groups by similar features without labels.
Bank analyses transactions for suspicious fraud (no "fraud" labels given).	Unsupervised — identifies outliers / anomalies.

🎮 3. Reinforcement Learning

Reinforcement Learning enables the computer to make a series of decisions that maximise a reward metric — without human intervention and without being explicitly programmed. The machine learns through a repeated trial-and-error method.

Fruit Recognition:
• Show the machine an apple — it predicts "cherry". You give negative feedback: "incorrect".
• Show apple again — it knows it's not a cherry. Predicts "apple" — you give positive feedback: "correct".
• Machine has now learned: this is an apple.

🔹 When to Use Reinforcement Learning

You face large, complex problem spaces.
You must respond to unforeseen environments.
You don't have sufficient labelled data for specific scenarios.
The environment may change — the system needs to be adaptive.

Reinforcement Learning is powerful because it doesn't require a lot of pre-existing data to provide useful solutions.

🔹 Real-World RL Examples

Parking a car automatically.
Humanoid robot learning to walk.
AlphaGo, chess-playing AI, self-driving cars.
Game-playing bots that learn from wins/losses.

🔹 Summary of 3 ML Categories

Category	Used When…
Supervised	We want to determine relationships through training.
Unsupervised	We want to discover new patterns from data.
Reinforcement	We want to implement ML through a reward mechanism.

2.5 Sub-Categories of Supervised Learning

Supervised Learning has two sub-types based on the type of output:

🎯 1. Classification Model

Data is classified according to labels. Output is discrete / categorical (yes/no, spam/not spam, cat/dog). Example: grading students, email spam filter.

📈 2. Regression Model

Works on continuous data. Output is a continuous value like temperature, price, income, age. Example: predicting salary, house price.

🔹 Classification — Examples

Weather Classifier: trained on labelled "hot" or "cold" weather data — considers location, season, humidity, high/low temperature. Predicts tomorrow as hot or cold.
Email Spam Filter: analyses clues in an email → classifies as "spam" or "not spam". Modern email also categorises into social, promotions, notifications.

🔹 Regression — Examples

Predicting Temperature — temperature is continuous, so regression is well-suited.
House Price Prediction: Features (independent): number of bedrooms, carpet size, garage area. Label (dependent): Price.
Used-Car Price Prediction: predicts continuous price using fuel type, years of service, number of previous owners, kilometres driven, transmission type.

🔹 Identify Model — Classification or Regression?

Case	Answer
Predict whether a customer is eligible for a bank loan.	Classification (yes/no — discrete)
Predict the weather for the next 24 hours.	Regression (continuous range)

2.6 Sub-Categories of Unsupervised Learning

Unsupervised Learning has two sub-types:

🔵 1. Clustering

Groups data based on similarity of characteristics. Example: grouping birds and animals without labels — cluster 1 = all animals, cluster 2 = all birds.

🔗 2. Association

Finds interesting relationships between variables in the database. Example: customer who buys bread will most probably also buy butter.

Clustering vs Classification:
• Classification uses predefined classes to assign objects (labels known).
• Clustering finds similarities between objects and places them in the same cluster, differentiating from others (labels unknown).

🔹 Clustering Example — Jim's Music

Jim likes music with slow tempo & soft intensity. He dislikes fast tempo & high intensity. The model creates 2 clusters: songs he likes vs songs he dislikes. If a new song X has slow tempo + soft intensity → it goes into the "liked" cluster. Netflix and Spotify use similar clustering for recommendations.

🔹 Association Example — Shop Purchases

Customer A buys bread + butter. Customer B buys bread + butter + jam. Pattern: customers who buy bread usually also buy butter. Store can recommend butter to anyone buying bread. This is the Association Rule.

2.7 Sub-Categories of Deep Learning

Deep Learning has two main types:

🧠 1. Artificial Neural Networks (ANN)

Modelled on the human brain and nervous system. Every node is essentially an ML algorithm. ANNs extract features automatically — no programmer input. Useful for very large datasets.

📷 2. Convolutional Neural Networks (CNN)

DL algorithm that takes an input image, assigns importance (learnable weights & biases) to various aspects, and differentiates one image from another. Used in computer vision & image recognition.

Learning Outcome 3: Understand Neural Networks & how AI makes decisions

2.8 What is a Neural Network?

Neural Networks are loosely modelled after how neurons in the human brain behave. The key advantage: they can extract data features automatically without needing programmer input. A fast, efficient way to solve problems with very large datasets like images.

🔹 Architecture — Layers of a Neural Network

1. Input Layer

Acquires data and feeds it into the network. No processing occurs here.

2. Hidden Layer(s)

Where the whole processing happens. Each node runs its own ML algorithm on data from the previous layer. There can be multiple hidden layers; number depends on problem complexity.

3. Output Layer

Receives processed data from the last hidden layer and delivers the final output to the user. No processing here.

🔹 How Hidden Layers Work — Weights, Biases, Activation

Each hidden-layer node computes using weights and biases.
Information passes through an activation function from one layer to the next.
The network learns by trial & error — with each try, weights are adjusted based on the error between desired output and network output.
This process of finding the right output is called training.

🔹 Real-World Applications of Neural Networks

Face recognition (phone unlock, airport security).
Customer-support chatbots.
Vegetable & stock price prediction.
Medical image analysis (MRI, X-ray).
Handwritten digit recognition.
Self-driving car vision.

2.9 How Does AI Make a Decision?

Scenario: You want to go out to the park today. Should you go?

🔹 Step 1 — Identify Inputs (Factors)

Four inputs influence your decision:

Do I have a jacket?
Do I have an umbrella?
Is it sunny now?
What is the weather forecast for later?

🔹 Step 2 — Assign Importance (Weights)

Not all factors are equal. Rank them:

"Is it sunny now?" → most important.
"Weather forecast later" → very important.
"Having a jacket" → important.
"Having an umbrella" → least important.

🔹 Step 3 — Perceptron Model

A Perceptron is the simplest neural network unit. It takes multiple inputs (X1 – X4), each with an associated weight (W1 – W4), plus a bias (B) with its own weight (WB). All are summed and compared with a threshold to decide the output.

Output = (X1·W1 + X2·W2 + X3·W3 + X4·W4 + B·WB) compared with threshold (0)

🔹 Where Do Weight Values Come From?

From experience — sunny now usually means a bright day ahead, so weight is high.
From personal preference — a person may judge one factor more heavily than another.
Bias WB — cautious person sets WB higher (stay indoors unless sure); daring person sets WB lower (go out regardless).

There is no right or wrong value for weights. They shape the outcome. That's why different people make different decisions with the same information.

🔹 Example Calculation 1 — Go to Park

Given: I have a jacket (1), no umbrella (0), it IS sunny now (1), forecast = rain (1). Bias always = 1.
Calculation with sample weights: output = 0.5 → higher than threshold (0) → I WILL go to the park.

🔹 Example Calculation 2 — Stay Home

Given: No jacket (0), I have umbrella (1), NOT sunny now (0), forecast = OK (0). Bias = 1.
Calculation: output = −0.5 → lower than threshold (0) → I will NOT go to the park.

2.10 Activity — Human Neural Network (The Game)

Goal: Experience how a neural network works by becoming one!

Setup — 20 students arranged in 4 layers:

Input Layer: 7 students (each writes 6 chits)
Hidden Layer 1: 6 students (each receives 7 chits, writes 4 chits)
Hidden Layer 2: 6 students (receives chits, writes 2 chits)
Output Layer: 1 student (receives 12 chits, makes a final guess)

How to play:

Facilitator shows an image only to the Input Layer.
Each input student writes 6 words describing it on 6 chits.
Chits pass to Hidden Layer 1 → re-written as 4 new chits each.
Chits pass to Hidden Layer 2 → re-written as 2 chits each.
All chits reach the Output node who guesses the image and writes a ≤5-line summary.
If the summary matches the actual image → the whole network wins! Otherwise, it loses.

Ground Rules: No talking · no discussion · one word per chit · process fast · use your own discretion.

Quick Revision — Key Points to Remember

AI = machines mimicking human intelligence. ML = AI that improves with experience. DL = ML that self-trains with vast data. Relation: AI ⊇ ML ⊇ DL.
Data terms: Data · Features (columns) · Labels (tags) · Labeled vs Unlabeled · Training data · Testing data.
2 modelling approaches: Rule-Based (developer defines rules — e.g., FAQ chatbot) · Learning-Based (machine finds patterns — e.g., spam filter).
3 ML categories: Supervised (labelled data) · Unsupervised (unlabelled data) · Reinforcement (reward-based trial & error).
Supervised analogy: teacher with solved examples. Example: coin-weight → currency.
Unsupervised analogy: child swimming alone. Example: supermarket clustering, Netflix recommendations.
Reinforcement analogy: learn from reward/penalty. Example: apple/cherry feedback, humanoid walking, self-parking car.
Supervised sub-categories: Classification (discrete — spam/not spam, loan yes/no) · Regression (continuous — temperature, house price, car price).
Unsupervised sub-categories: Clustering (similarity groups — Jim's music) · Association (relationships — bread→butter).
DL sub-categories: ANN (general neural nets) · CNN (image recognition).
Neural Network layers: Input (acquires data) · Hidden (processes with weights/biases) · Output (delivers result).
Perceptron: inputs × weights + bias × WB → compare with threshold → output.
Weights/biases come from experience, preference or caution — different people = different decisions.
Human Neural Network Game: 20 students across 4 layers passing chits to guess a hidden image — direct experience of how NNs process information.

🧠Practice Quiz — test yourself on this chapter→