AdaBoostC

Step - 1 : Initialize sample weights: - Assign equal weights to all training examples. For a dataset with N samples, each sample weight (w_i) is initially set to 1/N.
Step - 2 : Boosting iterations: - For a pre-defined number of boosting iterations (denoted by T), repeat steps 3 to 6.
Step - 3 : Train a weak learner: - Train a weak learner (e.g., decision stump) on the training data. A weak learner is a simple model that performs slightly better than random guessing. Typically, a decision stump is used, which is a decision tree with a single split (one level).
Step - 4 : Compute the weighted error: - Calculate the weighted error (ε_t) of the weak learner on the training data. Weighted error is the sum of weights of misclassified samples divided by the sum of all weights.

Weighted Error (ε_t) = Σ (w_i * (1 if misclassified, 0 otherwise)) / Σ w_i

Step - 5 : Calculate the weak learner's weight in the final classifier: - Compute the weight (α_t) of the weak learner based on its performance (weighted error) during training.

Weak Learner Weight (α_t) = 0.5 * ln((1 - ε_t) / ε_t)

Note: If the weighted error (ε_t) is very close to 0 or 1, the weak learner's weight (α_t) approaches infinity, indicating that its predictions are not reliable.

Step - 6 : Update sample weights: - Update the sample weights based on the performance of the weak learner. - Increase the weights of misclassified samples so that they get more emphasis in the next iteration. - Decrease the weights of correctly classified samples to reduce their impact in the next iteration.

For misclassified samples (i) => w_i = w_i * exp(α_t)

For correctly classified samples (i) => w_i = w_i * exp(-α_t)

Normalize the sample weights so that they sum up to 1.
Step - 7 : Combine weak learners into a strong classifier: - Combine the predictions of all weak learners using their weights (α_t) obtained in Step 5. - The final classifier is a weighted sum of weak learner predictions:

F(x) = Σ (α_t * h_t(x))

where F(x) is the output of the final classifier, h_t(x) is the prediction of the weak learner t for input x, and α_t is the weight of the weak learner t.

Step - 8 : Make predictions: - To make predictions for new data, pass the input through the combined weak learners and their respective weights as calculated in Step 7. - The final prediction is the class with the highest weighted vote.

AdaBoost is an iterative algorithm, and it keeps improving its predictions in each iteration by focusing on the previously misclassified samples. The final classifier is a weighted combination of weak learners that are good at handling different regions of the feature space, resulting in improved generalization and accuracy.