https://www.youtube.com/watch?v=LsK-xG1cLYA&ab_channel=StatQuestwithJoshStarmer

Main idea 1

Short trees with AdaBoost: In Random Forrest , each time you make a tree, you make a full sized tree. Some trees are bigger than other, but there is no predetermined maximum depth. In contrast, in a Forest of Trees made with AdaBoost, the trees are usually just a node and two leaves (=stump). In other words: AdaBoost combines a lot of weak learners which are almost always stumps.

Stumps are technically “weak learners”, because they only use one variable to make a decision

Main idea 2

In Random Forrest, each tree has an equal vote on the final classification.

AdaBoost: Some stumps get more to say than others!

Main idea 3

In Random Forrest, each tree is made independently of the others. In contrast, in a Forest of Trees made with AdaBoost the trees are not independend; Each stump is made by taking the previous stumps into account.

Building first stump (with Gini index)

Im wesentlichen schauen wir welches Feauture die beste Vorraussetzung machen. Das ist dann unseres Feauture für den ersten Stump. Basierend auf der Gleichung “Amount of Stump” skalieren wir diesen Stump. Ist er besonders Aussagekräftig, wollen wir ihn auch besonders hoch skalieren. Wenn er die Gegenteiliege Aussage macht, können wir ihn auch negativ skalieren.