Cover 1
Copyright 3
Contributors 6
Table of Contents 8
Preface 14
Chapter 1: Introduction to Ensemble Techniques 20
Datasets 23
Hypothyroid 24
Waveform 25
German Credit 26
Iris 27
Pima Indians Diabetes 27
US Crime 28
Overseas visitors 29
Primary Biliary Cirrhosis 31
Multishapes 31
Board Stiffness 33
Statistical/machine learning models 33
Logistic regression model 34
Logistic regression for hypothyroid classification 35
Neural networks 36
Neural network for hypothyroid classification 38
Naïve Bayes classifier 39
Naïve Bayes for hypothyroid classification 39
Decision tree 39
Decision tree for hypothyroid classification 40
Support vector machines 41
SVM for hypothyroid classification 42
The right model dilemma! 42
An ensemble purview 45
Complementary statistical tests 49
Permutation test 49
Chi-square and McNemar test 51
ROC test 53
Summary 54
Chapter 2: Bootstrapping 56
Technical requirements 57
The jackknife technique 57
The jackknife method for mean and variance 58
Pseudovalues method for survival data 61
Bootstrap – a statistical method 64
The standard error of correlation coefficient 66
The parametric bootstrap 71
Eigen values 74
Rule of thumb 78
The boot package 78
Bootstrap and testing hypotheses 81
Bootstrapping regression models 83
Bootstrapping survival models* 86
Bootstrapping time series models* 88
Summary 89
Chapter 3: Bagging 90
Technical requirements 90
Classification trees and pruning 91
Bagging 95
k-NN classifier 105
Analyzing waveform data 106
k-NN bagging 107
Summary 110
Chapter 4: Random Forests 112
Technical requirements 113
Random Forests 113
Variable importance 119
Proximity plots 124
Random Forest nuances 126
Comparisons with bagging 132
Missing data imputation 134
Clustering with Random Forest 137
Summary 138
Chapter 5: The Bare Bones
Boosting Algorithms 140
Technical requirements 141
The general boosting algorithm 142
Adaptive boosting 143
Gradient boosting 153
Building it from scratch 154
Squared-error loss function 159
Using the adabag and gbm packages 161
Variable importance 164
Comparing bagging, random forests,
and boosting 164
Summary 166
Chapter 6: Boosting Refinements 168
Technical requirements 168
Why does boosting work? 169
The gbm package 173
Boosting for count data 176
Boosting for survival data 181
The xgboost package 182
The h2o package 188
Summary 190
Chapter 7: The General Ensemble Technique 192
Technical requirements 193
Why does ensembling work? 193
Ensembling by voting 204
Majority voting 205
Weighted voting 206
Ensembling by averaging 209
Simple averaging 211
Weight averaging 213
Stack ensembling 214
Summary 216
Chapter 8: Ensemble Diagnostics 218
Technical requirements 218
What is ensemble diagnostics? 219
Ensemble diversity 221
Numeric prediction 221
Class prediction 223
Pairwise measure 229
Disagreement measure 230
Yule's or Q-statistic 231
Correlation coefficient measure 232
Cohen's statistic 233
Double-fault measure 235
Interrating agreement 236
Entropy measure 236
Kohavi-Wolpert measure 237
Disagreement measure for ensemble 239
Measurement of interrater agreement 240
Summary 241
Chapter 9: Ensembling Regression Models 242
Technical requirements 243
Pre-processing the housing data 243
Visualization and variable reduction 250
Variable clustering 257
Regression models 261
Linear regression model 261
Neural networks 272
Regression tree 274
Prediction for regression models 276
Bagging and Random Forests 278
Boosting regression models 282
Stacking methods for regression models 284
Summary 286
Chapter 10: Ensembling Survival Models 288
Core concepts of survival analysis 289
Nonparametric inference 297
Regression models – parametric and Cox proportional hazards models 303
Survival tree 311
Ensemble survival models 314
Summary 317
Chapter 11: Ensembling Time
Series Models 318
Technical requirements 319
Time series datasets 319
AirPassengers 320
co2 321
uspop 321
gas 321
Car Sales 321
austres 321
WWWusage 321
Time series visualization 322
Core concepts and metrics 329
Essential time series models 337
Naïve forecasting 337
Seasonal, trend, and loess fitting 338
Exponential smoothing state space model 341
Auto-regressive Integrated Moving Average (ARIMA) models 343
Auto-regressive neural networks 346
Messing it all up 347
Bagging and time series 349
Ensemble time series models 351
Summary 352
Chapter 12: What's Next? 354
Bibliography 358
References 358
R package references 362
Other Books You May Enjoy 368
Index 372

Adobe InDesign CS6 (Windows)

producers:
Adobe PDF Library 10.0.1

2024-12-23