fitcauto
Автоматический выбор классификационной модели с оптимизированными гиперпараметрами
Синтаксис
Описание
Данные предиктора и отклика, fitcauto автоматически пытается выбрать типы классификационных моделей с различными значениями гиперзначений параметров. Функция использует байесовскую оптимизацию, чтобы выбрать модели и их значения гиперзначений параметров, и вычисляет ошибку классификации перекрестной валидации для каждой модели. После завершения оптимизации, fitcauto возвращает модель, обученную на целом наборе данных, которая, как ожидается, лучше всего классифицирует новые данные. Вы можете использовать predict и loss функции объекта возвращенной модели для классификации новых данных и вычисления ошибки классификации тестового набора, соответственно.
Использовать fitcauto когда вы не уверены, какие типы классификаторов лучше всего соответствуют вашим данным. Для получения информации об альтернативных методах настройки гиперпараметров классификационных моделей, см. «Альтернативная функциональность».
Mdl = fitcauto(Tbl,ResponseVarName)Mdl с настроенными гиперпараметрами. Таблица Tbl содержит переменные предиктора и переменную отклика, где ResponseVarName - имя переменной отклика.
Mdl = fitcauto(___,Name,Value)HyperparameterOptimizationOptions аргумент пары "имя-значение", чтобы задать, как выполняется байесовская оптимизация.
[ дополнительно возвращается Mdl,OptimizationResults] = fitcauto(___)OptimizationResults, а BayesianOptimization объект, содержащий результаты выбора модели и процесса настройки гиперпараметра.
Примеры
Автоматический выбор классификатора с использованием табличных данных
Использование fitcauto чтобы автоматически выбрать классификационную модель с оптимизированными гиперпараметрами, данные предиктора и отклика, сохраненные в таблице.
Загрузка данных
Загрузите carbig набор данных, содержащий измерения автомобилей 1970-х и начала 1980-х годов.
load carbigКлассифицировать автомобили исходя из того, были ли они произведены в США.
Origin = categorical(cellstr(Origin));
Origin = mergecats(Origin,{'France','Japan','Germany', ...
'Sweden','Italy','England'},'NotUSA');Создайте таблицу, содержащую переменные предиктора Acceleration, Displacement, и так далее, а также переменная отклика Origin.
cars = table(Acceleration,Displacement,Horsepower, ...
Model_Year,MPG,Weight,Origin);Данные о разделах
Разделите данные на обучающие и тестовые наборы. Используйте приблизительно 80% наблюдений для процесса выбора модели и настройки гиперпараметра и 20% наблюдений, чтобы проверить эффективность конечной модели, возвращенной fitcauto. Использование cvpartition для разбиения данных на разделы.
rng('default') % For reproducibility of the data partition c = cvpartition(Origin,'Holdout',0.2); trainingIdx = training(c); % Training set indices carsTrain = cars(trainingIdx,:); testIdx = test(c); % Test set indices carsTest = cars(testIdx,:);
Выполняйте fitcauto
Передайте обучающие данные в fitcauto. По умолчанию fitcauto определяет соответствующие типы модели, чтобы попробовать, использует байесовскую оптимизацию, чтобы найти хорошие значения гиперзначений параметров, и возвращает обученную модель Mdl с наилучшей ожидаемой эффективностью. Кроме того, fitcauto предоставляет график оптимизации и итерационное отображение результатов оптимизации. Для получения дополнительной информации о том, как интерпретировать эти результаты, смотрите Подробное отображение.
Ожидайте, что этот процесс займет некоторое время. Чтобы ускорить процесс оптимизации, рассмотрите установку, чтобы запустить оптимизацию параллельно, если у вас есть лицензия Parallel Computing Toolbox™. Для этого передайте 'HyperparameterOptimizationOptions',struct('UseParallel',true) на fitcauto как аргумент пары "имя-значение".
Mdl = fitcauto(carsTrain,'Origin');Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Learner types to explore: ensemble, knn, nb, svm, tree Total iterations (MaxObjectiveEvaluations): 150 Total time (MaxTime): Inf |=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 1 | Best | 0.14154 | 10.563 | 0.14154 | 0.14154 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 201 | | | | | | | | | MinLeafSize: 7 |
| 2 | Accept | 0.18269 | 0.57392 | 0.14154 | 0.14154 | knn | NumNeighbors: 3 |
| 3 | Accept | 0.23397 | 0.1264 | 0.14154 | 0.14154 | knn | NumNeighbors: 91 |
| 4 | Accept | 0.16308 | 12.867 | 0.14154 | 0.15468 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 274 | | | | | | | | | MinLeafSize: 15 |
| 5 | Accept | 0.20833 | 0.124 | 0.14154 | 0.15468 | knn | NumNeighbors: 4 |
| 6 | Accept | 0.22115 | 0.079641 | 0.14154 | 0.15468 | knn | NumNeighbors: 28 |
| 7 | Accept | 0.16923 | 0.20013 | 0.14154 | 0.15468 | tree | MinLeafSize: 105 |
| 8 | Accept | 0.37179 | 0.59222 | 0.14154 | 0.15468 | svm | BoxConstraint: 0.022186 | | | | | | | | | KernelScale: 0.085527 |
| 9 | Accept | 0.37179 | 0.11659 | 0.14154 | 0.15468 | svm | BoxConstraint: 0.045899 | | | | | | | | | KernelScale: 0.0024758 |
| 10 | Accept | 0.24615 | 0.98386 | 0.14154 | 0.15468 | nb | DistributionNames: kernel | | | | | | | | | Width: 1.1327 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 11 | Accept | 0.16923 | 0.079996 | 0.14154 | 0.15468 | tree | MinLeafSize: 78 |
| 12 | Accept | 0.26923 | 0.10923 | 0.14154 | 0.15468 | svm | BoxConstraint: 11.063 | | | | | | | | | KernelScale: 15.114 |
| 13 | Best | 0.12923 | 0.11568 | 0.12923 | 0.15468 | tree | MinLeafSize: 3 |
| 14 | Accept | 0.21154 | 0.084406 | 0.12923 | 0.15468 | knn | NumNeighbors: 2 |
| 15 | Accept | 0.14154 | 0.080022 | 0.12923 | 0.15294 | tree | MinLeafSize: 1 |
| 16 | Accept | 0.14769 | 0.092395 | 0.12923 | 0.15097 | tree | MinLeafSize: 2 |
| 17 | Accept | 0.14154 | 10.869 | 0.12923 | 0.14872 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 208 | | | | | | | | | MinLeafSize: 10 |
| 18 | Accept | 0.37179 | 0.12386 | 0.12923 | 0.14872 | svm | BoxConstraint: 116.46 | | | | | | | | | KernelScale: 0.52908 |
| 19 | Accept | 0.22769 | 0.15545 | 0.12923 | 0.14872 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 20 | Accept | 0.22115 | 0.070813 | 0.12923 | 0.14872 | knn | NumNeighbors: 8 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 21 | Accept | 0.37179 | 0.11553 | 0.12923 | 0.14872 | svm | BoxConstraint: 45.341 | | | | | | | | | KernelScale: 0.76949 |
| 22 | Accept | 0.12923 | 0.080362 | 0.12923 | 0.14194 | tree | MinLeafSize: 3 |
| 23 | Best | 0.10154 | 0.079656 | 0.10154 | 0.13213 | tree | MinLeafSize: 5 |
| 24 | Accept | 0.22769 | 0.2529 | 0.10154 | 0.13213 | nb | DistributionNames: kernel | | | | | | | | | Width: 0.42571 |
| 25 | Accept | 0.11385 | 0.080085 | 0.10154 | 0.1289 | tree | MinLeafSize: 11 |
| 26 | Accept | 0.13782 | 0.092228 | 0.10154 | 0.1289 | svm | BoxConstraint: 9.7286 | | | | | | | | | KernelScale: 293.41 |
| 27 | Accept | 0.22769 | 0.073346 | 0.10154 | 0.1289 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 28 | Accept | 0.21795 | 0.074914 | 0.10154 | 0.1289 | knn | NumNeighbors: 42 |
| 29 | Accept | 0.24308 | 0.27621 | 0.10154 | 0.1289 | nb | DistributionNames: kernel | | | | | | | | | Width: 4.4662 |
| 30 | Accept | 0.16308 | 12.328 | 0.10154 | 0.1289 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 267 | | | | | | | | | MinLeafSize: 131 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 31 | Accept | 0.24308 | 0.22334 | 0.10154 | 0.1289 | nb | DistributionNames: kernel | | | | | | | | | Width: 0.66296 |
| 32 | Accept | 0.22115 | 0.066711 | 0.10154 | 0.1289 | knn | NumNeighbors: 28 |
| 33 | Accept | 0.13846 | 0.079934 | 0.10154 | 0.12465 | tree | MinLeafSize: 25 |
| 34 | Accept | 0.21474 | 0.085438 | 0.10154 | 0.12465 | knn | NumNeighbors: 14 |
| 35 | Accept | 0.16615 | 10.05 | 0.10154 | 0.12465 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 215 | | | | | | | | | MinLeafSize: 13 |
| 36 | Accept | 0.14154 | 12.866 | 0.10154 | 0.12465 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 254 | | | | | | | | | MinLeafSize: 31 |
| 37 | Accept | 0.22769 | 0.070251 | 0.10154 | 0.12465 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 38 | Accept | 0.37179 | 0.077924 | 0.10154 | 0.12465 | svm | BoxConstraint: 0.0073633 | | | | | | | | | KernelScale: 774.33 |
| 39 | Accept | 0.16923 | 0.068411 | 0.10154 | 0.12552 | tree | MinLeafSize: 82 |
| 40 | Accept | 0.20833 | 0.064563 | 0.10154 | 0.12552 | knn | NumNeighbors: 4 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 41 | Accept | 0.16308 | 12.932 | 0.10154 | 0.12552 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 274 | | | | | | | | | MinLeafSize: 150 |
| 42 | Accept | 0.22462 | 0.24365 | 0.10154 | 0.12552 | nb | DistributionNames: kernel | | | | | | | | | Width: 121.64 |
| 43 | Accept | 0.20308 | 11.027 | 0.10154 | 0.12552 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 229 | | | | | | | | | MinLeafSize: 117 |
| 44 | Accept | 0.16923 | 0.069279 | 0.10154 | 0.12291 | tree | MinLeafSize: 84 |
| 45 | Accept | 0.22769 | 0.078716 | 0.10154 | 0.12291 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 46 | Accept | 0.22769 | 0.068458 | 0.10154 | 0.12291 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 47 | Accept | 0.16615 | 9.9849 | 0.10154 | 0.12291 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 212 | | | | | | | | | MinLeafSize: 49 |
| 48 | Accept | 0.14769 | 14.541 | 0.10154 | 0.12291 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 288 | | | | | | | | | MinLeafSize: 25 |
| 49 | Accept | 0.23077 | 0.21379 | 0.10154 | 0.12291 | nb | DistributionNames: kernel | | | | | | | | | Width: 73.249 |
| 50 | Accept | 0.37179 | 0.091937 | 0.10154 | 0.12291 | svm | BoxConstraint: 0.0036501 | | | | | | | | | KernelScale: 1.0504 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 51 | Accept | 0.21474 | 0.098808 | 0.10154 | 0.12291 | svm | BoxConstraint: 64.859 | | | | | | | | | KernelScale: 23.779 |
| 52 | Accept | 0.37179 | 0.10415 | 0.10154 | 0.12291 | svm | BoxConstraint: 0.16622 | | | | | | | | | KernelScale: 4.4901 |
| 53 | Accept | 0.25846 | 0.2444 | 0.10154 | 0.12291 | nb | DistributionNames: kernel | | | | | | | | | Width: 0.079498 |
| 54 | Accept | 0.21154 | 0.074835 | 0.10154 | 0.12291 | knn | NumNeighbors: 2 |
| 55 | Accept | 0.13846 | 12.173 | 0.10154 | 0.12291 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 234 | | | | | | | | | MinLeafSize: 8 |
| 56 | Accept | 0.36538 | 0.10958 | 0.10154 | 0.12291 | svm | BoxConstraint: 271.6 | | | | | | | | | KernelScale: 2.743 |
| 57 | Accept | 0.16615 | 11.482 | 0.10154 | 0.12291 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 248 | | | | | | | | | MinLeafSize: 117 |
| 58 | Accept | 0.37179 | 0.095419 | 0.10154 | 0.12291 | svm | BoxConstraint: 7.5785 | | | | | | | | | KernelScale: 0.0066815 |
| 59 | Accept | 0.37179 | 0.097469 | 0.10154 | 0.12291 | svm | BoxConstraint: 0.0017765 | | | | | | | | | KernelScale: 0.86786 |
| 60 | Accept | 0.37179 | 0.11284 | 0.10154 | 0.12291 | svm | BoxConstraint: 0.011465 | | | | | | | | | KernelScale: 0.02747 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 61 | Accept | 0.11692 | 0.077515 | 0.10154 | 0.12239 | tree | MinLeafSize: 12 |
| 62 | Accept | 0.29167 | 0.091617 | 0.10154 | 0.12239 | svm | BoxConstraint: 11.939 | | | | | | | | | KernelScale: 11.002 |
| 63 | Accept | 0.21795 | 0.067171 | 0.10154 | 0.12239 | knn | NumNeighbors: 6 |
| 64 | Accept | 0.18269 | 0.062887 | 0.10154 | 0.12239 | knn | NumNeighbors: 3 |
| 65 | Accept | 0.12923 | 0.075704 | 0.10154 | 0.11989 | tree | MinLeafSize: 3 |
| 66 | Accept | 0.16923 | 0.065889 | 0.10154 | 0.12048 | tree | MinLeafSize: 56 |
| 67 | Accept | 0.1891 | 0.068215 | 0.10154 | 0.12048 | knn | NumNeighbors: 1 |
| 68 | Accept | 0.13231 | 14.135 | 0.10154 | 0.12048 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 270 | | | | | | | | | MinLeafSize: 4 |
| 69 | Accept | 0.22769 | 0.060902 | 0.10154 | 0.12048 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 70 | Accept | 0.37231 | 0.24511 | 0.10154 | 0.12048 | nb | DistributionNames: kernel | | | | | | | | | Width: 1629.5 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 71 | Accept | 0.16923 | 0.069135 | 0.10154 | 0.11947 | tree | MinLeafSize: 61 |
| 72 | Accept | 0.22769 | 0.060552 | 0.10154 | 0.11947 | nb | DistributionNames: normal | | | | | | | | | Width: NaN |
| 73 | Accept | 0.16308 | 10.133 | 0.10154 | 0.11947 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 217 | | | | | | | | | MinLeafSize: 70 |
| 74 | Accept | 0.13231 | 13.055 | 0.10154 | 0.11947 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 257 | | | | | | | | | MinLeafSize: 2 |
| 75 | Accept | 0.21474 | 0.069312 | 0.10154 | 0.11947 | knn | NumNeighbors: 49 |
| 76 | Accept | 0.13846 | 0.083714 | 0.10154 | 0.1214 | tree | MinLeafSize: 25 |
| 77 | Accept | 0.12 | 0.069041 | 0.10154 | 0.11923 | tree | MinLeafSize: 6 |
| 78 | Accept | 0.10154 | 0.077399 | 0.10154 | 0.1118 | tree | MinLeafSize: 5 |
| 79 | Accept | 0.12 | 0.076269 | 0.10154 | 0.11007 | tree | MinLeafSize: 4 |
| 80 | Accept | 0.10154 | 0.09325 | 0.10154 | 0.10878 | tree | MinLeafSize: 5 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 81 | Accept | 0.10154 | 0.074541 | 0.10154 | 0.10737 | tree | MinLeafSize: 5 |
| 82 | Accept | 0.12 | 0.069929 | 0.10154 | 0.1063 | tree | MinLeafSize: 4 |
| 83 | Accept | 0.10154 | 0.072591 | 0.10154 | 0.10514 | tree | MinLeafSize: 5 |
| 84 | Accept | 0.10154 | 0.071254 | 0.10154 | 0.10366 | tree | MinLeafSize: 5 |
| 85 | Accept | 0.10154 | 0.077378 | 0.10154 | 0.10361 | tree | MinLeafSize: 5 |
| 86 | Accept | 0.10154 | 0.070643 | 0.10154 | 0.10348 | tree | MinLeafSize: 5 |
| 87 | Accept | 0.12 | 0.070551 | 0.10154 | 0.10286 | tree | MinLeafSize: 4 |
| 88 | Accept | 0.12 | 0.078438 | 0.10154 | 0.1029 | tree | MinLeafSize: 6 |
| 89 | Accept | 0.10154 | 0.072996 | 0.10154 | 0.10262 | tree | MinLeafSize: 5 |
| 90 | Accept | 0.10154 | 0.078162 | 0.10154 | 0.10246 | tree | MinLeafSize: 5 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 91 | Accept | 0.10154 | 0.07038 | 0.10154 | 0.10267 | tree | MinLeafSize: 5 |
| 92 | Accept | 0.10154 | 0.07752 | 0.10154 | 0.10257 | tree | MinLeafSize: 5 |
| 93 | Accept | 0.10154 | 0.076155 | 0.10154 | 0.10217 | tree | MinLeafSize: 5 |
| 94 | Accept | 0.10154 | 0.075983 | 0.10154 | 0.10221 | tree | MinLeafSize: 5 |
| 95 | Accept | 0.10154 | 0.07407 | 0.10154 | 0.10211 | tree | MinLeafSize: 5 |
| 96 | Accept | 0.10154 | 0.080633 | 0.10154 | 0.10207 | tree | MinLeafSize: 5 |
| 97 | Accept | 0.10154 | 0.086164 | 0.10154 | 0.10205 | tree | MinLeafSize: 5 |
| 98 | Accept | 0.10154 | 0.080264 | 0.10154 | 0.10191 | tree | MinLeafSize: 5 |
| 99 | Accept | 0.12308 | 0.076015 | 0.10154 | 0.1021 | tree | MinLeafSize: 17 |
| 100 | Accept | 0.10154 | 0.074579 | 0.10154 | 0.1019 | tree | MinLeafSize: 5 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 101 | Accept | 0.10154 | 0.072077 | 0.10154 | 0.10186 | tree | MinLeafSize: 5 |
| 102 | Accept | 0.10154 | 0.071287 | 0.10154 | 0.10199 | tree | MinLeafSize: 5 |
| 103 | Accept | 0.10154 | 0.080003 | 0.10154 | 0.10186 | tree | MinLeafSize: 5 |
| 104 | Accept | 0.12 | 0.07462 | 0.10154 | 0.10189 | tree | MinLeafSize: 13 |
| 105 | Accept | 0.10154 | 0.077244 | 0.10154 | 0.10198 | tree | MinLeafSize: 5 |
| 106 | Accept | 0.12 | 0.080302 | 0.10154 | 0.10173 | tree | MinLeafSize: 4 |
| 107 | Accept | 0.10154 | 0.071858 | 0.10154 | 0.10183 | tree | MinLeafSize: 5 |
| 108 | Accept | 0.10154 | 0.076013 | 0.10154 | 0.10166 | tree | MinLeafSize: 5 |
| 109 | Accept | 0.10154 | 0.079106 | 0.10154 | 0.10164 | tree | MinLeafSize: 5 |
| 110 | Accept | 0.10154 | 0.075807 | 0.10154 | 0.10172 | tree | MinLeafSize: 5 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 111 | Accept | 0.10154 | 0.076903 | 0.10154 | 0.10159 | tree | MinLeafSize: 5 |
| 112 | Accept | 0.10154 | 0.077409 | 0.10154 | 0.10163 | tree | MinLeafSize: 5 |
| 113 | Accept | 0.10154 | 0.069569 | 0.10154 | 0.10165 | tree | MinLeafSize: 5 |
| 114 | Accept | 0.12308 | 0.076744 | 0.10154 | 0.10156 | tree | MinLeafSize: 18 |
| 115 | Accept | 0.19551 | 0.097443 | 0.10154 | 0.10156 | svm | BoxConstraint: 1.2977 | | | | | | | | | KernelScale: 33.654 |
| 116 | Accept | 0.26603 | 0.099474 | 0.10154 | 0.10156 | svm | BoxConstraint: 0.082725 | | | | | | | | | KernelScale: 139.98 |
| 117 | Accept | 0.37179 | 0.077589 | 0.10154 | 0.10156 | svm | BoxConstraint: 0.0065156 | | | | | | | | | KernelScale: 559.53 |
| 118 | Accept | 0.15385 | 0.08281 | 0.10154 | 0.10156 | svm | BoxConstraint: 8.5337 | | | | | | | | | KernelScale: 482.07 |
| 119 | Accept | 0.20833 | 0.074271 | 0.10154 | 0.10156 | svm | BoxConstraint: 5.1729 | | | | | | | | | KernelScale: 980.38 |
| 120 | Accept | 0.17949 | 0.081488 | 0.10154 | 0.10156 | svm | BoxConstraint: 6.8028 | | | | | | | | | KernelScale: 578.14 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 121 | Accept | 0.22115 | 0.076901 | 0.10154 | 0.10156 | svm | BoxConstraint: 0.19345 | | | | | | | | | KernelScale: 367.75 |
| 122 | Accept | 0.17308 | 0.076171 | 0.10154 | 0.10156 | svm | BoxConstraint: 10.344 | | | | | | | | | KernelScale: 679.05 |
| 123 | Accept | 0.125 | 0.09187 | 0.10154 | 0.10156 | svm | BoxConstraint: 72.626 | | | | | | | | | KernelScale: 228.42 |
| 124 | Accept | 0.13462 | 0.15746 | 0.10154 | 0.10156 | svm | BoxConstraint: 586.5 | | | | | | | | | KernelScale: 176.02 |
| 125 | Accept | 0.13462 | 0.093366 | 0.10154 | 0.10156 | svm | BoxConstraint: 83.771 | | | | | | | | | KernelScale: 117.21 |
| 126 | Accept | 0.22436 | 0.10124 | 0.10154 | 0.10156 | svm | BoxConstraint: 962.52 | | | | | | | | | KernelScale: 20.898 |
| 127 | Accept | 0.15705 | 0.1133 | 0.10154 | 0.10156 | svm | BoxConstraint: 29.038 | | | | | | | | | KernelScale: 66.563 |
| 128 | Accept | 0.16346 | 0.10388 | 0.10154 | 0.10156 | svm | BoxConstraint: 156.8 | | | | | | | | | KernelScale: 62.775 |
| 129 | Accept | 0.13782 | 0.087555 | 0.10154 | 0.10156 | svm | BoxConstraint: 94.357 | | | | | | | | | KernelScale: 932.27 |
| 130 | Accept | 0.12821 | 0.093318 | 0.10154 | 0.10156 | svm | BoxConstraint: 25.982 | | | | | | | | | KernelScale: 247.06 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 131 | Accept | 0.13462 | 0.08648 | 0.10154 | 0.10156 | svm | BoxConstraint: 16.818 | | | | | | | | | KernelScale: 352.46 |
| 132 | Accept | 0.14103 | 0.090667 | 0.10154 | 0.10156 | svm | BoxConstraint: 15.817 | | | | | | | | | KernelScale: 130.15 |
| 133 | Accept | 0.12179 | 0.089208 | 0.10154 | 0.10156 | svm | BoxConstraint: 74.054 | | | | | | | | | KernelScale: 555.66 |
| 134 | Accept | 0.21474 | 0.082634 | 0.10154 | 0.10156 | svm | BoxConstraint: 0.80097 | | | | | | | | | KernelScale: 239.08 |
| 135 | Accept | 0.125 | 0.088947 | 0.10154 | 0.10156 | svm | BoxConstraint: 47.244 | | | | | | | | | KernelScale: 214.51 |
| 136 | Accept | 0.13141 | 0.07954 | 0.10154 | 0.10156 | svm | BoxConstraint: 13.389 | | | | | | | | | KernelScale: 145.55 |
| 137 | Accept | 0.13782 | 0.091167 | 0.10154 | 0.10156 | svm | BoxConstraint: 60.755 | | | | | | | | | KernelScale: 131.88 |
| 138 | Accept | 0.125 | 0.087338 | 0.10154 | 0.10156 | svm | BoxConstraint: 11.454 | | | | | | | | | KernelScale: 176.64 |
| 139 | Accept | 0.125 | 0.080567 | 0.10154 | 0.10156 | svm | BoxConstraint: 11.85 | | | | | | | | | KernelScale: 160.69 |
| 140 | Accept | 0.16667 | 0.083852 | 0.10154 | 0.10156 | svm | BoxConstraint: 11.155 | | | | | | | | | KernelScale: 652.28 |
|=================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=================================================================================================================================| | 141 | Accept | 0.125 | 0.089769 | 0.10154 | 0.10156 | svm | BoxConstraint: 10.933 | | | | | | | | | KernelScale: 172.97 |
| 142 | Accept | 0.21474 | 0.1003 | 0.10154 | 0.10156 | svm | BoxConstraint: 16.172 | | | | | | | | | KernelScale: 22.373 |
| 143 | Accept | 0.13782 | 0.0833 | 0.10154 | 0.10156 | svm | BoxConstraint: 7.6332 | | | | | | | | | KernelScale: 178.51 |
| 144 | Accept | 0.13462 | 0.10562 | 0.10154 | 0.10156 | svm | BoxConstraint: 6.7646 | | | | | | | | | KernelScale: 160.72 |
| 145 | Accept | 0.14103 | 0.076631 | 0.10154 | 0.10156 | svm | BoxConstraint: 6.8617 | | | | | | | | | KernelScale: 189.52 |
| 146 | Accept | 0.15385 | 0.082538 | 0.10154 | 0.10156 | svm | BoxConstraint: 6.0384 | | | | | | | | | KernelScale: 332.89 |
| 147 | Accept | 0.14423 | 0.077025 | 0.10154 | 0.10156 | svm | BoxConstraint: 5.7255 | | | | | | | | | KernelScale: 182.19 |
| 148 | Accept | 0.13782 | 0.089308 | 0.10154 | 0.10156 | svm | BoxConstraint: 70.212 | | | | | | | | | KernelScale: 818.46 |
| 149 | Accept | 0.14103 | 0.08348 | 0.10154 | 0.10156 | svm | BoxConstraint: 5.493 | | | | | | | | | KernelScale: 230.68 |
| 150 | Accept | 0.13782 | 0.079744 | 0.10154 | 0.10156 | svm | BoxConstraint: 5.3564 | | | | | | | | | KernelScale: 111.96 |
__________________________________________________________ Optimization completed. Total iterations: 150 Total elapsed time: 835.2167 seconds Total time for training and validation: 193.4979 seconds Best observed learner is a tree model with: MinLeafSize: 5 Observed validation loss: 0.10154 Time for training and validation: 0.079656 seconds Best estimated learner (returned model) is a tree model with: MinLeafSize: 5 Estimated validation loss: 0.10156 Estimated time for training and validation: 0.076205 seconds Documentation for fitcauto display
Конечная модель, возвращенная fitcauto соответствует лучшему оцененному ученику. Перед возвращением модели функция переобучает ее, используя все обучающие данные (carsTrain), перечисленные Learner (или модель) тип и отображенные значения гиперзначений параметров.
Оценка эффективности тестового набора
Оцените эффективность модели на тестовом наборе.
testAccuracy = 1 - loss(Mdl,carsTest,'Origin')testAccuracy = 0.9143
confusionchart(carsTest.Origin,predict(Mdl,carsTest))
Автоматический выбор классификатора с использованием матричных данных
Использование fitcauto чтобы автоматически выбрать классификационную модель с оптимизированными гиперпараметрами, данные предиктора и отклика, сохраненные в отдельных переменных.
Загрузка данных
Загрузите humanactivity набор данных. Этот набор данных содержит 24 075 наблюдений за пятью физическими действиями человека: Сидя (1), Стоя (2), Ходьба (3), Бегом (4) и Танцами (5). Каждое наблюдение имеет 60 функции, извлеченных из данных об ускорении, измеренных датчиками акселерометра смартфона. Переменная feat содержит матрицу данных предиктора из 60 функций для 24 075 наблюдений и переменную отклика actid содержит идентификаторы действия для наблюдений в виде целых чисел.
load humanactivityДанные о разделах
Разделите данные на обучающие и тестовые наборы. Используйте 90% наблюдений для выбора модели и 10% наблюдений для валидации окончательной модели, возвращенной fitcauto. Использование cvpartition зарезервировать 10% наблюдений для проверки.
rng('default') % For reproducibility of the partition c = cvpartition(actid,'Holdout',0.10); trainingIndices = training(c); % Indices for the training set XTrain = feat(trainingIndices,:); YTrain = actid(trainingIndices); testIndices = test(c); % Indices for the test set XTest = feat(testIndices,:); YTest = actid(testIndices);
Выполняйте fitcauto
Передайте обучающие данные в fitcauto. По умолчанию fitcauto определяет соответствующие типы модели (или обучающегося), чтобы попробовать, использует байесовскую оптимизацию, чтобы найти хорошие значения гиперзначений параметров для этих моделей, и возвращает обученную модель с наилучшей ожидаемой эффективностью. Задайте, чтобы запустить оптимизацию параллельно (требует Parallel Computing Toolbox™). Верните второй выход OptimizationResults который содержит детали байесовской оптимизации.
Ожидайте, что этот процесс выбора модели займет некоторое время. По умолчанию fitcauto предоставляет график оптимизации и итерационное отображение результатов оптимизации. Для получения дополнительной информации о том, как интерпретировать эти результаты, смотрите Подробное отображение.
options = struct('UseParallel',true); [Mdl,OptimizationResults] = fitcauto(XTrain,YTrain,'HyperparameterOptimizationOptions',options);
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 6). Copying objective function to workers... Done copying objective function to workers.
Learner types to explore: ensemble, knn, nb, svm, tree Total iterations (MaxObjectiveEvaluations): 150 Total time (MaxTime): Inf
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 1 | 6 | Best | 0.28088 | 48.752 | 0.28088 | 0.28088 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.22686 | | | | | | | | | | KernelScale: 330.4 |
| 2 | 6 | Best | 0.036459 | 51.455 | 0.036459 | 0.036459 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 254 | | | | | | | | | | MinLeafSize: 1786 | | | | | | | | | | MaxNumSplits: 12 |
| 3 | 6 | Best | 0.025845 | 5.1379 | 0.025845 | 0.025845 | tree | MinLeafSize: 59 |
| 4 | 6 | Best | 0.006415 | 60.999 | 0.006415 | 0.021738 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 214 | | | | | | | | | | MinLeafSize: 5 | | | | | | | | | | MaxNumSplits: 23 |
| 5 | 6 | Accept | 0.025845 | 4.8823 | 0.006415 | 0.021738 | tree | MinLeafSize: 59 |
| 6 | 6 | Accept | 0.017768 | 5.4601 | 0.006415 | 0.021738 | tree | MinLeafSize: 9 |
| 7 | 6 | Accept | 0.050212 | 1.1024 | 0.006415 | 0.021738 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 8 | 6 | Accept | 0.050212 | 0.64183 | 0.006415 | 0.021738 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 9 | 6 | Accept | 0.019568 | 151.32 | 0.006415 | 0.021152 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 218 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 63 |
| 10 | 6 | Accept | 0.026537 | 165.42 | 0.006415 | 0.022035 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 264 | | | | | | | | | | MinLeafSize: 7 | | | | | | | | | | MaxNumSplits: 36 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 11 | 6 | Accept | 0.59166 | 29.107 | 0.006415 | 0.022035 | nb | DistributionNames: kernel | | | | | | | | | | Width: 4.7212e-14 |
| 12 | 6 | Accept | 0.021645 | 50.626 | 0.006415 | 0.022122 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 243 | | | | | | | | | | MinLeafSize: 1247 | | | | | | | | | | MaxNumSplits: 45 |
| 13 | 6 | Accept | 0.043567 | 24.583 | 0.006415 | 0.022122 | knn | NumNeighbors: 144 |
| 14 | 6 | Accept | 0.028844 | 22.503 | 0.006415 | 0.022122 | knn | NumNeighbors: 18 |
| 15 | 6 | Accept | 0.04389 | 157.12 | 0.006415 | 0.022122 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.068467 | | | | | | | | | | KernelScale: 117.01 |
| 16 | 6 | Accept | 0.024598 | 20.721 | 0.006415 | 0.022122 | knn | NumNeighbors: 7 |
| 17 | 6 | Accept | 0.03009 | 20.969 | 0.006415 | 0.022122 | knn | NumNeighbors: 27 |
| 18 | 6 | Accept | 0.016753 | 6.5065 | 0.006415 | 0.021547 | tree | MinLeafSize: 2 |
| 19 | 6 | Accept | 0.040059 | 4.3496 | 0.006415 | 0.022122 | tree | MinLeafSize: 166 |
| 20 | 6 | Accept | 0.060319 | 2.2673 | 0.006415 | 0.022122 | tree | MinLeafSize: 1881 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 21 | 6 | Accept | 0.050212 | 0.97596 | 0.006415 | 0.022122 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 22 | 6 | Accept | 0.036552 | 20.775 | 0.006415 | 0.022122 | knn | NumNeighbors: 67 |
| 23 | 6 | Accept | 0.050212 | 0.55594 | 0.006415 | 0.022122 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 24 | 6 | Accept | 0.11076 | 36.229 | 0.006415 | 0.022122 | knn | NumNeighbors: 2637 |
| 25 | 6 | Accept | 0.27884 | 66.582 | 0.006415 | 0.024532 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 287 | | | | | | | | | | MinLeafSize: 4344 | | | | | | | | | | MaxNumSplits: 48 |
| 26 | 6 | Accept | 0.58127 | 31.861 | 0.006415 | 0.024532 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.6293e-06 |
| 27 | 6 | Accept | 0.01583 | 5.7511 | 0.006415 | 0.020656 | tree | MinLeafSize: 1 |
| 28 | 6 | Accept | 0.069319 | 1.806 | 0.006415 | 0.02077 | tree | MinLeafSize: 2284 |
| 29 | 6 | Accept | 0.59166 | 352.55 | 0.006415 | 0.02077 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 790.4 | | | | | | | | | | KernelScale: 0.014348 |
| 30 | 6 | Accept | 0.043336 | 3.7865 | 0.006415 | 0.020133 | tree | MinLeafSize: 432 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 31 | 6 | Accept | 0.10555 | 34.294 | 0.006415 | 0.020133 | knn | NumNeighbors: 2430 |
| 32 | 6 | Accept | 0.021276 | 4.582 | 0.006415 | 0.018661 | tree | MinLeafSize: 17 |
| 33 | 5 | Accept | 0.030829 | 159.57 | 0.006415 | 0.018642 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 288 | | | | | | | | | | MinLeafSize: 45 | | | | | | | | | | MaxNumSplits: 23 | | 34 | 5 | Accept | 0.014307 | 49.903 | 0.006415 | 0.018642 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 234 | | | | | | | | | | MinLeafSize: 587 | | | | | | | | | | MaxNumSplits: 11 |
| 35 | 5 | Accept | 0.050212 | 1.025 | 0.006415 | 0.018642 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 36 | 6 | Accept | 0.74165 | 24.131 | 0.006415 | 0.018661 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 217 | | | | | | | | | | MinLeafSize: 8856 | | | | | | | | | | MaxNumSplits: 36 |
| 37 | 6 | Accept | 0.4226 | 558.84 | 0.006415 | 0.018661 | nb | DistributionNames: kernel | | | | | | | | | | Width: 72.906 |
| 38 | 6 | Accept | 0.57615 | 317.96 | 0.006415 | 0.018661 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.2347 | | | | | | | | | | KernelScale: 0.16176 |
| 39 | 6 | Accept | 0.59166 | 26.64 | 0.006415 | 0.018661 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.191e-07 |
| 40 | 6 | Accept | 0.087087 | 30.271 | 0.006415 | 0.018661 | knn | NumNeighbors: 1634 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 41 | 6 | Accept | 0.73985 | 551.47 | 0.006415 | 0.018661 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1055.8 |
| 42 | 4 | Accept | 0.025983 | 146.08 | 0.006415 | 0.018661 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 234 | | | | | | | | | | MinLeafSize: 73 | | | | | | | | | | MaxNumSplits: 43 | | 43 | 4 | Accept | 0.02566 | 146.73 | 0.006415 | 0.018661 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 234 | | | | | | | | | | MinLeafSize: 73 | | | | | | | | | | MaxNumSplits: 43 | | 44 | 4 | Accept | 0.024922 | 124.65 | 0.006415 | 0.018661 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 206 | | | | | | | | | | MinLeafSize: 5 | | | | | | | | | | MaxNumSplits: 39 |
| 45 | 6 | Accept | 0.025891 | 23.638 | 0.006415 | 0.018661 | knn | NumNeighbors: 6 |
| 46 | 5 | Accept | 0.025891 | 24.038 | 0.006415 | 0.018661 | knn | NumNeighbors: 6 | | 47 | 5 | Accept | 0.025891 | 23.646 | 0.006415 | 0.018661 | knn | NumNeighbors: 6 |
| 48 | 6 | Accept | 0.03009 | 188.48 | 0.006415 | 0.018661 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 299 | | | | | | | | | | MinLeafSize: 31 | | | | | | | | | | MaxNumSplits: 25 |
| 49 | 6 | Accept | 0.017214 | 6.0889 | 0.006415 | 0.017935 | tree | MinLeafSize: 4 |
| 50 | 6 | Accept | 0.01726 | 5.6027 | 0.006415 | 0.017303 | tree | MinLeafSize: 5 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 51 | 6 | Accept | 0.037244 | 158.2 | 0.006415 | 0.017303 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 5.9571 | | | | | | | | | | KernelScale: 840.87 |
| 52 | 6 | Accept | 0.046474 | 190.01 | 0.006415 | 0.017303 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 2.9119 | | | | | | | | | | KernelScale: 12.771 |
| 53 | 6 | Accept | 0.032398 | 155.47 | 0.006415 | 0.017303 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 253 | | | | | | | | | | MinLeafSize: 14 | | | | | | | | | | MaxNumSplits: 22 |
| 54 | 6 | Accept | 0.054135 | 3.0156 | 0.006415 | 0.017093 | tree | MinLeafSize: 783 |
| 55 | 6 | Accept | 0.049797 | 28.421 | 0.006415 | 0.017093 | knn | NumNeighbors: 331 |
| 56 | 6 | Accept | 0.046566 | 27.524 | 0.006415 | 0.017093 | knn | NumNeighbors: 193 |
| 57 | 6 | Accept | 0.36307 | 711.57 | 0.006415 | 0.017093 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.0011107 | | | | | | | | | | KernelScale: 0.80966 |
| 58 | 6 | Accept | 0.022706 | 23.417 | 0.006415 | 0.017093 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.3505 | | | | | | | | | | KernelScale: 127.55 |
| 59 | 6 | Accept | 0.028798 | 4.1232 | 0.006415 | 0.01733 | tree | MinLeafSize: 84 |
| 60 | 6 | Accept | 0.041351 | 28.037 | 0.006415 | 0.01733 | knn | NumNeighbors: 124 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 61 | 6 | Accept | 0.030044 | 26.265 | 0.006415 | 0.01733 | knn | NumNeighbors: 26 |
| 62 | 6 | Accept | 0.11838 | 284.42 | 0.006415 | 0.01733 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.0027874 | | | | | | | | | | KernelScale: 33.944 |
| 63 | 6 | Accept | 0.47116 | 54.038 | 0.006415 | 0.01733 | nb | DistributionNames: kernel | | | | | | | | | | Width: 4.7553e-05 |
| 64 | 6 | Accept | 0.26574 | 104.86 | 0.006415 | 0.01733 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.0011441 |
| 65 | 6 | Accept | 0.050212 | 0.72243 | 0.006415 | 0.01733 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 66 | 6 | Accept | 0.01726 | 5.6362 | 0.006415 | 0.016846 | tree | MinLeafSize: 5 |
| 67 | 6 | Accept | 0.077072 | 268.17 | 0.006415 | 0.016846 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.026902 |
| 68 | 6 | Accept | 0.031613 | 25.909 | 0.006415 | 0.016846 | knn | NumNeighbors: 33 |
| 69 | 6 | Accept | 0.02003 | 92.379 | 0.006415 | 0.016846 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 609.47 | | | | | | | | | | KernelScale: 39.88 |
| 70 | 6 | Accept | 0.1145 | 92.795 | 0.006415 | 0.016846 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 292 | | | | | | | | | | MinLeafSize: 3870 | | | | | | | | | | MaxNumSplits: 33 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 71 | 6 | Accept | 0.012968 | 53.299 | 0.006415 | 0.016846 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 208 | | | | | | | | | | MinLeafSize: 5 | | | | | | | | | | MaxNumSplits: 10 |
| 72 | 6 | Accept | 0.011999 | 53.54 | 0.006415 | 0.016846 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 215 | | | | | | | | | | MinLeafSize: 5 | | | | | | | | | | MaxNumSplits: 11 |
| 73 | 6 | Accept | 0.011999 | 50.887 | 0.006415 | 0.012076 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 205 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 11 |
| 74 | 6 | Accept | 0.039921 | 40.808 | 0.006415 | 0.012733 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 200 | | | | | | | | | | MinLeafSize: 1921 | | | | | | | | | | MaxNumSplits: 11 |
| 75 | 6 | Accept | 0.04232 | 98.684 | 0.006415 | 0.012754 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 220 | | | | | | | | | | MinLeafSize: 1495 | | | | | | | | | | MaxNumSplits: 11 |
| 76 | 6 | Accept | 0.094702 | 397.88 | 0.006415 | 0.012754 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 671.76 | | | | | | | | | | KernelScale: 4.6418 |
| 77 | 6 | Accept | 0.065165 | 44.801 | 0.006415 | 0.013179 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 233 | | | | | | | | | | MinLeafSize: 2586 | | | | | | | | | | MaxNumSplits: 35 |
| 78 | 6 | Accept | 0.22503 | 164.64 | 0.006415 | 0.013179 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 57.71 | | | | | | | | | | KernelScale: 2.0632 |
| 79 | 6 | Accept | 0.03069 | 44.42 | 0.006415 | 0.013027 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | | MinLeafSize: 1619 | | | | | | | | | | MaxNumSplits: 95 |
| 80 | 6 | Accept | 0.11459 | 71.56 | 0.006415 | 0.012064 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 212 | | | | | | | | | | MinLeafSize: 2669 | | | | | | | | | | MaxNumSplits: 31 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 81 | 6 | Accept | 0.044582 | 41.462 | 0.006415 | 0.013176 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 207 | | | | | | | | | | MinLeafSize: 2211 | | | | | | | | | | MaxNumSplits: 56 |
| 82 | 6 | Accept | 0.014722 | 223.7 | 0.006415 | 0.013176 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 832.84 | | | | | | | | | | KernelScale: 392.65 |
| 83 | 6 | Accept | 0.018322 | 23.34 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 322.92 | | | | | | | | | | KernelScale: 542.76 |
| 84 | 6 | Accept | 0.016984 | 28.833 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 727.84 | | | | | | | | | | KernelScale: 558.25 |
| 85 | 6 | Accept | 0.59166 | 2135.3 | 0.006415 | 0.013176 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.045413 | | | | | | | | | | KernelScale: 0.0034709 |
| 86 | 6 | Accept | 0.012461 | 39.005 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 806.52 | | | | | | | | | | KernelScale: 244.64 |
| 87 | 6 | Accept | 0.016753 | 22.649 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 173.38 | | | | | | | | | | KernelScale: 329.41 |
| 88 | 6 | Accept | 0.016845 | 28.292 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 242.37 | | | | | | | | | | KernelScale: 62.644 |
| 89 | 6 | Accept | 0.016799 | 20.771 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 80.674 | | | | | | | | | | KernelScale: 242.97 |
| 90 | 6 | Accept | 0.041259 | 175.27 | 0.006415 | 0.013176 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 2.0147 | | | | | | | | | | KernelScale: 549.95 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 91 | 6 | Accept | 0.017953 | 19.778 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 63.367 | | | | | | | | | | KernelScale: 263.72 |
| 92 | 6 | Accept | 0.019568 | 19.245 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 55.612 | | | | | | | | | | KernelScale: 344.11 |
| 93 | 6 | Accept | 0.016061 | 19.511 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 60.019 | | | | | | | | | | KernelScale: 185.49 |
| 94 | 6 | Accept | 0.11847 | 149.5 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 770.37 | | | | | | | | | | KernelScale: 3.1188 |
| 95 | 6 | Accept | 0.017953 | 18.428 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 43.94 | | | | | | | | | | KernelScale: 234.89 |
| 96 | 6 | Accept | 0.025106 | 25.204 | 0.006415 | 0.013176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.1294 | | | | | | | | | | KernelScale: 237.71 |
| 97 | 6 | Accept | 0.011676 | 70.358 | 0.006415 | 0.012446 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 283 | | | | | | | | | | MinLeafSize: 4 | | | | | | | | | | MaxNumSplits: 10 |
| 98 | 6 | Accept | 0.031983 | 37.179 | 0.006415 | 0.012446 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.832 | | | | | | | | | | KernelScale: 449.73 |
| 99 | 6 | Accept | 0.0097379 | 76.639 | 0.006415 | 0.011402 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 299 | | | | | | | | | | MinLeafSize: 4 | | | | | | | | | | MaxNumSplits: 12 |
| 100 | 6 | Accept | 0.22416 | 684.65 | 0.006415 | 0.011402 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 632.44 | | | | | | | | | | KernelScale: 1.8647 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 101 | 6 | Accept | 0.11478 | 87.263 | 0.006415 | 0.011913 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 272 | | | | | | | | | | MinLeafSize: 3176 | | | | | | | | | | MaxNumSplits: 54 |
| 102 | 6 | Accept | 0.0081687 | 75.177 | 0.006415 | 0.01045 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 291 | | | | | | | | | | MinLeafSize: 6 | | | | | | | | | | MaxNumSplits: 14 |
| 103 | 6 | Accept | 0.010753 | 70.01 | 0.006415 | 0.010258 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 284 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 11 |
| 104 | 6 | Accept | 0.36076 | 77 | 0.006415 | 0.010258 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0054235 | | | | | | | | | | KernelScale: 149.67 |
| 105 | 6 | Accept | 0.0084456 | 62.748 | 0.006415 | 0.0092051 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 214 | | | | | | | | | | MinLeafSize: 3 | | | | | | | | | | MaxNumSplits: 16 |
| 106 | 6 | Accept | 0.012738 | 32.34 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 529.51 | | | | | | | | | | KernelScale: 161.83 |
| 107 | 6 | Accept | 0.031521 | 41.482 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.6738 | | | | | | | | | | KernelScale: 289.2 |
| 108 | 6 | Accept | 0.021368 | 38.604 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 567.35 | | | | | | | | | | KernelScale: 36.573 |
| 109 | 6 | Accept | 0.050212 | 0.59557 | 0.006415 | 0.0092051 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 110 | 6 | Accept | 0.57883 | 127.15 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0011506 | | | | | | | | | | KernelScale: 211.98 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 111 | 6 | Accept | 0.050212 | 0.685 | 0.006415 | 0.0092051 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 112 | 6 | Accept | 0.029583 | 33.504 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.49417 | | | | | | | | | | KernelScale: 209.72 |
| 113 | 6 | Accept | 0.012138 | 41.159 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 967.85 | | | | | | | | | | KernelScale: 153.85 |
| 114 | 6 | Accept | 0.030921 | 38.482 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.57771 | | | | | | | | | | KernelScale: 262.15 |
| 115 | 6 | Accept | 0.1295 | 52.128 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.35331 | | | | | | | | | | KernelScale: 329.77 |
| 116 | 6 | Accept | 0.028475 | 33.553 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.37306 | | | | | | | | | | KernelScale: 157.01 |
| 117 | 6 | Accept | 0.050212 | 0.62316 | 0.006415 | 0.0092051 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 118 | 6 | Accept | 0.030737 | 37 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.37996 | | | | | | | | | | KernelScale: 204.24 |
| 119 | 6 | Accept | 0.021229 | 36.97 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 803.51 | | | | | | | | | | KernelScale: 38.271 |
| 120 | 6 | Accept | 0.015184 | 24.004 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 119.72 | | | | | | | | | | KernelScale: 145.99 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 121 | 6 | Accept | 0.06955 | 101.38 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 39.152 | | | | | | | | | | KernelScale: 8.5572 |
| 122 | 6 | Accept | 0.028291 | 32.637 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.32264 | | | | | | | | | | KernelScale: 134.73 |
| 123 | 6 | Accept | 0.041951 | 51.65 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.28203 | | | | | | | | | | KernelScale: 237.1 |
| 124 | 6 | Accept | 0.039921 | 216.38 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.049329 | | | | | | | | | | KernelScale: 190.24 |
| 125 | 6 | Accept | 0.048828 | 91.888 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 180.12 | | | | | | | | | | KernelScale: 12.235 |
| 126 | 6 | Accept | 0.030644 | 38.563 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.25725 | | | | | | | | | | KernelScale: 171.19 |
| 127 | 6 | Accept | 0.029537 | 37.409 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.25269 | | | | | | | | | | KernelScale: 145.85 |
| 128 | 6 | Accept | 0.02949 | 31.222 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.24034 | | | | | | | | | | KernelScale: 59.755 |
| 129 | 6 | Accept | 0.029537 | 130.19 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 76.533 | | | | | | | | | | KernelScale: 22.982 |
| 130 | 6 | Accept | 0.094702 | 174.14 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.043666 | | | | | | | | | | KernelScale: 25.411 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 131 | 6 | Accept | 0.037382 | 46.207 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.23867 | | | | | | | | | | KernelScale: 204.62 |
| 132 | 6 | Accept | 0.022152 | 40.588 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 887.62 | | | | | | | | | | KernelScale: 32.987 |
| 133 | 6 | Accept | 0.032629 | 32.651 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.22511 | | | | | | | | | | KernelScale: 52.294 |
| 134 | 6 | Accept | 0.02806 | 131.12 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.20346 | | | | | | | | | | KernelScale: 122.85 |
| 135 | 6 | Accept | 0.59166 | 3480 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 658.37 | | | | | | | | | | KernelScale: 0.0016161 |
| 136 | 6 | Accept | 0.59166 | 3481.3 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 658.37 | | | | | | | | | | KernelScale: 0.0016161 |
| 137 | 6 | Accept | 0.032121 | 44.172 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.20648 | | | | | | | | | | KernelScale: 162.33 |
| 138 | 6 | Accept | 0.30755 | 71.9 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.015886 | | | | | | | | | | KernelScale: 120.23 |
| 139 | 6 | Accept | 0.040936 | 233.89 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.046614 | | | | | | | | | | KernelScale: 213.78 |
| 140 | 5 | Accept | 0.040521 | 224.72 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.043998 | | | | | | | | | | KernelScale: 195.58 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 141 | 5 | Accept | 0.22208 | 64.659 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.049755 | | | | | | | | | | KernelScale: 151.01 |
| 142 | 6 | Accept | 0.029444 | 30.014 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.21789 | | | | | | | | | | KernelScale: 64.023 |
| 143 | 6 | Accept | 0.046013 | 49.025 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.14805 | | | | | | | | | | KernelScale: 178.98 |
| 144 | 6 | Accept | 0.013291 | 42.726 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 960.14 | | | | | | | | | | KernelScale: 334.83 |
| 145 | 6 | Accept | 0.59166 | 3493.5 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 17.982 | | | | | | | | | | KernelScale: 0.0043756 |
| 146 | 6 | Accept | 0.089533 | 189.71 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.2326 | | | | | | | | | | KernelScale: 15.411 |
| 147 | 6 | Accept | 0.03369 | 49.586 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.21482 | | | | | | | | | | KernelScale: 180.5 |
| 148 | 6 | Accept | 0.013384 | 42.876 | 0.006415 | 0.0092051 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 920.16 | | | | | | | | | | KernelScale: 334.75 |
| 149 | 6 | Accept | 0.027368 | 117.6 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.22783 | | | | | | | | | | KernelScale: 105.59 |
| 150 | 6 | Accept | 0.041674 | 234.61 | 0.006415 | 0.0092051 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.032335 | | | | | | | | | | KernelScale: 182.17 |
__________________________________________________________ Optimization completed. Total iterations: 150 Total elapsed time: 4534.2478 seconds Total time for training and validation: 24883.8563 seconds Best observed learner is an ensemble model with: Method: AdaBoostM2 NumLearningCycles: 214 MinLeafSize: 5 MaxNumSplits: 23 Observed validation loss: 0.006415 Time for training and validation: 60.9987 seconds Best estimated learner (returned model) is an ensemble model with: Method: AdaBoostM2 NumLearningCycles: 214 MinLeafSize: 3 MaxNumSplits: 16 Estimated validation loss: 0.0092051 Estimated time for training and validation: 57.8146 seconds Documentation for fitcauto display
Конечная модель, возвращенная fitcauto соответствует лучшему оцененному ученику. Перед возвращением модели функция переобучает ее, используя все обучающие данные (XTrain и YTrain), перечисленные Learner (или модель) тип и отображенные значения гиперзначений параметров.
Оценка эффективности тестового набора
Оцените окончательную производительность модели на наборе тестовых данных.
testAccuracy = 1 - loss(Mdl,XTest,YTest)
testAccuracy = 0.9917
Итоговая модель правильно классифицирует более 99% наблюдений.
Объединение выбора признаков и автоматического выбора классификаторов
Использование fitcauto чтобы автоматически выбрать классификационную модель с оптимизированными гиперпараметрами, данные предиктора и отклика, сохраненные в таблице. Перед передачей данных в fitcauto, выполните выбор признаков, чтобы удалить неважные предикторы из набора данных.
Загрузка и разбиение данных
Прочтите образец файла CreditRating_Historical.dat в таблицу. Данные предиктора состоят из финансовых коэффициентов и отраслевой информации для списка корпоративных клиентов. Переменная ответа состоит из кредитных рейтингов, присвоенных рейтинговым агентством. Предварительный просмотр первых нескольких строк набора данных.
creditrating = readtable('CreditRating_Historical.dat');
head(creditrating)ans=8×8 table
ID WC_TA RE_TA EBIT_TA MVE_BVTD S_TA Industry Rating
_____ ______ ______ _______ ________ _____ ________ _______
62394 0.013 0.104 0.036 0.447 0.142 3 {'BB' }
48608 0.232 0.335 0.062 1.969 0.281 8 {'A' }
42444 0.311 0.367 0.074 1.935 0.366 1 {'A' }
48631 0.194 0.263 0.062 1.017 0.228 4 {'BBB'}
43768 0.121 0.413 0.057 3.647 0.466 12 {'AAA'}
39255 -0.117 -0.799 0.01 0.179 0.082 4 {'CCC'}
62236 0.087 0.158 0.049 0.816 0.324 2 {'BBB'}
39354 0.005 0.181 0.034 2.597 0.388 7 {'AA' }
Потому что каждое значение в ID переменная является уникальным идентификатором клиента, то есть length(unique(creditrating.ID)) равно количеству наблюдений в creditrating, а ID переменная является плохим предиктором. Удалите ID переменная из таблицы и преобразуйте Industry переменная к categorical переменная.
creditrating = removevars(creditrating,'ID');
creditrating.Industry = categorical(creditrating.Industry);Разделите данные на обучающие и тестовые наборы. Используйте приблизительно 85% наблюдений для процесса выбора модели и настройки гиперпараметра и 15% наблюдений, чтобы проверить эффективность конечной модели, возвращенной fitcauto на новых данных. Использование cvpartition для разбиения данных на разделы.
rng('default') % For reproducibility of the partition c = cvpartition(creditrating.Rating,'Holdout',0.15); trainingIndices = training(c); % Indices for the training set testIndices = test(c); % Indices for the test set creditTrain = creditrating(trainingIndices,:); creditTest = creditrating(testIndices,:);
Выполните выбор признаков
Перед передачей обучающих данных в fitcauto, найти важные предикторы при помощи fscchi2 функция. Визуализируйте счета предиктора при помощи bar функция. Потому что некоторые счета могут быть Inf, и bar отбрасывает Inf значения, сначала постройте график конечных счетов, а затем постройте график конечного представления Inf счета другого цвета.
[idx,scores] = fscchi2(creditTrain,'Rating'); bar(scores(idx)) % Represents finite scores hold on veryImportant = isinf(scores); finiteMax = max(scores(~veryImportant)); bar(finiteMax*veryImportant(idx)) % Represents Inf scores hold off xticklabels(strrep(creditTrain.Properties.VariableNames(idx),'_','\_')) xtickangle(45) legend({'Finite Scores','Inf Scores'})
Заметьте, что Industry предиктор имеет низкий счет, соответствующий p-значению, которое больше 0,05, что указывает на то, что Industry возможно, не является важной функцией. Удалите Industry функция из наборов обучающих и тестовых данных.
creditTrain = removevars(creditTrain,'Industry'); creditTest = removevars(creditTest,'Industry');
Выполняйте fitcauto
Передайте обучающие данные в fitcauto. Функция использует байесовскую оптимизацию, чтобы выбрать модели и их значения гиперзначений параметров, и возвращает обученную модель Mdl с наилучшей ожидаемой эффективностью. Задайте, чтобы попробовать все доступные типы учащихся и запустить оптимизацию параллельно (требует Parallel Computing Toolbox™). Верните второй выход Results который содержит детали байесовской оптимизации.
Ожидайте, что этот процесс займет некоторое время. По умолчанию fitcauto предоставляет график оптимизации и итерационное отображение результатов оптимизации. Для получения дополнительной информации о том, как интерпретировать эти результаты, смотрите Подробное отображение.
options = struct('UseParallel',true); [Mdl,Results] = fitcauto(creditTrain,'Rating', ... 'Learners','all','HyperparameterOptimizationOptions',options);
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 6).
Copying objective function to workers... Done copying objective function to workers.
Learner types to explore: discr, ensemble, kernel, knn, linear, nb, svm, tree Total iterations (MaxObjectiveEvaluations): 240 Total time (MaxTime): Inf
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 1 | 6 | Best | 0.42716 | 3.0379 | 0.42716 | 0.42716 | discr | Delta: 0.00046441 | | | | | | | | | | Gamma: 0.2485 |
| 2 | 4 | Accept | 0.74185 | 4.7899 | 0.24948 | 0.29794 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.48455 | | | | | | | | | | KernelScale: 354.44 | | 3 | 4 | Best | 0.24948 | 5.0813 | 0.24948 | 0.29794 | linear | Coding: onevsone | | | | | | | | | | Lambda: 6.3551e-08 | | | | | | | | | | Learner: logistic | | 4 | 4 | Accept | 0.29794 | 3.7295 | 0.24948 | 0.29794 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 12 | | | | | | | | | | LearnRate: 0.063776 | | | | | | | | | | MinLeafSize: 277 |
| 5 | 3 | Accept | 0.25097 | 9.2655 | 0.24948 | 0.25067 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 7.8433 | | | | | | | | | | Lambda: 1.4468e-06 | | 6 | 3 | Accept | 0.25067 | 0.81139 | 0.24948 | 0.25067 | knn | NumNeighbors: 105 | | | | | | | | | | Distance: minkowski |
| 7 | 6 | Accept | 0.52917 | 2.3362 | 0.24948 | 0.25067 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.002417 | | | | | | | | | | KernelScale: 356.9 |
| 8 | 3 | Accept | 0.55818 | 0.63908 | 0.24948 | 0.25067 | discr | Delta: 0.98612 | | | | | | | | | | Gamma: 0.86519 | | 9 | 3 | Accept | 0.3781 | 1.6777 | 0.24948 | 0.25067 | linear | Coding: onevsall | | | | | | | | | | Lambda: 1.0412e-06 | | | | | | | | | | Learner: logistic | | 10 | 3 | Accept | 0.43225 | 0.80766 | 0.24948 | 0.25067 | discr | Delta: 0.00013711 | | | | | | | | | | Gamma: 0.60585 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 11 | 3 | Accept | 0.47712 | 3.3756 | 0.24948 | 0.25067 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 2.7347 | | | | | | | | | | KernelScale: 24.465 |
| 12 | 6 | Accept | 0.25695 | 2.3709 | 0.24948 | 0.25067 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.057566 |
| 13 | 4 | Accept | 0.26413 | 0.49941 | 0.24379 | 0.25067 | tree | MinLeafSize: 30 | | 14 | 4 | Accept | 0.42327 | 0.85101 | 0.24379 | 0.25067 | knn | NumNeighbors: 56 | | | | | | | | | | Distance: cosine | | 15 | 4 | Best | 0.24379 | 1.8084 | 0.24379 | 0.25067 | linear | Coding: onevsone | | | | | | | | | | Lambda: 5.9172e-05 | | | | | | | | | | Learner: svm |
| 16 | 3 | Accept | 0.81544 | 4.9586 | 0.24379 | 0.25067 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0043375 | | | | | | | | | | Lambda: 0.0023789 | | 17 | 3 | Accept | 0.45169 | 0.96723 | 0.24379 | 0.25067 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.0028505 | | | | | | | | | | Learner: svm |
| 18 | 6 | Accept | 0.33712 | 0.19972 | 0.24379 | 0.25695 | knn | NumNeighbors: 1 | | | | | | | | | | Distance: cityblock |
| 19 | 3 | Accept | 0.4834 | 0.38951 | 0.24379 | 0.25695 | knn | NumNeighbors: 72 | | | | | | | | | | Distance: correlation | | 20 | 3 | Accept | 0.46336 | 0.78881 | 0.24379 | 0.25695 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.0075732 | | | | | | | | | | Learner: svm | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 21 | 3 | Accept | 0.82082 | 2.9223 | 0.24379 | 0.25695 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0042587 | | | | | | | | | | Lambda: 0.0014754 | | 22 | 3 | Accept | 0.61292 | 1.8557 | 0.24379 | 0.25695 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 13 | | | | | | | | | | LearnRate: 0.055349 | | | | | | | | | | MinLeafSize: 910 |
| 23 | 6 | Accept | 0.43255 | 0.68689 | 0.24379 | 0.25695 | discr | Delta: 0.016844 | | | | | | | | | | Gamma: 0.64466 |
| 24 | 4 | Accept | 0.28866 | 1.9017 | 0.24379 | 0.25695 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 10 | | | | | | | | | | LearnRate: 0.11662 | | | | | | | | | | MinLeafSize: 181 | | 25 | 4 | Accept | 0.74185 | 1.3546 | 0.24379 | 0.25695 | knn | NumNeighbors: 1314 | | | | | | | | | | Distance: hamming | | 26 | 4 | Accept | 0.42746 | 0.69002 | 0.24379 | 0.25695 | discr | Delta: 2.2544e-06 | | | | | | | | | | Gamma: 0.87275 |
| 27 | 3 | Accept | 0.25606 | 12.143 | 0.24379 | 0.25498 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 132 | | | | | | | | | | LearnRate: 0.92674 | | | | | | | | | | MinLeafSize: 127 | | 28 | 3 | Accept | 0.25366 | 2.4732 | 0.24379 | 0.25498 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.10033 |
| 29 | 6 | Accept | 0.66796 | 0.22938 | 0.24379 | 0.25498 | knn | NumNeighbors: 77 | | | | | | | | | | Distance: jaccard |
| 30 | 4 | Accept | 0.69488 | 1.9352 | 0.242 | 0.25498 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 8.4886 | | | | | | | | | | KernelScale: 192.19 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 31 | 4 | Best | 0.242 | 1.9467 | 0.242 | 0.25498 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 5.425 | | | | | | | | | | KernelScale: 2.434 | | 32 | 4 | Accept | 0.32306 | 0.48104 | 0.242 | 0.25498 | tree | MinLeafSize: 2 |
| 33 | 4 | Accept | 0.43225 | 0.10463 | 0.242 | 0.25498 | discr | Delta: 0.00015292 | | | | | | | | | | Gamma: 0.51045 |
| 34 | 4 | Accept | 0.32994 | 0.2065 | 0.242 | 0.25498 | tree | MinLeafSize: 3 |
| 35 | 6 | Accept | 0.53814 | 2.8748 | 0.242 | 0.25498 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 6.8148 | | | | | | | | | | KernelScale: 382.11 |
| 36 | 4 | Accept | 0.24529 | 105.62 | 0.242 | 0.25498 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.25488 | | | | | | | | | | KernelScale: 0.0037823 | | 37 | 4 | Accept | 0.53814 | 3.3272 | 0.242 | 0.25498 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 6.8148 | | | | | | | | | | KernelScale: 382.11 | | 38 | 4 | Accept | 0.53814 | 3.8593 | 0.242 | 0.25498 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 6.8148 | | | | | | | | | | KernelScale: 382.11 |
| 39 | 4 | Accept | 0.25965 | 14.211 | 0.242 | 0.25498 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 150 | | | | | | | | | | LearnRate: 0.014842 | | | | | | | | | | MinLeafSize: 21 |
| 40 | 5 | Accept | 0.42656 | 0.16731 | 0.242 | 0.25498 | discr | Delta: 0.0020866 | | | | | | | | | | Gamma: 0.091054 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 41 | 5 | Accept | 0.42656 | 0.11476 | 0.242 | 0.25498 | discr | Delta: 0.0020866 | | | | | | | | | | Gamma: 0.091054 |
| 42 | 4 | Accept | 0.2767 | 21.389 | 0.242 | 0.25498 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 221 | | | | | | | | | | LearnRate: 0.0028588 | | | | | | | | | | MinLeafSize: 1 | | 43 | 4 | Accept | 0.29973 | 0.1848 | 0.242 | 0.25498 | tree | MinLeafSize: 7 |
| 44 | 4 | Accept | 0.25935 | 20.084 | 0.242 | 0.25498 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 304 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 100 |
| 45 | 3 | Accept | 0.24499 | 6.5071 | 0.242 | 0.26328 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.019387 | | | | | | | | | | KernelScale: 0.0047515 | | 46 | 3 | Accept | 0.28059 | 0.19378 | 0.242 | 0.26328 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 47 | 6 | Accept | 0.27281 | 0.13181 | 0.242 | 0.26328 | knn | NumNeighbors: 8 | | | | | | | | | | Distance: chebychev |
| 48 | 3 | Accept | 0.2429 | 1.5474 | 0.242 | 0.26328 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.16719 | | | | | | | | | | KernelScale: 0.11257 | | 49 | 3 | Accept | 0.2423 | 1.5219 | 0.242 | 0.26328 | linear | Coding: onevsone | | | | | | | | | | Lambda: 4.28e-07 | | | | | | | | | | Learner: svm | | 50 | 3 | Accept | 0.67125 | 0.68464 | 0.242 | 0.26328 | knn | NumNeighbors: 86 | | | | | | | | | | Distance: jaccard | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 51 | 3 | Accept | 0.46485 | 0.8417 | 0.242 | 0.26328 | tree | MinLeafSize: 645 |
| 52 | 6 | Accept | 0.63147 | 3.1926 | 0.242 | 0.26328 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 176.2 | | | | | | | | | | Lambda: 2.3903e-06 |
| 53 | 4 | Accept | 0.37571 | 0.11086 | 0.242 | 0.26706 | tree | MinLeafSize: 473 | | 54 | 4 | Accept | 0.29136 | 0.43908 | 0.242 | 0.26706 | knn | NumNeighbors: 354 | | | | | | | | | | Distance: euclidean | | 55 | 4 | Accept | 0.28059 | 0.50642 | 0.242 | 0.26706 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 56 | 3 | Accept | 0.36375 | 5.2617 | 0.242 | 0.26706 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 28.598 | | | | | | | | | | Lambda: 8.3238e-05 | | 57 | 3 | Accept | 0.27251 | 0.13425 | 0.242 | 0.26706 | tree | MinLeafSize: 20 |
| 58 | 6 | Accept | 0.43225 | 0.083255 | 0.242 | 0.26706 | discr | Delta: 0.021467 | | | | | | | | | | Gamma: 0.66016 |
| 59 | 3 | Accept | 0.28059 | 0.10921 | 0.242 | 0.26106 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 60 | 3 | Accept | 0.42537 | 0.15885 | 0.242 | 0.26106 | discr | Delta: 0.001728 | | | | | | | | | | Gamma: 0.89471 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 61 | 3 | Accept | 0.81484 | 2.9209 | 0.242 | 0.26106 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0063987 | | | | | | | | | | Lambda: 0.0075855 | | 62 | 3 | Accept | 0.24948 | 2.4493 | 0.242 | 0.26106 | linear | Coding: onevsone | | | | | | | | | | Lambda: 1.3237e-07 | | | | | | | | | | Learner: logistic |
| 63 | 6 | Accept | 0.68531 | 0.40944 | 0.242 | 0.26106 | knn | NumNeighbors: 260 | | | | | | | | | | Distance: jaccard |
| 64 | 3 | Accept | 0.32426 | 9.8071 | 0.242 | 0.26007 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 132 | | | | | | | | | | LearnRate: 0.0014516 | | | | | | | | | | MinLeafSize: 104 | | 65 | 3 | Accept | 0.55369 | 0.69919 | 0.242 | 0.26007 | knn | NumNeighbors: 615 | | | | | | | | | | Distance: correlation | | 66 | 3 | Accept | 0.24319 | 1.5742 | 0.242 | 0.26007 | linear | Coding: onevsone | | | | | | | | | | Lambda: 4.3001e-09 | | | | | | | | | | Learner: svm | | 67 | 3 | Accept | 0.70894 | 1.9822 | 0.242 | 0.26007 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.45564 | | | | | | | | | | KernelScale: 51.195 |
| 68 | 6 | Accept | 0.46485 | 0.17082 | 0.242 | 0.26007 | tree | MinLeafSize: 611 |
| 69 | 5 | Accept | 0.74185 | 2.1172 | 0.242 | 0.26007 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.94197 | | | | | | | | | | KernelScale: 524.56 | | 70 | 5 | Accept | 0.83548 | 6.3585 | 0.242 | 0.26007 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0038762 | | | | | | | | | | Lambda: 2.288e-06 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 71 | 3 | Accept | 0.25905 | 16.096 | 0.242 | 0.26007 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 160 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 2 | | 72 | 3 | Accept | 0.28448 | 13.826 | 0.242 | 0.26007 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 161 | | | | | | | | | | LearnRate: 0.7581 | | | | | | | | | | MinLeafSize: 14 | | 73 | 3 | Accept | 0.25695 | 0.24642 | 0.242 | 0.26007 | knn | NumNeighbors: 14 | | | | | | | | | | Distance: cityblock |
| 74 | 6 | Accept | 0.32396 | 1.8799 | 0.242 | 0.26007 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.86904 | | | | | | | | | | Lambda: 0.29724 |
| 75 | 4 | Accept | 0.32456 | 0.20481 | 0.242 | 0.26007 | tree | MinLeafSize: 5 | | 76 | 4 | Accept | 0.32994 | 0.35799 | 0.242 | 0.26007 | tree | MinLeafSize: 3 | | 77 | 4 | Accept | 0.26054 | 4.1734 | 0.242 | 0.26007 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 32 | | | | | | | | | | LearnRate: 0.06853 | | | | | | | | | | MinLeafSize: 19 |
| 78 | 4 | Accept | 0.43703 | 0.92964 | 0.242 | 0.24831 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.013265 | | | | | | | | | | Learner: logistic |
| 79 | 4 | Accept | 0.31588 | 17.116 | 0.242 | 0.24831 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 201 | | | | | | | | | | LearnRate: 0.0012955 | | | | | | | | | | MinLeafSize: 319 |
| 80 | 3 | Accept | 0.25277 | 7.8173 | 0.242 | 0.24831 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.78697 | | | | | | | | | | Lambda: 4.1197e-06 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 81 | 3 | Accept | 0.43015 | 0.10239 | 0.242 | 0.24831 | discr | Delta: 0.0069822 | | | | | | | | | | Gamma: 0.49526 |
| 82 | 6 | Accept | 0.42208 | 0.096953 | 0.242 | 0.24831 | discr | Delta: 0.057485 | | | | | | | | | | Gamma: 0.045714 |
| 83 | 3 | Accept | 0.52617 | 2.3915 | 0.242 | 0.24831 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.26869 | | | | | | | | | | KernelScale: 17.595 | | 84 | 3 | Accept | 0.43344 | 1.2875 | 0.242 | 0.24831 | knn | NumNeighbors: 119 | | | | | | | | | | Distance: mahalanobis | | 85 | 3 | Accept | 0.30093 | 1.3003 | 0.242 | 0.24831 | linear | Coding: onevsone | | | | | | | | | | Lambda: 0.047624 | | | | | | | | | | Learner: svm | | 86 | 3 | Accept | 0.42267 | 0.64506 | 0.242 | 0.24831 | knn | NumNeighbors: 48 | | | | | | | | | | Distance: cosine |
| 87 | 6 | Accept | 0.32905 | 0.26891 | 0.242 | 0.24831 | knn | NumNeighbors: 65 | | | | | | | | | | Distance: seuclidean |
| 88 | 4 | Accept | 0.24349 | 1.9282 | 0.242 | 0.24684 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0024196 | | | | | | | | | | KernelScale: 0.0082547 | | 89 | 4 | Accept | 0.24499 | 1.6543 | 0.242 | 0.24684 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.3697e-06 | | | | | | | | | | Learner: svm | | 90 | 4 | Accept | 0.24469 | 2.1463 | 0.242 | 0.24684 | linear | Coding: onevsone | | | | | | | | | | Lambda: 6.5777e-05 | | | | | | | | | | Learner: logistic |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 91 | 4 | Accept | 0.28059 | 0.15481 | 0.242 | 0.24684 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 92 | 5 | Accept | 0.28059 | 0.38138 | 0.242 | 0.24684 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 93 | 5 | Accept | 0.28059 | 0.15804 | 0.242 | 0.24684 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 94 | 4 | Accept | 0.29674 | 11.234 | 0.242 | 0.24684 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 147 | | | | | | | | | | LearnRate: 0.95321 | | | | | | | | | | MinLeafSize: 112 | | 95 | 4 | Accept | 0.28059 | 0.12279 | 0.242 | 0.24684 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 96 | 4 | Accept | 0.29704 | 1.1016 | 0.242 | 0.24684 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 11 | | | | | | | | | | LearnRate: 0.0072731 | | | | | | | | | | MinLeafSize: 12 |
| 97 | 4 | Accept | 0.28059 | 0.099833 | 0.242 | 0.24684 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 98 | 4 | Best | 0.2411 | 1.3634 | 0.2411 | 0.24471 | linear | Coding: onevsone | | | | | | | | | | Lambda: 0.00056045 | | | | | | | | | | Learner: svm |
| 99 | 4 | Accept | 0.74185 | 0.10441 | 0.2411 | 0.24471 | discr | Delta: 39.281 | | | | | | | | | | Gamma: 0.77032 |
| 100 | 4 | Accept | 0.30093 | 0.11043 | 0.2411 | 0.24471 | tree | MinLeafSize: 135 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 101 | 4 | Accept | 0.46844 | 0.17182 | 0.2411 | 0.24471 | knn | NumNeighbors: 2 | | | | | | | | | | Distance: cosine |
| 102 | 4 | Accept | 0.25426 | 3.1634 | 0.2411 | 0.24471 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 38 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 28 |
| 103 | 4 | Accept | 0.24469 | 1.6558 | 0.2411 | 0.24558 | linear | Coding: onevsone | | | | | | | | | | Lambda: 0.0012413 | | | | | | | | | | Learner: logistic |
| 104 | 4 | Accept | 0.43823 | 3.6572 | 0.2411 | 0.24558 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 13.293 | | | | | | | | | | Lambda: 0.0031304 |
| 105 | 4 | Accept | 0.31977 | 1.3423 | 0.2411 | 0.24556 | linear | Coding: onevsone | | | | | | | | | | Lambda: 0.040102 | | | | | | | | | | Learner: logistic |
| 106 | 6 | Accept | 0.24678 | 2.0849 | 0.2411 | 0.24529 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.9918e-05 | | | | | | | | | | Learner: logistic |
| 107 | 5 | Accept | 0.24678 | 1.9627 | 0.2411 | 0.24441 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.9918e-05 | | | | | | | | | | Learner: logistic | | 108 | 5 | Accept | 0.24678 | 2.3413 | 0.2411 | 0.24441 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.9918e-05 | | | | | | | | | | Learner: logistic |
| 109 | 4 | Accept | 0.29435 | 14.698 | 0.2411 | 0.24441 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 249 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 292 | | 110 | 4 | Accept | 0.37152 | 0.098701 | 0.2411 | 0.24441 | tree | MinLeafSize: 375 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 111 | 4 | Accept | 0.25666 | 5.7386 | 0.2411 | 0.24441 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 61 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 3 | | 112 | 4 | Accept | 0.28059 | 0.10645 | 0.2411 | 0.24441 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 113 | 4 | Accept | 0.28059 | 0.10515 | 0.2411 | 0.24441 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 114 | 6 | Accept | 0.74185 | 2.7447 | 0.2411 | 0.24441 | nb | DistributionNames: kernel | | | | | | | | | | Width: 74.975 |
| 115 | 4 | Accept | 0.78552 | 10.495 | 0.2411 | 0.24441 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.0050713 | | | | | | | | | | Lambda: 3.7406e-06 | | 116 | 4 | Accept | 0.74185 | 2.7544 | 0.2411 | 0.24441 | nb | DistributionNames: kernel | | | | | | | | | | Width: 74.975 | | 117 | 4 | Accept | 0.74185 | 2.6565 | 0.2411 | 0.24441 | nb | DistributionNames: kernel | | | | | | | | | | Width: 74.975 |
| 118 | 4 | Accept | 0.45797 | 11.614 | 0.2411 | 0.24441 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.94716 | | | | | | | | | | KernelScale: 0.072905 |
| 119 | 4 | Accept | 0.65271 | 1.6037 | 0.2411 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010356 | | | | | | | | | | KernelScale: 1.3521 |
| 120 | 4 | Accept | 0.3108 | 1.4953 | 0.2411 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.080708 | | | | | | | | | | KernelScale: 2.7439 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 121 | 4 | Accept | 0.2414 | 1.2954 | 0.2411 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0048396 | | | | | | | | | | KernelScale: 0.02413 |
| 122 | 4 | Accept | 0.5157 | 9.6657 | 0.2411 | 0.24441 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.071659 | | | | | | | | | | Lambda: 1.3447e-05 |
| 123 | 4 | Accept | 0.53186 | 390.32 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 416.46 | | | | | | | | | | KernelScale: 0.0019087 | | 124 | 4 | Best | 0.2402 | 1.4571 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.033468 | | | | | | | | | | KernelScale: 0.073489 |
| 125 | 4 | Accept | 0.2402 | 1.3713 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0040249 | | | | | | | | | | KernelScale: 0.030373 |
| 126 | 3 | Accept | 0.24529 | 39.503 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.32116 | | | | | | | | | | KernelScale: 0.0076281 | | 127 | 3 | Accept | 0.25576 | 1.4031 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.018022 | | | | | | | | | | KernelScale: 0.3703 |
| 128 | 6 | Accept | 0.2426 | 1.3555 | 0.2402 | 0.24229 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.029999 | | | | | | | | | | KernelScale: 0.046245 |
| 129 | 3 | Accept | 0.24499 | 9.5236 | 0.2402 | 0.24419 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.38346 | | | | | | | | | | KernelScale: 0.01786 | | 130 | 3 | Accept | 0.28059 | 0.12828 | 0.2402 | 0.24419 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 131 | 3 | Accept | 0.6886 | 1.4213 | 0.2402 | 0.24419 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.0763 | | | | | | | | | | Learner: logistic | | 132 | 3 | Accept | 0.4487 | 3.111 | 0.2402 | 0.24419 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 4.845 | | | | | | | | | | KernelScale: 0.74028 |
| 133 | 6 | Accept | 0.24469 | 8.3042 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.45335 | | | | | | | | | | KernelScale: 0.02034 |
| 134 | 3 | Accept | 0.26204 | 1.7109 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0011217 | | | | | | | | | | KernelScale: 0.10651 | | 135 | 3 | Accept | 0.74185 | 4.0388 | 0.2402 | 0.24441 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 10.518 | | | | | | | | | | Lambda: 0.20458 | | 136 | 3 | Accept | 0.29734 | 9.9598 | 0.2402 | 0.24441 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 107 | | | | | | | | | | LearnRate: 0.0023672 | | | | | | | | | | MinLeafSize: 197 | | 137 | 3 | Accept | 0.44391 | 2.0399 | 0.2402 | 0.24441 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.00025608 |
| 138 | 6 | Accept | 0.24559 | 3.2102 | 0.2402 | 0.24276 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.5211 | | | | | | | | | | KernelScale: 0.04163 |
| 139 | 4 | Accept | 0.25067 | 1.4995 | 0.2402 | 0.24276 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.030306 | | | | | | | | | | KernelScale: 0.37391 | | 140 | 4 | Accept | 0.24559 | 1.6227 | 0.2402 | 0.24276 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.0973e-07 | | | | | | | | | | Learner: svm | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 141 | 4 | Accept | 0.74634 | 7.0945 | 0.2402 | 0.24276 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.01094 | | | | | | | | | | Lambda: 0.0013866 |
| 142 | 4 | Accept | 0.29076 | 1.3294 | 0.2402 | 0.24334 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.006875 | | | | | | | | | | KernelScale: 0.38629 |
| 143 | 4 | Accept | 0.24379 | 1.4595 | 0.2402 | 0.24269 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010315 | | | | | | | | | | KernelScale: 0.0079737 |
| 144 | 4 | Accept | 0.24499 | 4.9134 | 0.2402 | 0.24226 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.62692 | | | | | | | | | | KernelScale: 0.033311 |
| 145 | 4 | Accept | 0.24469 | 12.208 | 0.2402 | 0.24242 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010122 | | | | | | | | | | KernelScale: 0.0010167 |
| 146 | 4 | Accept | 0.38588 | 1.5037 | 0.2402 | 0.24388 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0011127 | | | | | | | | | | KernelScale: 0.52838 |
| 147 | 4 | Accept | 0.24589 | 1.3026 | 0.2402 | 0.24228 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.1424 | | | | | | | | | | KernelScale: 0.58035 |
| 148 | 4 | Accept | 0.2408 | 1.2582 | 0.2402 | 0.24165 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010168 | | | | | | | | | | KernelScale: 0.032668 |
| 149 | 4 | Accept | 0.24469 | 1.2764 | 0.2402 | 0.24197 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.13008 | | | | | | | | | | KernelScale: 0.53455 |
| 150 | 5 | Accept | 0.24469 | 6.7111 | 0.2402 | 0.24222 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.45516 | | | | | | | | | | KernelScale: 0.024796 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 151 | 5 | Accept | 0.30422 | 1.5471 | 0.2402 | 0.24244 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.36805 | | | | | | | | | | KernelScale: 4.7124 |
| 152 | 6 | Accept | 0.24559 | 1.5701 | 0.2402 | 0.2414 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.21039 | | | | | | | | | | KernelScale: 0.63504 |
| 153 | 5 | Accept | 0.24529 | 66.311 | 0.2402 | 0.24243 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.45714 | | | | | | | | | | KernelScale: 0.0069546 | | 154 | 5 | Accept | 0.24589 | 1.604 | 0.2402 | 0.24243 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010712 | | | | | | | | | | KernelScale: 0.043866 |
| 155 | 5 | Accept | 0.29345 | 1.4823 | 0.2402 | 0.24262 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.032053 | | | | | | | | | | KernelScale: 0.88175 |
| 156 | 6 | Accept | 0.25247 | 1.4792 | 0.2402 | 0.24289 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.14103 | | | | | | | | | | KernelScale: 0.91489 |
| 157 | 6 | Accept | 0.2405 | 1.5732 | 0.2402 | 0.242 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010011 | | | | | | | | | | KernelScale: 0.012195 |
| 158 | 6 | Accept | 0.2426 | 1.5232 | 0.2402 | 0.24223 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.52301 | | | | | | | | | | KernelScale: 0.69511 |
| 159 | 6 | Accept | 0.25456 | 1.9467 | 0.2402 | 0.24253 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.63305 | | | | | | | | | | KernelScale: 2.1073 |
| 160 | 5 | Accept | 0.24559 | 133.37 | 0.2402 | 0.24253 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.73914 | | | | | | | | | | KernelScale: 0.005832 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 161 | 5 | Accept | 0.57314 | 8.1723 | 0.2402 | 0.24253 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.071267 | | | | | | | | | | Lambda: 1.4009e-06 |
| 162 | 5 | Accept | 0.24649 | 1.6878 | 0.2402 | 0.24325 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.19825 | | | | | | | | | | KernelScale: 0.74742 |
| 163 | 5 | Accept | 0.2417 | 1.5824 | 0.2402 | 0.24297 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010619 | | | | | | | | | | KernelScale: 0.01008 |
| 164 | 4 | Accept | 0.45528 | 140.46 | 0.2402 | 0.24441 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.00706 | | | | | | | | | | KernelScale: 0.030909 | | 165 | 4 | Accept | 0.31409 | 1.5573 | 0.2402 | 0.24441 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.56217 | | | | | | | | | | KernelScale: 7.5961 |
| 166 | 4 | Accept | 0.46575 | 2.2317 | 0.2402 | 0.24441 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.10607 | | | | | | | | | | KernelScale: 0.67446 |
| 167 | 4 | Accept | 0.24529 | 3.3215 | 0.2402 | 0.24334 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.4843 | | | | | | | | | | KernelScale: 0.069679 |
| 168 | 4 | Accept | 0.25396 | 1.3297 | 0.2402 | 0.24353 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.52671 | | | | | | | | | | KernelScale: 1.9394 |
| 169 | 5 | Accept | 0.2402 | 1.5257 | 0.2402 | 0.24313 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010378 | | | | | | | | | | KernelScale: 0.012749 |
| 170 | 6 | Accept | 0.3464 | 1.8583 | 0.2402 | 0.24237 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.50591 | | | | | | | | | | KernelScale: 9.2835 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 171 | 6 | Accept | 0.24499 | 12.298 | 0.2402 | 0.24203 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0018442 | | | | | | | | | | KernelScale: 0.0012506 |
| 172 | 6 | Accept | 0.30272 | 1.6031 | 0.2402 | 0.24186 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.8265 | | | | | | | | | | KernelScale: 11.897 |
| 173 | 6 | Accept | 0.24349 | 1.4247 | 0.2402 | 0.24134 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010459 | | | | | | | | | | KernelScale: 0.036404 |
| 174 | 6 | Accept | 0.29794 | 1.6502 | 0.2402 | 0.24299 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 5.4893 | | | | | | | | | | KernelScale: 13.359 |
| 175 | 5 | Accept | 0.24529 | 109.43 | 0.2402 | 0.24345 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.88969 | | | | | | | | | | KernelScale: 0.0075436 | | 176 | 5 | Accept | 0.25007 | 1.6406 | 0.2402 | 0.24345 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010212 | | | | | | | | | | KernelScale: 0.071763 |
| 177 | 5 | Accept | 0.24619 | 2.171 | 0.2402 | 0.24259 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010341 | | | | | | | | | | KernelScale: 0.0039146 |
| 178 | 5 | Accept | 0.32576 | 1.6841 | 0.2402 | 0.24185 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.6102 | | | | | | | | | | KernelScale: 14.532 |
| 179 | 5 | Accept | 0.8262 | 890.49 | 0.2402 | 0.24179 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.18649 | | | | | | | | | | KernelScale: 0.0010802 |
| 180 | 5 | Accept | 0.30212 | 1.5671 | 0.2402 | 0.24174 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 6.0833 | | | | | | | | | | KernelScale: 15.913 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 181 | 5 | Accept | 0.24499 | 14.697 | 0.2402 | 0.24192 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0078062 | | | | | | | | | | KernelScale: 0.0020664 |
| 182 | 5 | Accept | 0.24529 | 61.944 | 0.2402 | 0.2414 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.0466 | | | | | | | | | | KernelScale: 0.011265 |
| 183 | 5 | Accept | 0.24499 | 17.644 | 0.2402 | 0.24186 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0021279 | | | | | | | | | | KernelScale: 0.0010743 |
| 184 | 5 | Accept | 0.29794 | 1.6007 | 0.2402 | 0.24189 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010132 | | | | | | | | | | KernelScale: 0.17669 |
| 185 | 4 | Accept | 0.24918 | 279.94 | 0.2402 | 0.24236 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 891.39 | | | | | | | | | | KernelScale: 0.076868 | | 186 | 4 | Accept | 0.26593 | 1.4006 | 0.2402 | 0.24236 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.14381 | | | | | | | | | | KernelScale: 1.2859 |
| 187 | 4 | Accept | 0.24768 | 1.438 | 0.2402 | 0.24161 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.47763 | | | | | | | | | | KernelScale: 1.4081 |
| 188 | 4 | Accept | 0.24589 | 2.5142 | 0.2402 | 0.2416 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010362 | | | | | | | | | | KernelScale: 0.0024579 |
| 189 | 4 | Accept | 0.2405 | 1.2662 | 0.2402 | 0.24215 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 11.232 | | | | | | | | | | KernelScale: 1.4768 |
| 190 | 4 | Accept | 0.24678 | 1.2604 | 0.2402 | 0.24174 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.48152 | | | | | | | | | | KernelScale: 1.1771 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 191 | 4 | Accept | 0.25426 | 1.2846 | 0.2402 | 0.24186 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.47194 | | | | | | | | | | KernelScale: 1.7714 |
| 192 | 5 | Accept | 0.24768 | 1.3295 | 0.2402 | 0.24188 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.44397 | | | | | | | | | | KernelScale: 1.3493 |
| 193 | 6 | Accept | 0.2414 | 1.3979 | 0.2402 | 0.24154 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.8796 | | | | | | | | | | KernelScale: 1.4211 |
| 194 | 6 | Accept | 0.24499 | 18.984 | 0.2402 | 0.24225 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0018335 | | | | | | | | | | KernelScale: 0.0010104 |
| 195 | 6 | Accept | 0.25037 | 1.5112 | 0.2402 | 0.24212 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.41413 | | | | | | | | | | KernelScale: 1.4211 |
| 196 | 6 | Accept | 0.24499 | 6.007 | 0.2402 | 0.24191 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010001 | | | | | | | | | | KernelScale: 0.0013198 |
| 197 | 6 | Accept | 0.24529 | 111.16 | 0.2402 | 0.24215 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.046722 | | | | | | | | | | KernelScale: 0.0016388 |
| 198 | 5 | Accept | 0.24499 | 18.688 | 0.2402 | 0.24186 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0033281 | | | | | | | | | | KernelScale: 0.0012057 | | 199 | 5 | Accept | 0.2426 | 1.4789 | 0.2402 | 0.24186 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 3.0066 | | | | | | | | | | KernelScale: 1.3889 |
| 200 | 6 | Accept | 0.2417 | 1.499 | 0.2402 | 0.24212 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.7327 | | | | | | | | | | KernelScale: 1.3637 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 201 | 5 | Accept | 0.26473 | 277.53 | 0.2402 | 0.24176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.19781 | | | | | | | | | | KernelScale: 0.0010905 | | 202 | 5 | Accept | 0.30332 | 1.5632 | 0.2402 | 0.24176 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.075228 | | | | | | | | | | KernelScale: 1.8677 |
| 203 | 5 | Accept | 0.3108 | 1.5519 | 0.2402 | 0.24188 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.8086 | | | | | | | | | | KernelScale: 16.078 |
| 204 | 5 | Accept | 0.2414 | 1.4413 | 0.2402 | 0.24147 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 3.1547 | | | | | | | | | | KernelScale: 0.57043 |
| 205 | 5 | Accept | 0.3108 | 1.5576 | 0.2402 | 0.24195 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 3.0235 | | | | | | | | | | KernelScale: 16.787 |
| 206 | 5 | Accept | 0.2411 | 1.3939 | 0.2402 | 0.24163 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010447 | | | | | | | | | | KernelScale: 0.022855 |
| 207 | 5 | Accept | 0.29704 | 1.5568 | 0.2402 | 0.24155 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.11574 | | | | | | | | | | KernelScale: 1.8573 |
| 208 | 5 | Accept | 0.2426 | 1.4114 | 0.2402 | 0.24195 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.2749 | | | | | | | | | | KernelScale: 1.2395 |
| 209 | 5 | Accept | 0.24559 | 3.445 | 0.2402 | 0.2417 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 560.48 | | | | | | | | | | KernelScale: 1.4181 |
| 210 | 4 | Accept | 0.24678 | 266.64 | 0.2402 | 0.24136 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 11.849 | | | | | | | | | | KernelScale: 0.0095181 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 211 | 4 | Accept | 0.24589 | 2.1278 | 0.2402 | 0.24136 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 41.82 | | | | | | | | | | KernelScale: 0.71147 |
| 212 | 3 | Accept | 0.95034 | 1042.9 | 0.2402 | 0.24153 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 10.672 | | | | | | | | | | KernelScale: 0.0013453 | | 213 | 3 | Accept | 0.24649 | 2.0006 | 0.2402 | 0.24153 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 210.22 | | | | | | | | | | KernelScale: 1.4654 |
| 214 | 6 | Accept | 0.26653 | 1.2521 | 0.2402 | 0.24136 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.16417 | | | | | | | | | | KernelScale: 1.3626 |
| 215 | 4 | Accept | 0.2429 | 1.5969 | 0.2402 | 0.24136 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010295 | | | | | | | | | | KernelScale: 0.008743 | | 216 | 4 | Accept | 0.33353 | 1.2604 | 0.2402 | 0.24136 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.0017992 | | 217 | 4 | Accept | 0.28059 | 0.2661 | 0.2402 | 0.24136 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN |
| 218 | 4 | Accept | 0.24798 | 1.3024 | 0.2402 | 0.24173 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 66.132 | | | | | | | | | | KernelScale: 16.829 |
| 219 | 4 | Accept | 0.24529 | 1.255 | 0.2402 | 0.24156 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.0858 | | | | | | | | | | KernelScale: 1.3599 |
| 220 | 4 | Accept | 0.2408 | 1.2539 | 0.2402 | 0.24183 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 3.8001 | | | | | | | | | | KernelScale: 1.3322 |
|===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 221 | 4 | Accept | 0.24589 | 1.8906 | 0.2402 | 0.24185 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 399.07 | | | | | | | | | | KernelScale: 2.1791 |
| 222 | 4 | Accept | 0.2414 | 1.2821 | 0.2402 | 0.24187 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 4.8589 | | | | | | | | | | KernelScale: 1.3542 |
| 223 | 4 | Accept | 0.2423 | 1.2804 | 0.2402 | 0.24131 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.6082 | | | | | | | | | | KernelScale: 1.3565 |
| 224 | 4 | Accept | 0.24559 | 3.1158 | 0.2402 | 0.24116 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 536.1 | | | | | | | | | | KernelScale: 1.4679 |
| 225 | 4 | Accept | 0.25067 | 1.3228 | 0.2402 | 0.24178 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 74.127 | | | | | | | | | | KernelScale: 18.528 |
| 226 | 3 | Accept | 0.28358 | 328 | 0.2402 | 0.24207 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.47384 | | | | | | | | | | KernelScale: 0.001006 | | 227 | 3 | Accept | 0.24499 | 6.2167 | 0.2402 | 0.24207 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0026293 | | | | | | | | | | KernelScale: 0.0019141 |
| 228 | 6 | Accept | 0.2402 | 1.2501 | 0.2402 | 0.24173 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0010356 | | | | | | | | | | KernelScale: 0.012867 |
| 229 | 3 | Accept | 0.24589 | 1.4248 | 0.2402 | 0.24173 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 104.21 | | | | | | | | | | KernelScale: 17.45 | | 230 | 3 | Accept | 0.43015 | 0.13765 | 0.2402 | 0.24173 | discr | Delta: 0.00951 | | | | | | | | | | Gamma: 0.68613 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 231 | 3 | Accept | 0.47383 | 2.5354 | 0.2402 | 0.24173 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 1.2431 | | | | | | | | | | KernelScale: 0.76632 | | 232 | 3 | Accept | 0.67664 | 0.30377 | 0.2402 | 0.24173 | knn | NumNeighbors: 8 | | | | | | | | | | Distance: jaccard |
| 233 | 6 | Accept | 0.2414 | 1.2369 | 0.2402 | 0.24133 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 820.98 | | | | | | | | | | KernelScale: 17.331 |
| 234 | 3 | Accept | 0.27759 | 1.4452 | 0.2402 | 0.24133 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 22.198 | | | | | | | | | | KernelScale: 17.97 | | 235 | 3 | Accept | 0.28059 | 0.13623 | 0.2402 | 0.24133 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 236 | 3 | Accept | 0.52408 | 1.503 | 0.2402 | 0.24133 | knn | NumNeighbors: 351 | | | | | | | | | | Distance: mahalanobis | | 237 | 3 | Accept | 0.27789 | 5.8867 | 0.2402 | 0.24133 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 68 | | | | | | | | | | LearnRate: 0.18331 | | | | | | | | | | MinLeafSize: 3 |
| 238 | 6 | Accept | 0.24499 | 1.2819 | 0.2402 | 0.24188 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 201.86 | | | | | | | | | | KernelScale: 18.062 |
| 239 | 3 | Accept | 0.25097 | 1.4345 | 0.2402 | 0.24116 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 105.8 | | | | | | | | | | KernelScale: 23.315 | | 240 | 3 | Accept | 0.30212 | 0.12275 | 0.2402 | 0.24116 | tree | MinLeafSize: 122 | |===========================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |===========================================================================================================================================| | 241 | 3 | Accept | 0.25307 | 11.978 | 0.2402 | 0.24116 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 119 | | | | | | | | | | LearnRate: 0.60308 | | | | | | | | | | MinLeafSize: 1 | | 242 | 3 | Accept | 0.44182 | 11.435 | 0.2402 | 0.24116 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 6.2783 | | | | | | | | | | KernelScale: 0.19364 |
__________________________________________________________ Optimization completed. Total iterations: 242 Total elapsed time: 1907.7403 seconds Total time for training and validation: 4936.5339 seconds Best observed learner is a multiclass svm model with: Coding (ECOC): onevsone BoxConstraint: 0.033468 KernelScale: 0.073489 Observed validation loss: 0.2402 Time for training and validation: 1.4571 seconds Best estimated learner (returned model) is a multiclass svm model with: Coding (ECOC): onevsone BoxConstraint: 0.0010378 KernelScale: 0.012749 Estimated validation loss: 0.24116 Estimated time for training and validation: 1.5595 seconds Documentation for fitcauto display
Конечная модель, возвращенная fitcauto соответствует лучшему оцененному ученику. Перед возвращением модели функция переобучает ее, используя все обучающие данные (creditTrain), перечисленные Learner (или модель) тип и отображенные значения гиперзначений параметров.
Оценка эффективности тестового набора
Модель Mdl соответствует лучшей точке в байесовской оптимизации согласно 'min-visited-mean' критерий. Чтобы оценить, как модель будет работать с новыми данными, посмотрите на наблюдаемую точность перекрестной валидации модели (cvAccuracy) и его общую расчетную эффективность, основанную на байесовской оптимизации (estimatedAccuracy).
[x,~,iteration] = bestPoint(Results,'Criterion','min-visited-mean'); cvError = Results.ObjectiveTrace(iteration); cvAccuracy = 1 - cvError
cvAccuracy = 0.7598
estimatedError = predictObjective(Results,x); estimatedAccuracy = 1 - estimatedError
estimatedAccuracy = 0.7588
Оцените эффективность модели на тестовом наборе. Создайте матрицу неточностей из результатов и задайте порядок классов в матрице неточностей.
testAccuracy = 1 - loss(Mdl,creditTest,'Rating')testAccuracy = 0.7438
cm = confusionchart(creditTest.Rating,predict(Mdl,creditTest));
sortClasses(cm,{'AAA','AA','A','BBB','BB','B','CCC'})
Входные параметры
Выходные аргументы
Подробнее о
Совет
В зависимости от размера ваших данных и количества учащихся, которые вы задаете,
fitcautoможет занять некоторое время, чтобы запустить. Если у вас есть лицензия Parallel Computing Toolbox, можно ускорить расчеты, запустив оптимизацию параллельно. Для этого задайте'HyperparameterOptimizationOptions',struct('UseParallel',true). Можно включать другие поля в структуру для управления другими аспектами оптимизации. См.HyperparameterOptimizationOptions.
Алгоритмы
Альтернативная функциональность
Если вы не уверены, какие модели работают лучше всего для вашего набора данных, можно также использовать приложение Classification Learner. Используя приложение, можно выполнить настройку гиперпараметра для различных моделей и выбрать оптимизированную модель, которая работает лучше всего. Несмотря на то, что перед настройкой гиперпараметров модели необходимо выбрать конкретную модель, Classification Learner обеспечивает большую гибкость для выбора оптимизируемых гиперпараметров и настройки значений гиперзначений параметров. Однако вы не можете оптимизировать параллельно, выберите
'linear'или'kernel'учащиеся, задайте веса наблюдений или укажите предыдущие вероятности в приложении. Для получения дополнительной информации смотрите Оптимизацию Гипероптимизации параметров управления в Приложении Classification Learner.Если вы знаете, какие модели могут удовлетворить вашим данным, можно также использовать соответствующие функции подгонки модели и задать
'OptimizeHyperparameters'аргумент пары "имя-значение" для настройки гиперпараметров. Можно сравнить результаты по моделям, чтобы выбрать лучший классификатор. Для примера этого процесса см. «Движение к автоматизации выбора модели с помощью байесовской оптимизации».
Расширенные возможности
См. также
fitcdiscr | fitcecoc | fitcensemble | fitckernel | fitcknn | fitclinear | fitcnb | fitcsvm | fitctree
Введенный в R2020a