Skip to content

Commit 198d9de

Browse files
author
Github Actions
committed
Ravin Kohli: robustness in get_next_num_run (#215)
1 parent d1cbdf1 commit 198d9de

19 files changed

+223
-223
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt‎

Lines changed: 56 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7fab88397f40> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7feb18f50fa0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -87,7 +87,7 @@ the search. Currently, there are two changes that can be made to the space:-
8787
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
8888
trainer:StandardTrainer:weighted_loss, Value: True
8989
trainer:__choice__, Value: 'StandardTrainer'
90-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0017666816711425781, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
90+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002032756805419922, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
9191
data_loader:batch_size, Value: 32
9292
encoder:__choice__, Value: 'OneHotEncoder'
9393
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -128,7 +128,7 @@ the search. Currently, there are two changes that can be made to the space:-
128128
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
129129
trainer:StandardTrainer:weighted_loss, Value: True
130130
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=1, ta_time_used=14.215990781784058, wallclock_time=15.694700717926025, budget=5.555555555555555), TrajEntry(train_perf=0.17543859649122806, incumbent_id=2, incumbent=Configuration:
131+
, ta_runs=1, ta_time_used=12.502921104431152, wallclock_time=14.065864086151123, budget=5.555555555555555), TrajEntry(train_perf=0.17543859649122806, incumbent_id=2, incumbent=Configuration:
132132
data_loader:batch_size, Value: 111
133133
encoder:__choice__, Value: 'OneHotEncoder'
134134
feature_preprocessor:TruncatedSVD:target_dim, Value: 5
@@ -175,21 +175,60 @@ the search. Currently, there are two changes that can be made to the space:-
175175
trainer:RowCutOutTrainer:use_stochastic_weight_averaging, Value: True
176176
trainer:RowCutOutTrainer:weighted_loss, Value: False
177177
trainer:__choice__, Value: 'RowCutOutTrainer'
178-
, ta_runs=5, ta_time_used=78.13991332054138, wallclock_time=89.18238615989685, budget=5.555555555555555)]
178+
, ta_runs=5, ta_time_used=70.34997701644897, wallclock_time=81.9458417892456, budget=5.555555555555555), TrajEntry(train_perf=0.16374269005847952, incumbent_id=3, incumbent=Configuration:
179+
data_loader:batch_size, Value: 32
180+
encoder:__choice__, Value: 'OneHotEncoder'
181+
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
182+
imputer:categorical_strategy, Value: 'most_frequent'
183+
imputer:numerical_strategy, Value: 'mean'
184+
lr_scheduler:ReduceLROnPlateau:factor, Value: 0.1
185+
lr_scheduler:ReduceLROnPlateau:mode, Value: 'min'
186+
lr_scheduler:ReduceLROnPlateau:patience, Value: 10
187+
lr_scheduler:__choice__, Value: 'ReduceLROnPlateau'
188+
network_backbone:MLPBackbone:activation, Value: 'relu'
189+
network_backbone:MLPBackbone:num_groups, Value: 5
190+
network_backbone:MLPBackbone:num_units_1, Value: 200
191+
network_backbone:MLPBackbone:num_units_2, Value: 200
192+
network_backbone:MLPBackbone:num_units_3, Value: 200
193+
network_backbone:MLPBackbone:num_units_4, Value: 200
194+
network_backbone:MLPBackbone:num_units_5, Value: 200
195+
network_backbone:MLPBackbone:use_dropout, Value: False
196+
network_backbone:__choice__, Value: 'MLPBackbone'
197+
network_embedding:__choice__, Value: 'NoEmbedding'
198+
network_head:__choice__, Value: 'fully_connected'
199+
network_head:fully_connected:activation, Value: 'relu'
200+
network_head:fully_connected:num_layers, Value: 2
201+
network_head:fully_connected:units_layer_1, Value: 128
202+
network_init:XavierInit:bias_strategy, Value: 'Normal'
203+
network_init:__choice__, Value: 'XavierInit'
204+
optimizer:AdamOptimizer:beta1, Value: 0.9
205+
optimizer:AdamOptimizer:beta2, Value: 0.9
206+
optimizer:AdamOptimizer:lr, Value: 0.01
207+
optimizer:AdamOptimizer:use_weight_decay, Value: True
208+
optimizer:AdamOptimizer:weight_decay, Value: 0.0001
209+
optimizer:__choice__, Value: 'AdamOptimizer'
210+
scaler:__choice__, Value: 'StandardScaler'
211+
trainer:StandardTrainer:Lookahead:la_alpha, Value: 0.6
212+
trainer:StandardTrainer:Lookahead:la_steps, Value: 6
213+
trainer:StandardTrainer:se_lastk, Constant: 3
214+
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
215+
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
216+
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
217+
trainer:StandardTrainer:weighted_loss, Value: True
218+
trainer:__choice__, Value: 'StandardTrainer'
219+
, ta_runs=11, ta_time_used=195.89716625213623, wallclock_time=221.9700572490692, budget=16.666666666666664)]
179220
{'accuracy': 0.8728323699421965}
180221
| | Preprocessing | Estimator | Weight |
181222
|---:|:------------------------------------------------------------------|:----------------------------------------------------------|---------:|
182-
| 0 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.22 |
183-
| 1 | None | RFClassifier | 0.14 |
184-
| 2 | None | CatBoostClassifier | 0.12 |
185-
| 3 | None | KNNClassifier | 0.12 |
186-
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.12 |
187-
| 5 | None | LGBMClassifier | 0.1 |
188-
| 6 | None | ExtraTreesClassifier | 0.08 |
223+
| 0 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
224+
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
225+
| 2 | None | CatBoostClassifier | 0.16 |
226+
| 3 | None | LGBMClassifier | 0.12 |
227+
| 4 | None | KNNClassifier | 0.12 |
228+
| 5 | None | ExtraTreesClassifier | 0.1 |
229+
| 6 | None | RFClassifier | 0.08 |
189230
| 7 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
190-
| 8 | None | SVC | 0.04 |
191-
| 9 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
192-
<smac.runhistory.runhistory.RunHistory object at 0x7fab883a9700> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
231+
<smac.runhistory.runhistory.RunHistory object at 0x7feb18e281c0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
193232
data_loader:batch_size, Value: 32
194233
encoder:__choice__, Value: 'NoEncoder'
195234
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -228,7 +267,7 @@ the search. Currently, there are two changes that can be made to the space:-
228267
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
229268
trainer:StandardTrainer:weighted_loss, Value: True
230269
trainer:__choice__, Value: 'StandardTrainer'
231-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.001605987548828125, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
270+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0018703937530517578, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
232271
data_loader:batch_size, Value: 32
233272
encoder:__choice__, Value: 'NoEncoder'
234273
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -267,7 +306,7 @@ the search. Currently, there are two changes that can be made to the space:-
267306
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
268307
trainer:StandardTrainer:weighted_loss, Value: True
269308
trainer:__choice__, Value: 'StandardTrainer'
270-
, ta_runs=1, ta_time_used=8.764430522918701, wallclock_time=10.22434663772583, budget=5.555555555555555)]
309+
, ta_runs=1, ta_time_used=8.10800313949585, wallclock_time=9.621322870254517, budget=5.555555555555555)]
271310
{'accuracy': 0.8728323699421965}
272311
| | Preprocessing | Estimator | Weight |
273312
|---:|:----------------|:---------------------|---------:|
@@ -404,7 +443,7 @@ the search. Currently, there are two changes that can be made to the space:-
404443
405444
.. rst-class:: sphx-glr-timing
406445

407-
**Total running time of the script:** ( 11 minutes 43.956 seconds)
446+
**Total running time of the script:** ( 11 minutes 40.716 seconds)
408447

409448

410449
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

0 commit comments

Comments
 (0)