Skip to content

Commit 2cd9448

Browse files
author
Github Actions
committed
Ravin Kohli: added no head (#218)
1 parent 198d9de commit 2cd9448

19 files changed

+171
-347
lines changed

‎refactor_development_regularization_cocktails/_downloads/2f0f54a9831653ac5c12ee0e06324a62/basics_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/8ef6602cf8ed40221edd89244fb031af/basics_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/b38295cf56f02b1c4547385a8f389f90/advanced_tabular_python.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_downloads/df038041811db9bd567de83692a8f994/advanced_tabular_jupyter.zip‎

0 Bytes
Binary file not shown.

‎refactor_development_regularization_cocktails/_sources/advanced_tabular/example_custom_configuration_space.rst.txt‎

Lines changed: 13 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ the search. Currently, there are two changes that can be made to the space:-
4646

4747
.. code-block:: none
4848
49-
<smac.runhistory.runhistory.RunHistory object at 0x7feb18f50fa0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
49+
<smac.runhistory.runhistory.RunHistory object at 0x7fb41833b2e0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
5050
data_loader:batch_size, Value: 32
5151
encoder:__choice__, Value: 'OneHotEncoder'
5252
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -87,7 +87,7 @@ the search. Currently, there are two changes that can be made to the space:-
8787
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
8888
trainer:StandardTrainer:weighted_loss, Value: True
8989
trainer:__choice__, Value: 'StandardTrainer'
90-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002032756805419922, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
90+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0021953582763671875, budget=0), TrajEntry(train_perf=0.21052631578947367, incumbent_id=1, incumbent=Configuration:
9191
data_loader:batch_size, Value: 32
9292
encoder:__choice__, Value: 'OneHotEncoder'
9393
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -128,107 +128,14 @@ the search. Currently, there are two changes that can be made to the space:-
128128
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
129129
trainer:StandardTrainer:weighted_loss, Value: True
130130
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=1, ta_time_used=12.502921104431152, wallclock_time=14.065864086151123, budget=5.555555555555555), TrajEntry(train_perf=0.17543859649122806, incumbent_id=2, incumbent=Configuration:
132-
data_loader:batch_size, Value: 111
133-
encoder:__choice__, Value: 'OneHotEncoder'
134-
feature_preprocessor:TruncatedSVD:target_dim, Value: 5
135-
feature_preprocessor:__choice__, Value: 'TruncatedSVD'
136-
imputer:categorical_strategy, Value: 'constant_!missing!'
137-
imputer:numerical_strategy, Value: 'mean'
138-
lr_scheduler:StepLR:gamma, Value: 0.3916380858102349
139-
lr_scheduler:StepLR:step_size, Value: 5
140-
lr_scheduler:__choice__, Value: 'StepLR'
141-
network_backbone:MLPBackbone:activation, Value: 'relu'
142-
network_backbone:MLPBackbone:dropout_1, Value: 0.1523423145853786
143-
network_backbone:MLPBackbone:num_groups, Value: 1
144-
network_backbone:MLPBackbone:num_units_1, Value: 339
145-
network_backbone:MLPBackbone:use_dropout, Value: True
146-
network_backbone:__choice__, Value: 'MLPBackbone'
147-
network_embedding:LearnedEntityEmbedding:dimension_reduction_0, Value: 0.40351439845395365
148-
network_embedding:LearnedEntityEmbedding:dimension_reduction_1, Value: 0.2782048949477085
149-
network_embedding:LearnedEntityEmbedding:dimension_reduction_2, Value: 0.3389277893759589
150-
network_embedding:LearnedEntityEmbedding:dimension_reduction_3, Value: 0.27434990235055623
151-
network_embedding:LearnedEntityEmbedding:dimension_reduction_4, Value: 0.8301775423300523
152-
network_embedding:LearnedEntityEmbedding:dimension_reduction_5, Value: 0.013609031443506336
153-
network_embedding:LearnedEntityEmbedding:dimension_reduction_6, Value: 0.3503750454780089
154-
network_embedding:LearnedEntityEmbedding:dimension_reduction_7, Value: 0.8336578175236892
155-
network_embedding:LearnedEntityEmbedding:min_unique_values_for_embedding, Value: 6
156-
network_embedding:__choice__, Value: 'LearnedEntityEmbedding'
157-
network_head:__choice__, Value: 'fully_connected'
158-
network_head:fully_connected:activation, Value: 'tanh'
159-
network_head:fully_connected:num_layers, Value: 3
160-
network_head:fully_connected:units_layer_1, Value: 267
161-
network_head:fully_connected:units_layer_2, Value: 100
162-
network_init:XavierInit:bias_strategy, Value: 'Zero'
163-
network_init:__choice__, Value: 'XavierInit'
164-
optimizer:RMSpropOptimizer:alpha, Value: 0.902456140339138
165-
optimizer:RMSpropOptimizer:lr, Value: 0.0010342617079041403
166-
optimizer:RMSpropOptimizer:momentum, Value: 0.3964739559326338
167-
optimizer:RMSpropOptimizer:use_weight_decay, Value: True
168-
optimizer:RMSpropOptimizer:weight_decay, Value: 0.03884467847997589
169-
optimizer:__choice__, Value: 'RMSpropOptimizer'
170-
scaler:__choice__, Value: 'MinMaxScaler'
171-
trainer:RowCutOutTrainer:cutout_prob, Value: 0.2919538652350733
172-
trainer:RowCutOutTrainer:patch_ratio, Value: 0.07508439713858062
173-
trainer:RowCutOutTrainer:use_lookahead_optimizer, Value: False
174-
trainer:RowCutOutTrainer:use_snapshot_ensemble, Value: False
175-
trainer:RowCutOutTrainer:use_stochastic_weight_averaging, Value: True
176-
trainer:RowCutOutTrainer:weighted_loss, Value: False
177-
trainer:__choice__, Value: 'RowCutOutTrainer'
178-
, ta_runs=5, ta_time_used=70.34997701644897, wallclock_time=81.9458417892456, budget=5.555555555555555), TrajEntry(train_perf=0.16374269005847952, incumbent_id=3, incumbent=Configuration:
179-
data_loader:batch_size, Value: 32
180-
encoder:__choice__, Value: 'OneHotEncoder'
181-
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
182-
imputer:categorical_strategy, Value: 'most_frequent'
183-
imputer:numerical_strategy, Value: 'mean'
184-
lr_scheduler:ReduceLROnPlateau:factor, Value: 0.1
185-
lr_scheduler:ReduceLROnPlateau:mode, Value: 'min'
186-
lr_scheduler:ReduceLROnPlateau:patience, Value: 10
187-
lr_scheduler:__choice__, Value: 'ReduceLROnPlateau'
188-
network_backbone:MLPBackbone:activation, Value: 'relu'
189-
network_backbone:MLPBackbone:num_groups, Value: 5
190-
network_backbone:MLPBackbone:num_units_1, Value: 200
191-
network_backbone:MLPBackbone:num_units_2, Value: 200
192-
network_backbone:MLPBackbone:num_units_3, Value: 200
193-
network_backbone:MLPBackbone:num_units_4, Value: 200
194-
network_backbone:MLPBackbone:num_units_5, Value: 200
195-
network_backbone:MLPBackbone:use_dropout, Value: False
196-
network_backbone:__choice__, Value: 'MLPBackbone'
197-
network_embedding:__choice__, Value: 'NoEmbedding'
198-
network_head:__choice__, Value: 'fully_connected'
199-
network_head:fully_connected:activation, Value: 'relu'
200-
network_head:fully_connected:num_layers, Value: 2
201-
network_head:fully_connected:units_layer_1, Value: 128
202-
network_init:XavierInit:bias_strategy, Value: 'Normal'
203-
network_init:__choice__, Value: 'XavierInit'
204-
optimizer:AdamOptimizer:beta1, Value: 0.9
205-
optimizer:AdamOptimizer:beta2, Value: 0.9
206-
optimizer:AdamOptimizer:lr, Value: 0.01
207-
optimizer:AdamOptimizer:use_weight_decay, Value: True
208-
optimizer:AdamOptimizer:weight_decay, Value: 0.0001
209-
optimizer:__choice__, Value: 'AdamOptimizer'
210-
scaler:__choice__, Value: 'StandardScaler'
211-
trainer:StandardTrainer:Lookahead:la_alpha, Value: 0.6
212-
trainer:StandardTrainer:Lookahead:la_steps, Value: 6
213-
trainer:StandardTrainer:se_lastk, Constant: 3
214-
trainer:StandardTrainer:use_lookahead_optimizer, Value: True
215-
trainer:StandardTrainer:use_snapshot_ensemble, Value: True
216-
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
217-
trainer:StandardTrainer:weighted_loss, Value: True
218-
trainer:__choice__, Value: 'StandardTrainer'
219-
, ta_runs=11, ta_time_used=195.89716625213623, wallclock_time=221.9700572490692, budget=16.666666666666664)]
220-
{'accuracy': 0.8728323699421965}
221-
| | Preprocessing | Estimator | Weight |
222-
|---:|:------------------------------------------------------------------|:----------------------------------------------------------|---------:|
223-
| 0 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
224-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
225-
| 2 | None | CatBoostClassifier | 0.16 |
226-
| 3 | None | LGBMClassifier | 0.12 |
227-
| 4 | None | KNNClassifier | 0.12 |
228-
| 5 | None | ExtraTreesClassifier | 0.1 |
229-
| 6 | None | RFClassifier | 0.08 |
230-
| 7 | SimpleImputer,OneHotEncoder,MinMaxScaler,TruncSVD | embedding,MLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
231-
<smac.runhistory.runhistory.RunHistory object at 0x7feb18e281c0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
131+
, ta_runs=1, ta_time_used=14.904691934585571, wallclock_time=16.4477219581604, budget=5.555555555555555)]
132+
{'accuracy': 0.8670520231213873}
133+
| | Preprocessing | Estimator | Weight |
134+
|---:|:----------------------------------------------|:-------------------------------------------|---------:|
135+
| 0 | SimpleImputer,OneHotEncoder,NoScaler,Nystroem | embedding,MLPBackbone,NoHead,nn.Sequential | 0.94 |
136+
| 1 | None | ExtraTreesClassifier | 0.04 |
137+
| 2 | None | LGBMClassifier | 0.02 |
138+
<smac.runhistory.runhistory.RunHistory object at 0x7fb418322370> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
232139
data_loader:batch_size, Value: 32
233140
encoder:__choice__, Value: 'NoEncoder'
234141
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -267,7 +174,7 @@ the search. Currently, there are two changes that can be made to the space:-
267174
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
268175
trainer:StandardTrainer:weighted_loss, Value: True
269176
trainer:__choice__, Value: 'StandardTrainer'
270-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0018703937530517578, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
177+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0016777515411376953, budget=0), TrajEntry(train_perf=0.1871345029239766, incumbent_id=1, incumbent=Configuration:
271178
data_loader:batch_size, Value: 32
272179
encoder:__choice__, Value: 'NoEncoder'
273180
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -306,7 +213,7 @@ the search. Currently, there are two changes that can be made to the space:-
306213
trainer:StandardTrainer:use_stochastic_weight_averaging, Value: True
307214
trainer:StandardTrainer:weighted_loss, Value: True
308215
trainer:__choice__, Value: 'StandardTrainer'
309-
, ta_runs=1, ta_time_used=8.10800313949585, wallclock_time=9.621322870254517, budget=5.555555555555555)]
216+
, ta_runs=1, ta_time_used=9.924669981002808, wallclock_time=11.428627252578735, budget=5.555555555555555)]
310217
{'accuracy': 0.8728323699421965}
311218
| | Preprocessing | Estimator | Weight |
312219
|---:|:----------------|:---------------------|---------:|
@@ -443,7 +350,7 @@ the search. Currently, there are two changes that can be made to the space:-
443350
444351
.. rst-class:: sphx-glr-timing
445352

446-
**Total running time of the script:** ( 11 minutes 40.716 seconds)
353+
**Total running time of the script:** ( 11 minutes 42.049 seconds)
447354

448355

449356
.. _sphx_glr_download_advanced_tabular_example_custom_configuration_space.py:

0 commit comments

Comments
 (0)