Skip to content

Commit 92eb0d3

Browse files
author
Github Actions
committed
Ravin Kohli: [refactor] Getting dataset properties from the dataset object (#164)
1 parent bad9dec commit 92eb0d3

22 files changed

+109
-245
lines changed

‎development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip‎

0 Bytes
Binary file not shown.

‎development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip‎

0 Bytes
Binary file not shown.

‎development/_images/sphx_glr_example_visualization_001.png‎

-253 Bytes
Loading

‎development/_images/sphx_glr_example_visualization_thumb.png‎

-1.19 KB
Loading

‎development/_modules/autoPyTorch/api/tabular_classification.html‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,6 @@ <h1>Source code for autoPyTorch.api.tabular_classification</h1><div class="highl
126126
<span class="n">TASK_TYPES_TO_STRING</span><span class="p">,</span>
127127
<span class="p">)</span>
128128
<span class="kn">from</span> <span class="nn">autoPyTorch.data.tabular_validator</span> <span class="kn">import</span> <span class="n">TabularInputValidator</span>
129-
<span class="kn">from</span> <span class="nn">autoPyTorch.datasets.base_dataset</span> <span class="kn">import</span> <span class="n">BaseDataset</span>
130129
<span class="kn">from</span> <span class="nn">autoPyTorch.datasets.resampling_strategy</span> <span class="kn">import</span> <span class="p">(</span>
131130
<span class="n">CrossValTypes</span><span class="p">,</span>
132131
<span class="n">HoldoutValTypes</span><span class="p">,</span>
@@ -210,17 +209,6 @@ <h1>Source code for autoPyTorch.api.tabular_classification</h1><div class="highl
210209
<span class="n">task_type</span><span class="o">=</span><span class="n">TASK_TYPES_TO_STRING</span><span class="p">[</span><span class="n">TABULAR_CLASSIFICATION</span><span class="p">],</span>
211210
<span class="p">)</span>
212211

213-
<span class="k">def</span> <span class="nf">_get_required_dataset_properties</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dataset</span><span class="p">:</span> <span class="n">BaseDataset</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">]:</span>
214-
<span class="k">if</span> <span class="ow">not</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">dataset</span><span class="p">,</span> <span class="n">TabularDataset</span><span class="p">):</span>
215-
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s2">&quot;Dataset is incompatible for the given task,: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
216-
<span class="nb">type</span><span class="p">(</span><span class="n">dataset</span><span class="p">)</span>
217-
<span class="p">))</span>
218-
<span class="k">return</span> <span class="p">{</span><span class="s1">&#39;task_type&#39;</span><span class="p">:</span> <span class="n">dataset</span><span class="o">.</span><span class="n">task_type</span><span class="p">,</span>
219-
<span class="s1">&#39;output_type&#39;</span><span class="p">:</span> <span class="n">dataset</span><span class="o">.</span><span class="n">output_type</span><span class="p">,</span>
220-
<span class="s1">&#39;issparse&#39;</span><span class="p">:</span> <span class="n">dataset</span><span class="o">.</span><span class="n">issparse</span><span class="p">,</span>
221-
<span class="s1">&#39;numerical_columns&#39;</span><span class="p">:</span> <span class="n">dataset</span><span class="o">.</span><span class="n">numerical_columns</span><span class="p">,</span>
222-
<span class="s1">&#39;categorical_columns&#39;</span><span class="p">:</span> <span class="n">dataset</span><span class="o">.</span><span class="n">categorical_columns</span><span class="p">}</span>
223-
224212
<div class="viewcode-block" id="TabularClassificationTask.build_pipeline"><a class="viewcode-back" href="../../../api.html#autoPyTorch.api.tabular_classification.TabularClassificationTask.build_pipeline">[docs]</a> <span class="k">def</span> <span class="nf">build_pipeline</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dataset_properties</span><span class="p">:</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="n">TabularClassificationPipeline</span><span class="p">:</span>
225213
<span class="k">return</span> <span class="n">TabularClassificationPipeline</span><span class="p">(</span><span class="n">dataset_properties</span><span class="o">=</span><span class="n">dataset_properties</span><span class="p">)</span></div>
226214

‎development/_sources/examples/20_basics/example_image_classification.rst.txt‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -89,22 +89,22 @@ Image Classification
8989
Pipeline Random Config:
9090
________________________________________
9191
Configuration:
92-
image_augmenter:GaussianBlur:sigma_min, Value: 0.0234546653439216
93-
image_augmenter:GaussianBlur:sigma_offset, Value: 1.9419513237876684
94-
image_augmenter:GaussianBlur:use_augmenter, Value: True
95-
image_augmenter:GaussianNoise:use_augmenter, Value: False
92+
image_augmenter:GaussianBlur:use_augmenter, Value: False
93+
image_augmenter:GaussianNoise:sigma_offset, Value: 1.6727659863298625
94+
image_augmenter:GaussianNoise:use_augmenter, Value: True
9695
image_augmenter:RandomAffine:use_augmenter, Value: False
97-
image_augmenter:RandomCutout:use_augmenter, Value: False
96+
image_augmenter:RandomCutout:p, Value: 0.5259508079807516
97+
image_augmenter:RandomCutout:use_augmenter, Value: True
9898
image_augmenter:Resize:use_augmenter, Value: True
99-
image_augmenter:ZeroPadAndCrop:percent, Value: 0.13952632480424082
100-
normalizer:__choice__, Value: 'NoNormalizer'
99+
image_augmenter:ZeroPadAndCrop:percent, Value: 0.0251962557279774
100+
normalizer:__choice__, Value: 'ImageNormalizer'
101101

102102
Fitting the pipeline...
103103
________________________________________
104104
ImageClassificationPipeline
105105
________________________________________
106106
0-) normalizer:
107-
NoNormalizer
107+
ImageNormalizer
108108

109109
1-) preprocessing:
110110
EarlyPreprocessing
@@ -176,7 +176,7 @@ Image Classification
176176
177177
.. rst-class:: sphx-glr-timing
178178

179-
**Total running time of the script:** ( 0 minutes 9.639 seconds)
179+
**Total running time of the script:** ( 0 minutes 9.969 seconds)
180180

181181

182182
.. _sphx_glr_download_examples_20_basics_example_image_classification.py:

‎development/_sources/examples/20_basics/example_tabular_classification.rst.txt‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ with AutoPyTorch
3636

3737
.. code-block:: none
3838
39-
<smac.runhistory.runhistory.RunHistory object at 0x7fdfb2391cd0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
39+
<smac.runhistory.runhistory.RunHistory object at 0x7f27db5ba0a0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
4040
data_loader:batch_size, Value: 64
4141
encoder:__choice__, Value: 'OneHotEncoder'
4242
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -68,7 +68,7 @@ with AutoPyTorch
6868
scaler:__choice__, Value: 'StandardScaler'
6969
trainer:StandardTrainer:weighted_loss, Value: True
7070
trainer:__choice__, Value: 'StandardTrainer'
71-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.003583669662475586, budget=0), TrajEntry(train_perf=0.16374269005847952, incumbent_id=1, incumbent=Configuration:
71+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0020754337310791016, budget=0), TrajEntry(train_perf=0.16374269005847952, incumbent_id=1, incumbent=Configuration:
7272
data_loader:batch_size, Value: 64
7373
encoder:__choice__, Value: 'OneHotEncoder'
7474
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -100,7 +100,7 @@ with AutoPyTorch
100100
scaler:__choice__, Value: 'StandardScaler'
101101
trainer:StandardTrainer:weighted_loss, Value: True
102102
trainer:__choice__, Value: 'StandardTrainer'
103-
, ta_runs=1, ta_time_used=5.496156692504883, wallclock_time=7.0571489334106445, budget=5.555555555555555), TrajEntry(train_perf=0.1578947368421053, incumbent_id=2, incumbent=Configuration:
103+
, ta_runs=1, ta_time_used=5.169630527496338, wallclock_time=6.611567974090576, budget=5.555555555555555), TrajEntry(train_perf=0.1578947368421053, incumbent_id=2, incumbent=Configuration:
104104
data_loader:batch_size, Value: 224
105105
encoder:__choice__, Value: 'OneHotEncoder'
106106
feature_preprocessor:Nystroem:kernel, Value: 'cosine'
@@ -158,7 +158,7 @@ with AutoPyTorch
158158
scaler:__choice__, Value: 'NoScaler'
159159
trainer:StandardTrainer:weighted_loss, Value: True
160160
trainer:__choice__, Value: 'StandardTrainer'
161-
, ta_runs=8, ta_time_used=76.34314274787903, wallclock_time=94.93465065956116, budget=5.555555555555555), TrajEntry(train_perf=0.1578947368421053, incumbent_id=3, incumbent=Configuration:
161+
, ta_runs=8, ta_time_used=77.50825238227844, wallclock_time=95.68161082267761, budget=5.555555555555555), TrajEntry(train_perf=0.1578947368421053, incumbent_id=3, incumbent=Configuration:
162162
data_loader:batch_size, Value: 64
163163
encoder:__choice__, Value: 'OneHotEncoder'
164164
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -190,7 +190,7 @@ with AutoPyTorch
190190
scaler:__choice__, Value: 'StandardScaler'
191191
trainer:StandardTrainer:weighted_loss, Value: True
192192
trainer:__choice__, Value: 'StandardTrainer'
193-
, ta_runs=11, ta_time_used=101.17933344841003, wallclock_time=128.4708001613617, budget=16.666666666666664)]
193+
, ta_runs=11, ta_time_used=100.9672498703003, wallclock_time=127.16906356811523, budget=16.666666666666664)]
194194
{'accuracy': 0.8670520231213873}
195195
| | Preprocessing | Estimator | Weight |
196196
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
@@ -283,7 +283,7 @@ with AutoPyTorch
283283
284284
.. rst-class:: sphx-glr-timing
285285

286-
**Total running time of the script:** ( 6 minutes 6.848 seconds)
286+
**Total running time of the script:** ( 6 minutes 1.437 seconds)
287287

288288

289289
.. _sphx_glr_download_examples_20_basics_example_tabular_classification.py:

‎development/_sources/examples/20_basics/example_tabular_regression.rst.txt‎

Lines changed: 6 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ with AutoPyTorch
3636

3737
.. code-block:: none
3838
39-
<smac.runhistory.runhistory.RunHistory object at 0x7fdfa8dea670> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
39+
<smac.runhistory.runhistory.RunHistory object at 0x7f27c9064400> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
4040
data_loader:batch_size, Value: 64
4141
encoder:__choice__, Value: 'OneHotEncoder'
4242
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -67,7 +67,7 @@ with AutoPyTorch
6767
optimizer:__choice__, Value: 'AdamOptimizer'
6868
scaler:__choice__, Value: 'StandardScaler'
6969
trainer:__choice__, Value: 'StandardTrainer'
70-
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0019443035125732422, budget=0), TrajEntry(train_perf=0.8271799717423096, incumbent_id=1, incumbent=Configuration:
70+
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.002293109893798828, budget=0), TrajEntry(train_perf=0.7575204056297503, incumbent_id=1, incumbent=Configuration:
7171
data_loader:batch_size, Value: 64
7272
encoder:__choice__, Value: 'OneHotEncoder'
7373
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
@@ -98,73 +98,11 @@ with AutoPyTorch
9898
optimizer:__choice__, Value: 'AdamOptimizer'
9999
scaler:__choice__, Value: 'StandardScaler'
100100
trainer:__choice__, Value: 'StandardTrainer'
101-
, ta_runs=1, ta_time_used=4.774270296096802, wallclock_time=8.881023645401001, budget=5.555555555555555), TrajEntry(train_perf=0.5705056242970682, incumbent_id=2, incumbent=Configuration:
102-
data_loader:batch_size, Value: 220
103-
encoder:__choice__, Value: 'OneHotEncoder'
104-
feature_preprocessor:PowerTransformer:standardize, Value: True
105-
feature_preprocessor:__choice__, Value: 'PowerTransformer'
106-
imputer:categorical_strategy, Value: 'constant_!missing!'
107-
imputer:numerical_strategy, Value: 'median'
108-
lr_scheduler:ExponentialLR:gamma, Value: 0.7297909296891054
109-
lr_scheduler:__choice__, Value: 'ExponentialLR'
110-
network_backbone:MLPBackbone:activation, Value: 'sigmoid'
111-
network_backbone:MLPBackbone:num_groups, Value: 1
112-
network_backbone:MLPBackbone:num_units_1, Value: 53
113-
network_backbone:MLPBackbone:use_dropout, Value: False
114-
network_backbone:__choice__, Value: 'MLPBackbone'
115-
network_embedding:LearnedEntityEmbedding:dimension_reduction_0, Value: 0.49162398882471625
116-
network_embedding:LearnedEntityEmbedding:dimension_reduction_1, Value: 0.9738223543779865
117-
network_embedding:LearnedEntityEmbedding:min_unique_values_for_embedding, Value: 3
118-
network_embedding:__choice__, Value: 'LearnedEntityEmbedding'
119-
network_head:__choice__, Value: 'fully_connected'
120-
network_head:fully_connected:activation, Value: 'tanh'
121-
network_head:fully_connected:num_layers, Value: 2
122-
network_head:fully_connected:units_layer_1, Value: 84
123-
network_init:XavierInit:bias_strategy, Value: 'Normal'
124-
network_init:__choice__, Value: 'XavierInit'
125-
optimizer:SGDOptimizer:lr, Value: 0.07384606967030707
126-
optimizer:SGDOptimizer:momentum, Value: 0.41566496944512654
127-
optimizer:SGDOptimizer:weight_decay, Value: 0.002766441795903385
128-
optimizer:__choice__, Value: 'SGDOptimizer'
129-
scaler:__choice__, Value: 'MinMaxScaler'
130-
trainer:__choice__, Value: 'StandardTrainer'
131-
, ta_runs=2, ta_time_used=9.929801225662231, wallclock_time=15.717073202133179, budget=5.555555555555555), TrajEntry(train_perf=0.21935886392409365, incumbent_id=3, incumbent=Configuration:
132-
data_loader:batch_size, Value: 64
133-
encoder:__choice__, Value: 'OneHotEncoder'
134-
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
135-
imputer:categorical_strategy, Value: 'most_frequent'
136-
imputer:numerical_strategy, Value: 'mean'
137-
lr_scheduler:ReduceLROnPlateau:factor, Value: 0.1
138-
lr_scheduler:ReduceLROnPlateau:mode, Value: 'min'
139-
lr_scheduler:ReduceLROnPlateau:patience, Value: 10
140-
lr_scheduler:__choice__, Value: 'ReduceLROnPlateau'
141-
network_backbone:ShapedMLPBackbone:activation, Value: 'relu'
142-
network_backbone:ShapedMLPBackbone:max_units, Value: 200
143-
network_backbone:ShapedMLPBackbone:mlp_shape, Value: 'funnel'
144-
network_backbone:ShapedMLPBackbone:num_groups, Value: 5
145-
network_backbone:ShapedMLPBackbone:output_dim, Value: 200
146-
network_backbone:ShapedMLPBackbone:use_dropout, Value: False
147-
network_backbone:__choice__, Value: 'ShapedMLPBackbone'
148-
network_embedding:__choice__, Value: 'NoEmbedding'
149-
network_head:__choice__, Value: 'fully_connected'
150-
network_head:fully_connected:activation, Value: 'relu'
151-
network_head:fully_connected:num_layers, Value: 2
152-
network_head:fully_connected:units_layer_1, Value: 128
153-
network_init:XavierInit:bias_strategy, Value: 'Normal'
154-
network_init:__choice__, Value: 'XavierInit'
155-
optimizer:AdamOptimizer:beta1, Value: 0.9
156-
optimizer:AdamOptimizer:beta2, Value: 0.9
157-
optimizer:AdamOptimizer:lr, Value: 0.01
158-
optimizer:AdamOptimizer:weight_decay, Value: 0.0
159-
optimizer:__choice__, Value: 'AdamOptimizer'
160-
scaler:__choice__, Value: 'StandardScaler'
161-
trainer:__choice__, Value: 'StandardTrainer'
162-
, ta_runs=11, ta_time_used=64.2962703704834, wallclock_time=93.74913358688354, budget=16.666666666666664)]
163-
{'r2': 0.9189154709278342}
101+
, ta_runs=1, ta_time_used=4.336726665496826, wallclock_time=8.26810622215271, budget=5.555555555555555)]
102+
{'r2': 0.9135738588575801}
164103
| | Preprocessing | Estimator | Weight |
165104
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
166-
| 0 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.64 |
167-
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.36 |
105+
| 0 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 1 |
168106
169107
170108
@@ -250,7 +188,7 @@ with AutoPyTorch
250188
251189
.. rst-class:: sphx-glr-timing
252190

253-
**Total running time of the script:** ( 5 minutes 14.190 seconds)
191+
**Total running time of the script:** ( 5 minutes 15.642 seconds)
254192

255193

256194
.. _sphx_glr_download_examples_20_basics_example_tabular_regression.py:

‎development/_sources/examples/20_basics/sg_execution_times.rst.txt‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@
55

66
Computation times
77
=================
8-
**11:30.678** total execution time for **examples_20_basics** files:
8+
**11:27.048** total execution time for **examples_20_basics** files:
99

1010
+--------------------------------------------------------------------------------------------------------------+-----------+--------+
11-
| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 06:06.848 | 0.0 MB |
11+
| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 06:01.437 | 0.0 MB |
1212
+--------------------------------------------------------------------------------------------------------------+-----------+--------+
13-
| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:14.190 | 0.0 MB |
13+
| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:15.642 | 0.0 MB |
1414
+--------------------------------------------------------------------------------------------------------------+-----------+--------+
15-
| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:09.639 | 0.0 MB |
15+
| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:09.969 | 0.0 MB |
1616
+--------------------------------------------------------------------------------------------------------------+-----------+--------+

0 commit comments

Comments
 (0)