Ravin Kohli: [FIX] results management and visualisation with missing test data (#465)

Author: Github Actions

development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip

@@ -33,22 +33,22 @@ Image Classification
 
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz
-      0%|          | 0/26421880 [00:00<?, ?it/s]      0%|          | 65536/26421880 [00:00<01:15, 351267.31it/s]      1%|          | 163840/26421880 [00:00<00:43, 609414.50it/s]      2%|1         | 425984/26421880 [00:00<00:24, 1062279.85it/s]      5%|5         | 1343488/26421880 [00:00<00:07, 3392978.15it/s]     13%|#3        | 3473408/26421880 [00:00<00:03, 7060198.72it/s]     29%|##9       | 7733248/26421880 [00:00<00:01, 16103078.25it/s]     43%|####2     | 11304960/26421880 [00:00<00:00, 21320558.99it/s]     59%|#####9    | 15597568/26421880 [00:01<00:00, 22502874.39it/s]     75%|#######4  | 19791872/26421880 [00:01<00:00, 27264935.08it/s]     89%|########9 | 23592960/26421880 [00:01<00:00, 30051582.76it/s]    100%|##########| 26421880/26421880 [00:01<00:00, 18599895.80it/s]
+      0%|          | 0/26421880 [00:00<?, ?it/s]      0%|          | 32768/26421880 [00:00<01:37, 269661.04it/s]      0%|          | 65536/26421880 [00:00<01:38, 267501.14it/s]      0%|          | 131072/26421880 [00:00<01:07, 388969.63it/s]      1%|          | 229376/26421880 [00:00<00:47, 551190.13it/s]      2%|1         | 458752/26421880 [00:00<00:25, 1024519.07it/s]      4%|3         | 950272/26421880 [00:00<00:12, 2034589.74it/s]      7%|7         | 1867776/26421880 [00:00<00:06, 3806328.31it/s]     14%|#4        | 3768320/26421880 [00:00<00:03, 7511243.63it/s]     26%|##6       | 6881280/26421880 [00:01<00:01, 13055905.73it/s]     38%|###7      | 9994240/26421880 [00:01<00:00, 16843136.89it/s]     49%|####9     | 13008896/26421880 [00:01<00:00, 19076794.87it/s]     61%|######    | 16089088/26421880 [00:01<00:00, 20876086.01it/s]     73%|#######2  | 19169280/26421880 [00:01<00:00, 22121784.53it/s]     84%|########4 | 22249472/26421880 [00:01<00:00, 22991935.53it/s]     96%|#########5| 25329664/26421880 [00:01<00:00, 23587015.14it/s]    100%|##########| 26421880/26421880 [00:01<00:00, 14248915.59it/s]
     Extracting ../datasets/FashionMNIST/raw/train-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw
 
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw/train-labels-idx1-ubyte.gz
-      0%|          | 0/29515 [00:00<?, ?it/s]    100%|##########| 29515/29515 [00:00<00:00, 326077.20it/s]
+      0%|          | 0/29515 [00:00<?, ?it/s]    100%|##########| 29515/29515 [00:00<00:00, 246986.62it/s]    100%|##########| 29515/29515 [00:00<00:00, 246193.85it/s]
     Extracting ../datasets/FashionMNIST/raw/train-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw
 
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw/t10k-images-idx3-ubyte.gz
-      0%|          | 0/4422102 [00:00<?, ?it/s]      1%|1         | 65536/4422102 [00:00<00:12, 357624.30it/s]      4%|4         | 196608/4422102 [00:00<00:05, 764127.24it/s]     11%|#1        | 491520/4422102 [00:00<00:03, 1253424.48it/s]     35%|###4      | 1540096/4422102 [00:00<00:00, 3979059.86it/s]     87%|########6 | 3833856/4422102 [00:00<00:00, 7916719.41it/s]    100%|##########| 4422102/4422102 [00:00<00:00, 6005980.87it/s]
+      0%|          | 0/4422102 [00:00<?, ?it/s]      1%|          | 32768/4422102 [00:00<00:16, 271365.89it/s]      1%|1         | 65536/4422102 [00:00<00:16, 270266.93it/s]      3%|2         | 131072/4422102 [00:00<00:10, 392338.29it/s]      5%|5         | 229376/4422102 [00:00<00:07, 556637.94it/s]     10%|#         | 458752/4422102 [00:00<00:03, 1036164.16it/s]     21%|##        | 917504/4422102 [00:00<00:01, 1967160.72it/s]     41%|####1     | 1835008/4422102 [00:00<00:00, 3788204.48it/s]     83%|########2 | 3670016/4422102 [00:00<00:00, 7365292.41it/s]    100%|##########| 4422102/4422102 [00:00<00:00, 4526747.11it/s]
     Extracting ../datasets/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to ../datasets/FashionMNIST/raw
 
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz
     Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz
-      0%|          | 0/5148 [00:00<?, ?it/s]    100%|##########| 5148/5148 [00:00<00:00, 30072809.18it/s]
+      0%|          | 0/5148 [00:00<?, ?it/s]    100%|##########| 5148/5148 [00:00<00:00, 31894057.60it/s]
     Extracting ../datasets/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to ../datasets/FashionMNIST/raw
 
     Pipeline CS:
@@ -85,24 +85,23 @@ Image Classification
     Configuration(values={
       'image_augmenter:GaussianBlur:use_augmenter': False,
       'image_augmenter:GaussianNoise:use_augmenter': False,
-      'image_augmenter:RandomAffine:rotate': 264,
-      'image_augmenter:RandomAffine:scale_offset': 0.1997598286556691,
-      'image_augmenter:RandomAffine:shear': 38,
-      'image_augmenter:RandomAffine:translate_percent_offset': 0.21531300486072077,
+      'image_augmenter:RandomAffine:rotate': 124,
+      'image_augmenter:RandomAffine:scale_offset': 0.06255439515201121,
+      'image_augmenter:RandomAffine:shear': 31,
+      'image_augmenter:RandomAffine:translate_percent_offset': 0.17281138629598677,
       'image_augmenter:RandomAffine:use_augmenter': True,
-      'image_augmenter:RandomCutout:p': 0.24443754548888208,
-      'image_augmenter:RandomCutout:use_augmenter': True,
-      'image_augmenter:Resize:use_augmenter': True,
-      'image_augmenter:ZeroPadAndCrop:percent': 0.37202634450268485,
-      'normalizer:__choice__': 'ImageNormalizer',
+      'image_augmenter:RandomCutout:use_augmenter': False,
+      'image_augmenter:Resize:use_augmenter': False,
+      'image_augmenter:ZeroPadAndCrop:percent': 0.1857189312490744,
+      'normalizer:__choice__': 'NoNormalizer',
     })
 
     Fitting the pipeline...
     ________________________________________
             ImageClassificationPipeline
     ________________________________________
     0-) normalizer: 
-            ImageNormalizer
+            NoNormalizer
 
     1-) preprocessing: 
             EarlyPreprocessing
@@ -174,7 +173,7 @@ Image Classification
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  5.806 seconds)
+   **Total running time of the script:** ( 0 minutes  6.707 seconds)
 
 
 .. _sphx_glr_download_examples_20_basics_example_image_classification.py:

development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip

@@ -132,7 +132,7 @@ Search for an ensemble of machine learning algorithms
  .. code-block:: none
 
 
-    <autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f1b6277e6a0>
+    <autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7fad73de7790>
 
 
 
@@ -174,9 +174,9 @@ Print the final ensemble performance
             Optimisation Metric: accuracy
             Best validation score: 0.8713450292397661
             Number of target algorithm runs: 27
-            Number of successful target algorithm runs: 25
+            Number of successful target algorithm runs: 26
             Number of crashed target algorithm runs: 0
-            Number of target algorithms that exceeded the time limit: 2
+            Number of target algorithms that exceeded the time limit: 1
             Number of target algorithms that exceeded the memory limit: 0
 
 
@@ -186,7 +186,7 @@ Print the final ensemble performance
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 5 minutes  31.873 seconds)
+   **Total running time of the script:** ( 5 minutes  32.227 seconds)
 
 
 .. _sphx_glr_download_examples_20_basics_example_tabular_classification.py:

development/_images/sphx_glr_example_plot_over_time_001.png

@@ -123,7 +123,7 @@ Search for an ensemble of machine learning algorithms
  .. code-block:: none
 
 
-    <autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f1ad2685730>
+    <autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7face7e10f40>
 
 
 
@@ -163,7 +163,7 @@ Print the final ensemble performance
     |  2 | SimpleImputer,Variance Threshold,NoCoalescer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential |     0.1  |
     |  3 | None                                                                                             | LGBMLearner                                                     |     0.04 |
     autoPyTorch results:
-            Dataset name: 63413bef-1a43-11ed-8830-056390bd6e17
+            Dataset name: 70461479-1a43-11ed-883e-c7f5fe1042ee
             Optimisation Metric: r2
             Best validation score: 0.8670098636440993
             Number of target algorithm runs: 23
@@ -179,7 +179,7 @@ Print the final ensemble performance
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 5 minutes  35.870 seconds)
+   **Total running time of the script:** ( 5 minutes  37.001 seconds)
 
 
 .. _sphx_glr_download_examples_20_basics_example_tabular_regression.py:

development/_images/sphx_glr_example_plot_over_time_thumb.png

@@ -150,7 +150,7 @@ Search for an ensemble of machine learning algorithms
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  58.963 seconds)
+   **Total running time of the script:** ( 1 minutes  8.766 seconds)
 
 
 .. _sphx_glr_download_examples_20_basics_example_time_series_forecasting.py: