Remove necessity for RecordComponent::SCALAR (#1154)

* Add helper: openPMD::auxiliary::overloaded

* Prepare Attributable for virtual inheritance

Use only zero-param constructors to avoid diamond initialization
pitfalls

* Fix default constructors/operators of Container and BaseRecordComponent

These derive virtually from Attributable, and this avoids that pitfalls
propagate to user code.

* Add m_datasetDefined

See in-code documentation.

* Prepare class structure without applying logic yet

BaseRecord is now derived by its contained RecordComponent type.
If it is scalar, the idea is that the BaseRecord itself is used as a
RecordComponent, without needing to retrieve the [SCALAR] entry.
No logic implemented yet around this, this just prepares the class
structure.
Notice that this will write some unnecessary attributes since the
RecordComponent types initialize some default attributes upon
construction.

* No longer use map entry SCALAR in application logic

Not yet supported: Backward compatibility for still allowing legacy
access to scalar entries

* Remove KEEP_SYNCHRONOUS task

No longer needed, as one object in the openPMD hierarchy is no longer
represented by possibly multiple Writable objects.

* Adapt Coretests

* No virtual methods in Container class

Either this way, or make all of them virtual

* Fully override Container methods in BaseRecord

Special care for legacy usage of SCALAR constant. Implement iteration
API such that it works for scalar components as well.

* Adapt Container API to C++17

insert() and reverse iterators

* Adapt myPath() functionality

* Factor out create_and_bind_container() function template

Will later be called by Record-type classes, too

* Factor out RecordComponent.__setitem__ and __getitem__

Similarly to the Container API, we will need to apply this to
Record-type classes.
Defining `__setitem__` and `__getitem__` for them is sufficient, as all
other members are inherited from RecordComponent.
`__setitem__` and `__getitem__` need special care, as they are inherited
from Container AND from RecordComponent, so some conflict resolution is
needed.

* Consistently use copy semantics in Python API

* Apply new class structure to Python API as well

* Adapt openpmd-pipe to new design

This somewhat demonstrates that this change is slightly API-breaking.
Since openpmd-pipe acts directly on the class structure via
`instanceof()`, fixes are necessary.

* Safeguard: No scalar and vector components side by side

"A scalar component can not be contained at the same time as one or more regular components."

* Remove [SCALAR] from all examples

* Avoid object slicing when using records as scalar components

* Documentation

* Adapt to refactored Python bindings after rebasing

Author: franzpoeschel

CMakeLists.txt

@@ -462,7 +462,6 @@ set(CORE_SOURCE
         src/auxiliary/JSON.cpp
         src/backend/Attributable.cpp
         src/backend/BaseRecordComponent.cpp
-        src/backend/Container.cpp
         src/backend/MeshRecordComponent.cpp
         src/backend/PatchRecord.cpp
         src/backend/PatchRecordComponent.cpp
@@ -605,7 +604,6 @@ if(openPMD_HAVE_PYTHON)
         src/binding/python/openPMD.cpp
         src/binding/python/Access.cpp
         src/binding/python/Attributable.cpp
-        src/binding/python/BaseRecord.cpp
         src/binding/python/BaseRecordComponent.cpp
         src/binding/python/ChunkInfo.cpp
         src/binding/python/Dataset.cpp

docs/source/usage/concepts.rst

@@ -20,6 +20,9 @@ A record is an data set with common properties, e.g. the electric field :math:`\
 A density field could be another record - which is scalar as it only has one component.
 
 In general, openPMD allows records with arbitrary number of components (tensors), as well as vector records and scalar records.
+In the case of vector records, the single components are stored as datasets within the record.
+In the case of scalar records, the record and component are equivalent.
+In the API, the record can be directly used as a component, and in the standard a scalar record is represented by the scalar dataset with attributes.
 
 Meshes and Particles
 --------------------

examples/10_streaming_write.py

@@ -69,7 +69,7 @@
         temperature.axis_labels = ["x", "y"]
         temperature.grid_spacing = [1., 1.]
         # temperature has no x,y,z components, so skip the last layer:
-        temperature_dataset = temperature[io.Mesh_Record_Component.SCALAR]
+        temperature_dataset = temperature
         # let's say we are in a 3x3 mesh
         temperature_dataset.reset_dataset(
             io.Dataset(np.dtype("double"), [3, 3]))

examples/12_span_write.cpp

@@ -90,8 +90,7 @@ void span_write(std::string const &filename)
 
         using mesh_type = position_t;
 
-        RecordComponent chargeDensity =
-            iteration.meshes["e_chargeDensity"][RecordComponent::SCALAR];
+        Mesh chargeDensity = iteration.meshes["e_chargeDensity"];
 
         /*
          * A similar memory optimization is possible by using a unique_ptr type

examples/13_write_dynamic_configuration.cpp

@@ -124,8 +124,7 @@ chunks = "auto"
         Dataset differentlyCompressedDataset{Datatype::INT, {10}};
         differentlyCompressedDataset.options = differentCompressionSettings;
 
-        auto someMesh = iteration.meshes["differentCompressionSettings"]
-                                        [RecordComponent::SCALAR];
+        auto someMesh = iteration.meshes["differentCompressionSettings"];
         someMesh.resetDataset(differentlyCompressedDataset);
         std::vector<int> dataVec(10, i);
         someMesh.storeChunk(dataVec, {0}, {10});

examples/13_write_dynamic_configuration.py

@@ -130,7 +130,7 @@ def main():
         temperature.axis_labels = ["x", "y"]
         temperature.grid_spacing = [1., 1.]
         # temperature has no x,y,z components, so skip the last layer:
-        temperature_dataset = temperature[io.Mesh_Record_Component.SCALAR]
+        temperature_dataset = temperature
         # let's say we are in a 3x3 mesh
         dataset = io.Dataset(np.dtype("double"), [3, 3])
         dataset.options = json.dumps(config)

examples/1_structure.cpp

@@ -54,10 +54,9 @@ int main()
      * differently.
      * https://github.com/openPMD/openPMD-standard/blob/latest/STANDARD.md#scalar-vector-and-tensor-records*/
     Record mass = electrons["mass"];
-    RecordComponent mass_scalar = mass[RecordComponent::SCALAR];
 
     Dataset dataset = Dataset(Datatype::DOUBLE, Extent{1});
-    mass_scalar.resetDataset(dataset);
+    mass.resetDataset(dataset);
 
     /* Required Records and RecordComponents are created automatically.
      * Initialization has to be done explicitly by the user. */

examples/2_read_serial.cpp

@@ -58,9 +58,7 @@ int main()
     }
 
     openPMD::ParticleSpecies electrons = i.particles["electrons"];
-    std::shared_ptr<double> charge =
-        electrons["charge"][openPMD::RecordComponent::SCALAR]
-            .loadChunk<double>();
+    std::shared_ptr<double> charge = electrons["charge"].loadChunk<double>();
     series.flush();
     cout << "And the first electron particle has a charge = "
          << charge.get()[0];

examples/2_read_serial.py

@@ -34,7 +34,7 @@
 
     # printing a scalar value
     electrons = i.particles["electrons"]
-    charge = electrons["charge"][io.Mesh_Record_Component.SCALAR]
+    charge = electrons["charge"]
     series.flush()
     print("And the first electron particle has a charge {}"
           .format(charge[0]))

examples/3_write_serial.cpp

@@ -49,8 +49,7 @@ int main(int argc, char *argv[])
     // in streaming setups, e.g. an iteration cannot be opened again once
     // it has been closed.
     // `Series::iterations` can be directly accessed in random-access workflows.
-    MeshRecordComponent rho =
-        series.writeIterations()[1].meshes["rho"][MeshRecordComponent::SCALAR];
+    Mesh rho = series.writeIterations()[1].meshes["rho"];
     cout << "Created a scalar mesh Record with all required openPMD "
             "attributes\n";