Meta-interface to wrap optionally executing interface

[effigies]

Any interest in adding a meta-interface like the following?

class OptionalInterface(SimpleInterface):
    def __init__(self, real_interface):
        class input_spec(real_interface.__class__.input_spec):
            pass
        class output_spec(real_interface.__class__.output_spec):
            pass

        self._interface = real_interface
        self.input_spec = input_spec
        self.output_spec = output_spec
        self._mandatory = [name for name, trait in input_spec.__class_traits__.items()
                           if trait.mandatory]

        for name in self._mandatory:
            trait = deepcopy(input_spec.__class_traits__[name])
            trait.mandatory = False
            self.input_spec.__base_traits__[name] = trait
            self.input_spec.__class_traits__[name] = trait

        super(OptionalInterface, self).__init__()
        self.inputs.trait_set(**self._interface.inputs.get_traitsfree())

    def _run_interface(self, runtime):
        do_run = all(isdefined(getattr(self.inputs, trait)) for trait in self._mandatory)

        if do_run:
            self._interface.inputs.set(**self.inputs.get_traitsfree())
            runtime = self._interface._run_interface(runtime)
            self._results.update(self._interface.aggregate_outputs(runtime).get_traitsfree())

        return runtime

This wraps an interface such that, if the mandatory inputs are not set, the outputs are left undefined.

I'm currently thinking of it for the purposes of inserting a validation step for an optional input to another node that always run, but it could also be chained to set up a part of a pipeline that will only execute if an input is present, but will silently pass Undefineds otherwise.

This is an initial hack, and some of the above might not be necessary. Currently testing with the following code:

class A(SimpleInterface):                              
     class input_spec(BaseInterfaceInputSpec):
         a = traits.Int(mandatory=True)
         b = traits.Int()
     class output_spec(TraitedSpec):
         c = traits.Int()
     def _run_interface(self, runtime):
         self._results['c'] = self.inputs.a + (self.inputs.b if isdefined(self.inputs.b) else 0)
         return runtime

In [1]: a = A(b=2)

In [2]: a.run().outputs
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
...

In [3]: b = OptionalInterface(A(b=2))

In [4]: b.run().outputs
Out[4]: 

c = <undefined>

In [5]: b.inputs.a = 5

In [6]: b.run().outputs
Out[6]: 

c = 7

[satra]

@effigies - seems like this would be a better addition at the node/workflow level rather than a separate interface.

and would this be any different from setting ignore_exception and extending ignore_exception to support input exceptions?

also, seems like you are thinking of this to implement some piece of branching logic, would it be useful to create a more general SwitchNode which embeds a set of conditionals to interfaces, and we can make the conditional have an option Exception. this would support if-else as well. and perhaps we can also add a WhileNode while we are at it! i think this kind of logic is better suited at the workflow level than interface. and people using interfaces directly can really on Python semantics.

mostly thinking out aloud over here.

[effigies]

Does ignore_exception allow downstream nodes to continue processing? I haven't really looked into its semantics, but that's the thrust of this idea, so maybe I'm reinventing the wheel.

And this is very limited branching logic, where the pathway is fully known and may just be skipped.

The closest equivalent I can think of is Maybe semantics in Haskell, which basically allow you to chain a failure through a series of steps until you reach a function that actually handles the potentially missing value without failing, itself. Also to think aloud, and follow on from the Haskell type system, there are two natural extensions of Maybe a (where a is a type), which are Either a b (sounds like your SwitchNode, if I'm reading you right), where Maybe a can be achieved with Either a None, or List a (like a MapNode where the number of outputs doesn't have to match number of inputs), where Maybe a is equivalent to only having 0 or 1 element.

But none of these is a true if-else situation where you activate one sub-workflow or another, except in the case where you can choose a Maybe cascade to work your way through.

branch = EitherNode(left=InterfaceA(), right=InterfaceB(), criterion=decision_func, ...)
postproc_a = MaybeNode(InterfaceC())
postproc_b = MaybeNode(InterfaceD())

wf.connect([
    (branch, postproc_a, [('left.out_file', 'in_file')]),
    (branch, postproc_b, [('right.out_file', 'in_file')]),
    ])

A WhileNode would be quite a different thing, by my way of thinking. That would effectively need to be a MapNode that creates nodes that are dependent on one another, and thus returns an generator of nodes, not a list of nodes, when expanded. This lends itself to a recursive idiom, and the EitherNode outlined above may be most of what would be needed to distinguish between a base case and an inductive case. And it might also conceptually dovetail nicely with a ListNode that can fan out or in, decoupling input length from output.

I'll give this some thought, and maybe find another insomniatic night to hack up a prototype of an EitherNode.

Happy to hear any thoughts on this.

[oesteban]

Worth looking at #1299, maybe you can recover some ideas from there.

[satra]

@TheChymera - perhaps @effigies suggestion here relates to your #2492 - more generally it would be good to determine the list of such nodes and their functionalities.