Eager ConcatMap

[benjchristensen]

Is there already a name for something in the ecosystem that is like an eager concatMap, or ordered flatMap?

I often see the pattern of people wanting to eagerly kick off several Observables, but merge them together in order.

It is absolutely not a good idea for infinite lists, but similar to how concatMap and flatMap can both be dangerous if used wrong, an eagerConcatMap that eagerly subscribes and caches the inner Observables could be useful.

Of course this can be done without adding anything to RxJava, it's just tedious.

Bad idea? Does it already have a name that I'm unaware of?

[akarnokd]

I can't help you with the name and it is non-trivial to change merge() to support it (#2928): fast-path would require unbounded buffers for the later sources; the scalar queue wouldn't really work if interleaved with regular sources and potentially one would need to wrap each value into an indexed container anyway.

In theory, after getting rid of the optimizations, the plain merge's round-robin collector can be modified to stick to the first buffer until it is terminated before moving onto the next.

(In my plans for the semi-auto-parallelization, I thought about the join option to be either ordered or unordered merge.)

[benjchristensen]

non-trivial to change merge() to support it

Merge does not need to be changed. I can implement this today by composition. For example, I iterate over an array of Observables, .cache().subscribe() them, then concat those together.

[abersnaze]

One use case for a eager concat was a bug that I ran into a while ago with using concat and publish with a closure.

source.publish((pub) -> Observable.eagerConcat(pub.map(...), pub.map(...)));

[stealthcode]

This would also be useful where the onSubscribe makes an async call. We will need a way to order the responses in the order that the producer generated them from a request. This would of course require buffering.

[davidmoten]

I like the idea. Out of interest I think concat(Observable<T)) is a little bit eager internally at the moment inasmuch as it requests the next observable before the first has completed. It doesn't subscribe to it before the current observable has completed though which is more what you are referring to.

[ypresto]

I was misunderstanding concat operator as it subscribes each observables eagerly.

http://reactivex.io/documentation/operators/concat.html

In this figure concat is very, very looks eager :P

[benjchristensen]

It doesn't subscribe to it before the current observable has completed though which is more what you are referring to.

That's the point. A request does not represent subscription work. It just means downstream is capable of receiving. Let's not confuse those points. An Iterable should be used if even request can't be handled.

[benjchristensen]

In this figure concat is very, very looks eager :P

Yes, that diagram looks wrong. It shows both as subscribing immediately, which it definitely does not do.

/cc @DavidMGross

[DavidMGross]

Ah; I thought I'd fixed that. I'll go back and make sure the right diagrams
get copied over. I can't fix the rxmarbles version, though; that's out of
my bailiwick (@andrestaltz)

On Wed, Jul 1, 2015 at 9:24 AM, Ben Christensen notifications@github.com
wrote:

In this figure concat is very, very looks eager :P

Yes, that diagram looks wrong. It shows both as subscribing immediately,
which it definitely does not do.

/cc @DavidMGross https://github.com/DavidMGross

—
Reply to this email directly or view it on GitHub
#3017 (comment).

David M. Gross
PLP Consulting

[DavidMGross]

Oh; I see what happened. That particular illustration (
http://reactivex.io/documentation/operators/images/concat.hot.png) is meant
to illustrate what happens when both of the observables passed to concat
are "hot" observables. In such a case, as I understood it, an onError
notification from the second Observable may "jump the queue" and interrupt
the emissions from the resulting Observable even if the first Observable's
emissions haven't finished yet. Is that not correct?

The other illustrations on the page (e.g. under the RxJava and RxGroovy
accordion-folds) show the more ordinary behavior with cold Observables and
without onError notifications, and these show the subscription to the
second Observable being delayed until the first Observable completes.

On Wed, Jul 1, 2015 at 2:42 PM, PLP Consulting (David Gross) <
davgross@netflix.com> wrote:

Ah; I thought I'd fixed that. I'll go back and make sure the right
diagrams get copied over. I can't fix the rxmarbles version, though;
that's out of my bailiwick (@andrestaltz)

On Wed, Jul 1, 2015 at 9:24 AM, Ben Christensen notifications@github.com
wrote:

In this figure concat is very, very looks eager :P

Yes, that diagram looks wrong. It shows both as subscribing immediately,
which it definitely does not do.

/cc @DavidMGross https://github.com/DavidMGross

—
Reply to this email directly or view it on GitHub
#3017 (comment).

David M. Gross
PLP Consulting

David M. Gross
PLP Consulting

[ypresto]

It is very useful to implement observable for data-store backed value.
Implementing with concat or startWith will miss update notification from backed store before first fetch request is finished (i.e. before subscribing to second observable).

Refer: pwittchen/prefser#47

(I know BehaviorSubject is enough to implement observable for just a variable. :)