Subscriptions RFC: Are Subscriptions and Live Queries the same thing?

[robzhu]

Re-define Live Queries? Can Live Queries make Subscriptions unnecessary?
@smolinari @paralin @laneyk @dschafer @taion @Siyfion @jamesgorman2 @leebyron

Continuing the conversation from: #267 (comment)

[taion]

I think in many cases, developers use subscriptions to approximate live queries, but subscriptions are more powerful and easier to implement.

For example, in my case, where I have many microservices on my backend, where some nested fields go to other services, it's not really straightforward to define how live queries would work, and I've chosen explicitly to model things as event streams.

Live queries would be a nice abstraction on top, but it's only that – it's not, in the general case, a great backend building block.

[stubailo]

I don't think this discussion should be judging about whether or not live queries are better than subscriptions, just whether they are different enough that they should be considered independently.

I think "building block" is a great way to look at it though - subscriptions are a great well-specified unit of realtime data push that can be used to build a lot of other cool stuff. The fact that it's very easy to implement a spec-compliant subscription on the server side is pretty awesome, even if it's not always the thing you want as a client-side developer.

[smolinari]

I'd like to know first what people consider live queries to be. What is the definition? I ask, because I think there are different perspectives or ideas in play here, and thus the discussion can run in unnecessary tangents.

So, what is a live query? 😄

Scott

[stubailo]

Here's my impression of a live query in one sentence:

"A live query is a query where some or all of the parts can be marked as 'live', and the client expects to receive updates whenever any of those parts would have ended up with a different result if refetched again."

In short, they should be a drop-in replacement for polling.

[robzhu]

I'll quote my original definition in the RFC:

Live Queries- the client issues a standard query. Whenever the answer to the query changes, the server pushes the new data to the client. The key difference between Live Queries and Event-based Subscriptions is that Live Queries do not depend on the notion of events. The data itself is live and includes mechanisms to communicate changes.

Another way @stubailo and I have described it is: "infinitely fast/cheap polling".

[smolinari]

Here is my definition.

A live query is a query, which is designated by the client as "live". This designation is passed on to the GraphQL server (one could say it is a subscription). The server then observes for triggers or data input from the underlying data sources needed to fulfill any part of the query. This in turn means any updates from the underlying data sources will be passed to the client automatically via bi-directional communication.

Scott

[paralin]

If you want a demo of a live query, complete with a GraphiQL editor, see http://rgraphql.org

Live queries are infinitely more powerful than subscriptions because you can model live, reactive data in a way that efficiently encodes changes all the way from the data source to the browser, into something like react and angular. And it's not true that they cannot be done at scale - is definitely possible with a good enough scheduler / balancer.

[stubailo]

Live queries are infinitely more powerful than subscriptions

I agree with this.

And it's not true that they cannot be done at scale - is definitely possible with a good enough scheduler / balancer.

I also agree with this.

However, I think they are significantly different from subscriptions nonetheless, and both have their place in the GraphQL ecosystem.

[paralin]

However, I think they are significantly different from subscriptions nonetheless, and both have their place in the GraphQL ecosystem.

Agreed. I don't think live queries are necessarily difficult to do, though, either - the argument I'm arguing against is - "live queries at scale isn't a solved science, so we're going to ignore the concept entirely."

[robzhu]

"live queries at scale isn't a solved science, so we're going to ignore the concept entirely."

I don't think anyone is ignoring the concept. But "live queries at scale isn't a solved science" has some truth to it. We hope to share more details in the coming months as we continue to learn from our live query experiments at Facebook. However, assuming live queries work perfectly, we believe live queries and subscriptions are different tools in the real-time API toolbox.

[stubailo]

Yeah I think it's important that the spec proposal doesn't say "this is the only thing we will ever do for realtime data" or even "this is the best way to do realtime data" - it should just say "this is the way of doing realtime data that is understood clearly enough to specify"

[robzhu]

For example, in my case, where I have many microservices on my backend, where some nested fields go to other services, it's not really straightforward to define how live queries would work, and I've chosen explicitly to model things as event streams.

@taion I'm curious to know if you think of subscriptions and live queries as semantically different. Suppose you had both subscriptions and live queries at your disposal, when would you use one over the other?

[smolinari]

That is a great question, but it blows my definition of live queries out of the water. Doesn't it? Hehehe... LOL! 😄

If I may answer too. I think with Christian's (@paralin) rqraphql system, live queries are a server-side and domain specific decision. From what I understand from the rgraphql docs, if you want a live query, the ability to observe for updates is "baked" into one or more resolvers for that query. And, I believe this is where this concept has a general concern (and something still missing in the spec too). It requires a front-end dev to have intimate knowledge of the back-end decisions, as the type of query (live or not) cannot be directly "seen" through introspection, whereas, it should be. Sure, one could add some type of comment, but is that really a good solution for flagging queries as "potentially live" with introspection?

The other question that burns in my mind is, how does the server know who to broadcast these updates to? The docs mention killing long running processes. That is only scratching the scaling issue.

I guess I am the stupid guy on the fence between these two solutions. I don't think GraphQL should be working with events internally. They aren't needed, as Christians's rgraphql system proves. Yet, I don't think pure live queries, without some sort of subscription system, are also the right solution either.

Oh. And just because a live query has a subscription system tagged to it, doesn't mean it can't be called a live query. 😉

Scott

[taion]

@robzhu

I do think of them as semantically different. It would be awkward to do a toast notification with a live query rather than an event-based subscription stream, for example.

That said, I am mostly using subscriptions as a poor man's live query system, with easier-to-understand semantics on the backend. If I had a reactive backend that supported live queries, I would mostly move to using live queries – but I don't, and I decided it wasn't worth the architectural trade-offs required to do so.

Additionally, I expect the majority of users of GraphQL subscriptions as-is to use them to do something somewhat similar to my use case of emulating live queries in an easier-to-implement manner for complex back end systems.