Expanding Community Involvement & Contribution

[benjchristensen]

Earlier this year we expanded the committers of RxJava to include non-Netflix engineers. I'd like to now further explore how we can broaden community involvement and lessen dependence on me. The recent "RxJava Truck Factor" issue was a reminder of this topic and pushed several of us inside Netflix to raise the priority on tackling this.

First, a note on how great the community has been thus far! We have had great contributors, several having contributed significant time, effort and skill and without whom RxJava would not be as good as it is. Many who are not shown with "code contributions" have helped immensely in answering questions on Twitter, StackOverflow, Google Groups, blogging, speaking at meetups and conferences, or helping build reactivex.io. In short, the community around RxJava is already great.

However, I'd like to achieve a few things:

1. Make it possible for others to fulfill the "release engineer" role I'm still playing.
2. Ensure there are no irreplaceable people on the project.
3. Improve the reliability for getting code reviews and questions answered in a reasonable time frame.
4. Improve the release cadence so it is more reliable, and less dependent on me getting bursts of time.
5. Make it easier for community to get involved by pointing to where they can help in ways other than writing code.
6. Find a way to recognize people for involvement in the many important areas other than lines of code committed.

A few ideas on making these happen include:

• More engineers from Netflix with time formally allocated. I will bootstrap them and ensure they have everything needed to do releases without me.
• More comprehensive CONTRIBUTING.md (Rust has a good example: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md)
• Ask community to help with StackOverflow, Google Groups, Issue/PR Triage, Twitter (and perhaps recognize people somehow who do these things?)

A key thing I want people to understand is that writing code is actually the easiest part of maintaining a project like this. The lifecycle is far harder and more time consuming. This includes:

• code reviews
• answering questions
• deciding whether a change should be made or not
• discussing design decisions and deciding which tradeoffs to make
• balancing change with stability
• writing documentation
• debugging issues
• replicating bugs with unit tests
• answering more questions
• tutorials, blogs, speaking, training, etc

All who are interested in RxJava, what ideas do you have for helping us improve and achieve the above? What else should we pursue and improve?

[benjchristensen]

Netflix Engineers involved in this discussion include: @stevegury @abersnaze @stealthcode @mattrjacobs

I am making sure they have all necessary credentials for Travis, BinTray and Maven Central so that in my absence, if there are issues with the automated pipeline, they can debug and fix. Additionally they will get the Twitter credentials so they can post to Twitter for releases and other such information.

We are working amongst ourselves to allocate time each week per engineer to help with code reviews (pull requests), Github issues, and releases.

[benjchristensen]

Various places of community engagement:

Google Group: https://groups.google.com/forum/#!forum/rxjava
Gitter: https://gitter.im/ReactiveX/RxJava
StackOverflow: http://stackoverflow.com/search?q=rx-java
Twitter: https://twitter.com/search?q=rxjava
Github: https://github.com/ReactiveX/RxJava/issues
Speakerdeck: https://speakerdeck.com/search?utf8=✓&q=rxjava

[sudaredd]

Benjchristensen, I'm not from Netflix and would like to contribute to RxJava.
Can you pls direct me how to contribute to RxJava?

Tks
Sudar

[davidmoten]

@sudaredd https://github.com/ReactiveX/RxJava/wiki/How-to-Contribute is a start (we could probably add more info to that page)

[hamen]

I like this, guys! Kudos! RxJava is a huge effort. @benjchristensen needs all the help we can give him.
Obviously, I expect I will get a cool Netflix t-shirt as bug hunter, funny conference speaker and books author 👅

[benjchristensen]

I expect I will get a cool Netflix t-shirt as bug hunter, funny conference speaker and books author

We'll see what we can do! :-)

@sudaredd The link @davidmoten gave is the current starting point. I'm trying to mature that with this discussion and figure out how we can improve community engagement.

The best way to get started in the project is to help us answer questions at the many places above, help triage and prove bugs (and fixes), and improve documentation.

[benjchristensen]

Anyone have suggestions on how to execute on my stated goals?

[hamen]

Here are my 2 cents:

• Knowing that Netflix is offering more full-time people to the project, I think that the releasing task should be still safely handled by Netflix;
• From a coding point of view, Github should handle the contribution recognition properly: PR, comments, tech discussions will shows active users;
• From a supporting point of view, StackOverflow offers tags and top users
  http://stackoverflow.com/tags/rx-java/topusers
  Btw, congrats @davidmoten! @mrsasha we definitely need to spend more time on SO 😄
• We could use the wiki to create a collection of useful teaching contents: videos, tutorials, example, and try to involve those authors
• A bit of marketing, promoting this new project with Android Weekly, for instance, and other equivalent Java communities.

What do you think?

[artem-zinnatullin]

Anyone have suggestions on how to execute on my stated goals?

->

3. Improve the reliability for getting code reviews and questions answered in a reasonable time frame.

There are 21 open pull requests, some of them are really old, also there are 166 open issues…

And looks like that at the moment, the only active (comments, code reviews and PRs almost every day) person in the project is David Karnok @akarnokd and I guess he has other things to deal with, so it will be good to add at least one another active person to the project.

[stealthcode]

@artem-zinnatullin that is certainly a motivating factor behind adding Netflix engineers to the project and by expanding the community involvement. We want to insure that issues and pull requests are responded to in a timely manner.

[benjchristensen]

Thanks @hamen for the input.

I think that the releasing task should be still safely handled by Netflix;

Agreed.

From a coding point of view, Github should handle the contribution recognition properly: PR, comments, tech discussions will shows active users;

Agreed. Though would it be helpful to call out in a "TEAM" doc those who are actively engaged so others know who to reach out to for different aspects of the project? For example, those for documentation, build tooling, release, etc?

From a supporting point of view, StackOverflow offers tags and top users

Good point. Though not everything is on StackOverflow.

The part I'm wondering about is whether it makes sense to have individuals called out as being part of "the team". If someone is involved, but then drifts away, perhaps we have a 3 month expiry so that if we haven't seen someone for a while, we move them into "emeritus" status so we recognize their involvement from the past, but let people know they aren't actively involved.

Does something like that make sense?

[benjchristensen]

Had a working meeting at Netflix today to start transferring domain knowledge that resulted in v1.0.14 being released, without me being the one pushing the buttons.

Going forward the intent is to have the Netflix RxJava team as:

• @stevegury
• @mattrjacobs
• @stealthcode
• @abersnaze
• @benjchristensen

I am bringing them up to speed on release and communication roles, and they will start involving themselves in pull requests. Over the next week we are going to triage the backlog of open issues.

This should help address the first 4 goals:

1. Make it possible for others to fulfill the "release engineer" role I'm still playing.
2. Ensure there are no irreplaceable people on the project.
3. Improve the reliability for getting code reviews and questions answered in a reasonable time frame.
4. Improve the release cadence so it is more reliable, and less dependent on me getting bursts of
   time.

Still need to figure out the others ...

[hamen]

If someone is involved, but then drifts away, perhaps we have a 3 month expiry so that if we haven't seen someone for a while, we move them into "emeritus" status so we recognize their involvement from the past, but let people know they aren't actively involved.

Agreed. We could schedule "involved developers" list updates every quarter as a "development task" (a Github issue could work).

Good point. Though not everything is on StackOverflow.

I have asked Github support guys if they have some kind of statistics for repo wiki. We could have users helping on the wiki and it will be fair to recognize them.

Had a working meeting at Netflix today to start transferring domain knowledge that resulted in v1.0.14 being released, without me being the one pushing the buttons.

Very nice, guys! Updating build.gradle right now.
Looking forward.

[hamen]

Github reply came in: it's not possible at the moment. We will figure out something else then.

[mrsasha]

From the "release engineering" point of view, maybe it would be useful to schedule some kind of release runs where a few people would go through PRs and issues and try to solve them before the release, and triaging the old ones to see if they're still applicable or not? Like a tentative release every month and a week before that is just for issues and open PRs?

As for contributing, I might have missed it, but I didn't see a style guide anywhere. Is there one?

[schrepfler]

Not sure of the actual entity and library impact but I think @davidmoten helped quite a bit in this last period.

[ZacSweers]

I'd like to revisit this discussion. It's about 6 months on from the initial discussion, and it seems that @akarnokd is still the only active collaborator on the project. I've had side discussions with a lot of fellow engineers, and some are starting to worry about the future of this project given the passiveness from Netflix on this since @benjchristensen left. 2.0 doesn't seem any closer with only three commits since October, and the future beyond that seems largely unclear (especially in regards to Java 9's Flow API).

This becomes particularly obvious when you consider that there was an agreement some time ago that 👍's from two collaborators are enough to get something merged, but there's only ~two actively reviewing collaborators period. There are 31 open PRs and a further 160+ open issues.

I'm not sure what the solution to this is, but the current trajectory seems that the project is beginning to stagnate due to a lack of active maintainers and leadership.

Some stats since August 1st, 2015

Generated from a script I wrote up for this comment: https://gist.github.com/hzsweers/1453fdcd92b4ab689021

Commits to 1.x

Who's moving the project forward.

122 akarnokd
36  stealthcode
23  abersnaze
23  artem-zinnatullin
13  zsxwing
8   davidmoten
5   DavidMGross
5   stevegury
3   hyleung
3   Turbo87
2   JakeWharton
2   vqvu
2   Chaoba
2   benjchristensen
2   vanniktech

Comments on issues

Who's maintaining the repo and engaging the community.

224 akarnokd
102 benjchristensen
76  davidmoten
56  artem-zinnatullin
28  stealthcode
28  zsxwing
26  JakeWharton
19  abersnaze
13  thomasnield
10  headinthebox
8   srvaroa
8   NachoSoto
8   Chaoba
7   NiteshKant
6   eirslett

Comments on PRs

Who's reviewing code.

27  akarnokd
26  artem-zinnatullin
10  davidmoten
7   thomasnield
4   zsxwing
3   srvaroa
2   JakeWharton
2   stevegury
1   JohnWowUs
1   msavitskiy
1   bcorne

Opened PRs

Who's moving the project forward.

18  akarnokd
2   davidmoten
2   artem-zinnatullin
1   thomasnield
1   JohnWowUs
1   msavitskiy
1   myArea51
1   bcorne
1   srvaroa
1   achinthagunasekara
1   Chaoba

Total contributions

391 akarnokd
107 artem-zinnatullin
104 benjchristensen
96  davidmoten
64  stealthcode
45  zsxwing
42  abersnaze
30  JakeWharton
21  thomasnield
12  srvaroa
11  Chaoba
10  stevegury
10  headinthebox
9   DavidMGross
8   NachoSoto

(This weighs a PR equal to comments, which is a little misleading. I don't know what fair weights would be, but considering PRs are the lifeblood of the repo, akarnokd's factor would be more through-the-roof than it already is)

[digitalbuddha]

Chiming in from NY Times. We were one of the early adopters and would hate to see project stagnate as we are very heavily invested. Artem was recently hired by my team (framework). We'd love to be more involved as contributors/collaborators to this project

[JakeWharton]

I read every issue and review every PR but refrain from commenting on 99% of them since my only issues are nitpick or style related which is just annoying (to other contributors) and there's no other value commenting since I'm not one of the contributors whose votes are required.

As a non-contributor reviewer, though, it's annoying to review every PR only to see them sit for long periods of time.

[akarnokd]

Yes, 2.0 is standing still, maybe its too big for anyone to review it.

2.0 doesn't seem any closer with only three commits since October

And 11 other waiting in line. I gave up on it because fixes started to depend on the order the PRs have to be merged and I don't want to rebase all the time.

As a non-contributor reviewer, though, it's annoying to review every PR only to see them sit for long periods of time.

As an active contributor, it makes me annoyed having things sitting there as well. I, lately, tend to forget what fix I posted to what project and what version and sometimes find myself unable to move because a fix or the addition of some utility in one PR is required by some other PR.

In addition, things have evolved beyond my 2.0 implementation over in reactive-streams-commons project, but I have hard time to imagine how those enhancements will ever get back into RxJava (any version) at the moment.

The policy on the review process really slowed things down; I'm on the favor of merge first, discuss later since there is the release-gate as the final resort to stop something very wrong. But if RxJava went on the release-early release-often path, even that wouldn't be an issue.

Certainly, @zsxwing (contributor so his 👍 counts) has become more active recently but generally @artem-zinnatullin does a good job at review too. I suggest accepting @artem-zinnatullin 's 👍 also towards merging a PR. Of course, due to timezone differences, some non-trivial PRs may need to wait 16 - 24 hours so all contributors have a chance to look at it.

Even though this speeds up merging PRs, the release right is still in Netflix' hand (but technically it's just pressing a button).

The benefit of having Neflix' banner over the project is that developers tend to chose libraries from big names over unknown Eastern European solo guy, but the drawback is that the project mostly mirrors the internal expectations and requirements of Netflix' own use and involves the risk that their use 'breaks' due to the community moving into a different direction. The other side of the risk is that Netflix can't cherry-pick fixes to support their own process from a free-running project with intertwined fixes and enhancements.

I've been thinking a lot about forking RxJava lately, which I technically did with the RxJava 2 backport and the JDK 9 conversion (thus I'm covered); but RxJava is the foundation to many other projects, RxNetty, RxAndroid, Rx..., that have to decide to switch as well. Otherwise, any fork without them is practically useless for real world use.

This affects 1.x mostly because reactive-streams is general enough you can have interoperating implementations thus using the RxJava 2.0 backport, Rsc, Project Reactor or Akka-Streams mixed is quite possible.

So what's the solution? As Microsoft let RxJS go, maybe Netflix should let RxJava go, or at least allow pressing the release button by other contributors.

[davidmoten]

I've been contributing for about two years now to this project and we do seem to be in a slow patch. The project is big enough that a full-time investment from Netflix wouldn't go astray if they seek to have input in such a potentially fast developing codebase (thanks largely to @akarnokd). I've been very happy with @akarnokd's high quality major contributions and oversight. He's clearly able to spend a lot of time on the project and I'd support him having more freedom with merging PRs. Two thumbs up from committers is clearly not working well enough, I assume the Netflix committers have other tasks on their plate (as do most of us!). A big thank you to @zsxwing for weighing in a lot more recently and to new guy @artem-zinnatullin for throwing himself at stuff too.

Like @JakeWharton I read every issue and PR that comes through and review and comment on the ones that interest me. I'd expect non-trivial PRs to be open for a short period to attract input and review but after that period I think @akarnokd's judgement generally is too good to prevent him from merging. There's nothing stopping somebody saying "don't merge this for a week or so I want to have a good look at it".

Recent history has curbed my contributions I should add. The version 2.0 explosion of commits stopped me in my tracks (so much to review and why contribute to 1.0 if 2.0 is just around the corner) as did the introduction of a bunch of new committers (curious about a new dynamic). 1.0 seems alive and well still so I may contribute more to it.

@akarnokd has some confidence in 2.0, leave the merges to him and push out an alpha release? Might at least attract more review and experimentation if at that point.

I'd definitely support much more frequent releases and think it's time Netflix shared the release button or dedicated resources to ensure that this happens.

[stevegury]

Be assured that Netflix is committed to making RxJava a success. Maybe more than any other company, most of our infrastructure rely on it, and we don't want to see it fail. @benjchristensen left a couple of months ago, and we needed time to reorganize the teams around RxJava.

I'm not opposed to giving more freedom to other external contributors, I need to discuss this internally with the management.
I think that @akarnokd is right to say that having Netflix behind RxJava give more credibility to the project, and, on the other hand, we want to weight in the technical decision made about the project.

[ZacSweers]

@stevegury that's reassuring to hear.

I don't think the release frequency has been much of a problem though, there have still been a good few releases in the past 6 months. I think that giving collaborators a little more involvement in the release process is good, but I think the most significant missing element right now is proper stewardship of the repo. Aside from @benjchristensen's code contributions, he was a great shepherd of the project itself. All issues were triaged, issues and PRs were discussed at length, others were called upon for their input, and the direction of the repo was largely clear. In a sense, the train was always chugging on. I think it's safe to say that much of RxJava's flourishing over the past 2-3 years was due to that incubation.

The impression I have is that other Netflix engineers were supposed to step into that maintainer role after Ben left, but that doesn't appear to have happened (at least not nearly to the same degree). I don't know who the right person would be for it if not Netflix (though there are some obvious potential candidates here), but I would be interested to hear your thoughts on it.

[stevegury]

Not yet nearly as the same degree indeed.
I'll make sure to dedicate myself more on this task.

[Lakedaemon]

2014 was RxJava's year on tweeter (2015 is Docker's) and countless developers have seen the greatness of declarative and reactive programming...

As a simple noname developer among many, I don't care if Netflix is behind RxJava.
I'm already using the stuff. It kicks ass.

What I want is bug fixes, more performance, great apis, some new nice features if possible but not bloat, high quality code, simple and painless upgrade paths, maintainers who care and know what they are doing, ...

I'm really grateful to Microsoft to have invented Rx and to Netflix to have brought it to Java.
But in case the ship was sinking or slowing down to a crawl and there was a healthy fork, battle tested, going forward and maintained by a few caring and dedicated maintainers (even if one of them is living in some unheard of village in eastern Europe), that would just require a package change to join, I would jump ship in an instant.

Just felt like chiming in what other RxJava's users might think...
please excuse me if I'm being rude (that was not my intention).

[ZacSweers]

@stevegury Do you think you/Netflix would be able to put together some more concrete plans going forward?

[stevegury]

Yes!

[akarnokd]

Closing in favor of #4013.