add a script for finding popular, but unused projects #29

adamdecaf · 2018-09-12T18:17:45Z

popular: the more godoc importers the more popular
unused: long time since last commit? few issue updates/comments

Issue: #14

adamdecaf · 2018-09-12T18:23:20Z

Here's an example run:

name                                           stars last commit (days) importers
github.com/go-martini/martini                  10264 478                2096
github.com/docker/libcontainer                 1037  664                1363
github.com/cihub/seelog                        1183  356                1118
github.com/bmizerany/pat                       1133  398                434
github.com/justinas/alice                      1630  329                328
github.com/google/gxui                         4138  1010               264
github.com/hoisie/web                          3122  454                166
github.com/braintree/manners                   828   461                140
github.com/bradfitz/http2                      1674  974                82
github.com/goraft/raft                         1883  1227               74
github.com/yhat/scrape                         1444  658                44
github.com/rcrowley/goagain                    1791  389                41
github.com/docker/libchan                      2329  326                30
github.com/rsms/gotalk                         884   295                15
github.com/gtank/cryptopasta                   1346  472                13
github.com/JoelOtter/termloop                  902   315                10
github.com/olebedev/when                       879   319                8
github.com/tleyden/open-ocr                    908   493                6
github.com/haxpax/gosms                        1157  437                6
github.com/pravj/geopattern                    961   495                6
github.com/alexflint/gallium                   3591  542                3
github.com/rakyll/coop                         1212  1076               2
github.com/codahale/sneaker                    784   610                1
github.com/kshvmdn/fsql                        3493  264                1
github.com/tobyhede/go-underscore              977   1175               1
github.com/mop-tracker/mop                     933   314                1

adamdecaf · 2018-09-12T18:26:05Z

Is this useful so far? I can easily add some sorting.

acln0

Looks nice. I'd like to request a few minor adjustments before we merge, though.

Besides the comments in the review, please squash the PR to one commit too.

Thanks.

acln0 · 2018-09-12T18:54:03Z

main.go

+	if err != nil {
+		return -1, fmt.Errorf("problem loading %s: %v", req.URL, err)
+	}
+	if resp.Body != nil {


The net/http documentation says "The http Client and Transport guarantee that Body is always non-nil, even on responses without a body or responses with a zero-length body."

I don't think we need this if statement.

acln0 · 2018-09-12T18:54:39Z

main.go

+
+	doc, err := html.Parse(resp.Body)
+	if err != nil {
+		panic(err)


Let's just return the error here. No need to panic.

acln0 · 2018-09-12T18:55:04Z

main.go

+func scrapeGodocImports(importPath string) (int, error) {
+	req, err := http.NewRequest("GET", "https://godoc.org/"+importPath, nil)
+	if err != nil {
+		panic(err)


Return -1, err please, instead of panicking.

acln0 · 2018-09-12T18:56:26Z

main.go

+		},
+	})
+	if err != nil {
+		panic(err)


Let's log err to standard error and bail out more gracefully than this. Avoid panic please.

acln0 · 2018-09-12T18:58:44Z

main.go

+	}
+}
+
+func createGithubClient(ctx context.Context) *github.Client {


I think this function should return an error. Currently, it prints a message to standard error if the github token isn't available, but then it continues creating the client regardless.

acln0 · 2018-09-12T18:59:33Z

main.go

+var (
+	ctx = context.Background()
+
+	ghClient = createGithubClient(ctx)


Move this inside of main, please, and let's handle the error it will return more gracefully, by bailing out, instead of continuing.

acln0 · 2018-09-12T19:00:48Z

main.go

+)
+
+var (
+	ctx = context.Background()


I think we should avoid this. For now, let's pass context.Background() to all things that need it, explicitly.

If the context we use changes in the future, it'll be easier to spot at a glance and adjust.

acln0 · 2018-09-12T19:04:19Z

Oh, something else too. I only just now noticed that this vendors dependencies, but it is using modules as well. Isn't that strange? I thought modules and vendoring were incompatible.

adamdecaf · 2018-09-12T21:19:07Z

@acln0 Thanks for the review. I dropped vendor/, but it's still used by go build. This script requires the internet already.

adamdecaf · 2018-09-12T21:47:59Z

There are a lot of -1's returned still, so I should probably look into that. I hope we can just make that refresh call and retry.

acln0 · 2018-09-13T08:45:18Z

Thanks. This looks pretty good now.

I guess the next question is if we want to have a main.go in the repository root, or if we should move it to cmd/something.

Ping @theckman for review and further thoughts.

theckman

If the option is between using modules and vendoring, I'd prefer we do what we need to have a functioning vendor directory. I'd like to make sure the binary is buildable no matter the state of any of its dependencies.

Can you please update this PR to add a vendor directory that ensures the script will build?

theckman · 2018-09-15T21:50:00Z

main.go

@@ -0,0 +1,115 @@
+package main


Let's make this cmd/finder/main.go or something similar.

Maybe add a // import "github.com/gofrs/help-requests/cmd/finder" comment here too?

Perhaps a package-level comment would be nice too, like here https://github.com/golang/tools/blob/master/cmd/stringer/stringer.go

Issue: gofrs#14

adamdecaf · 2018-09-17T15:43:29Z

Added vendor/ back. Updated the script a bit. There are still a lot of -1 godoc counts coming back. I think we should try a manual refresh on godoc.org to grab their count. Thoughts?

Edit: I updated the example: #29 (comment)

adamdecaf mentioned this pull request Sep 12, 2018

Review GitHub Go repositories with stars but old last commit #14

Closed

acln0 suggested changes Sep 12, 2018

View reviewed changes

adamdecaf force-pushed the init-script branch 2 times, most recently from 614b967 to 08a6a8c Compare September 12, 2018 21:07

theckman suggested changes Sep 15, 2018

View reviewed changes

adamdecaf force-pushed the init-script branch from 08a6a8c to 7220b86 Compare September 17, 2018 15:37

adding script to find popular, but unused projects

35923ca

Issue: gofrs#14

adamdecaf force-pushed the init-script branch from 7220b86 to 35923ca Compare September 17, 2018 15:43

add a script for finding popular, but unused projects #29

Are you sure you want to change the base?

add a script for finding popular, but unused projects #29

Uh oh!

Conversation

adamdecaf commented Sep 12, 2018

Uh oh!

adamdecaf commented Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adamdecaf commented Sep 12, 2018

Uh oh!

acln0 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

acln0 commented Sep 12, 2018

Uh oh!

adamdecaf commented Sep 12, 2018

Uh oh!

adamdecaf commented Sep 12, 2018 via email • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acln0 commented Sep 13, 2018

Uh oh!

theckman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamdecaf commented Sep 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

adamdecaf commented Sep 12, 2018 •

edited

Loading

adamdecaf commented Sep 12, 2018 via email •

edited

Loading

adamdecaf commented Sep 17, 2018 •

edited

Loading