Skip to content

Sorting result.Issues implementation (golangci/golangci-lint#1217) #1218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pkg/commands/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ func initFlagSet(fs *pflag.FlagSet, cfg *config.Config, m *lintersdb.Manager, is
fs.BoolVar(&oc.PrintIssuedLine, "print-issued-lines", true, wh("Print lines of code with issue"))
fs.BoolVar(&oc.PrintLinterName, "print-linter-name", true, wh("Print linter name in issue line"))
fs.BoolVar(&oc.UniqByLine, "uniq-by-line", true, wh("Make issues output unique by line"))
fs.BoolVar(&oc.SortResults, "sort-results", false, wh("Sort linter results"))
fs.BoolVar(&oc.PrintWelcomeMessage, "print-welcome", false, wh("Print welcome message"))
fs.StringVar(&oc.PathPrefix, "path-prefix", "", wh("Path prefix to add to output"))
hideFlag("print-welcome") // no longer used
Expand Down
1 change: 1 addition & 0 deletions pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -519,6 +519,7 @@ type Config struct {
PrintIssuedLine bool `mapstructure:"print-issued-lines"`
PrintLinterName bool `mapstructure:"print-linter-name"`
UniqByLine bool `mapstructure:"uniq-by-line"`
SortResults bool `mapstructure:"sort-results"`
PrintWelcomeMessage bool `mapstructure:"print-welcome"`
PathPrefix string `mapstructure:"path-prefix"`
}
Expand Down
1 change: 1 addition & 0 deletions pkg/lint/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ func NewRunner(cfg *config.Config, log logutils.Log, goenv *goutil.Env, es *lint
processors.NewPathShortener(),
getSeverityRulesProcessor(&cfg.Severity, log, lineCache),
processors.NewPathPrefixer(cfg.Output.PathPrefix),
processors.NewSortResults(cfg),
},
Log: log,
}, nil
Expand Down
173 changes: 173 additions & 0 deletions pkg/result/processors/sort_results.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
package processors

import (
"sort"
"strings"

"github.com/golangci/golangci-lint/pkg/config"
"github.com/golangci/golangci-lint/pkg/result"
)

// Base propose of this functionality to sort results (issues)
// produced by various linters by analyzing code. We achieving this
// by sorting results.Issues using processor step, and chain based
// rules that can compare different properties of the Issues struct.

var _ Processor = (*SortResults)(nil)

type SortResults struct {
cmp comparator
cfg *config.Config
}

func NewSortResults(cfg *config.Config) *SortResults {
// For sorting we are comparing (in next order): file names, line numbers,
// position, and finally - giving up.
return &SortResults{
cmp: ByName{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understand that your intention is to compose the sorting rules. However, if I looked at the Comparer we have now ByName along.

               ByName{
			next: ByLine{
                        ...
			},
		}

Comparing ByName implicitly checking Line number due to composition, but will ByName comparator should only deal with names, it doesn't require to know the next rule should be.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is Chain of Responsibility pattern, current resolver only needs to know about the exitance of the next one (res.isNeutral() we calling defines should we proceed to the next rule or not, but rules are different, strings compare, single numbers compare (line and column), two numbers(range) ). Guess we suppose to see the Comparator interface, instead of chain of its implementations. ByName or any other implementation of Comparator needs to know only that there is the next rule in the chain.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if chain of responsibility pattern is correct one to use here. Maybe I am mixing two different things (comparator interface + design pattern), but if we can interface as Comparator, it should just to compare two entities and nothing else.

The logic of calling next comparator is kind of repetitive in currently (e.g. check res is neutral or not, if not check the next one is there, then call next one), so adding new Comparator will require a fairly some level of understanding of existing one.

Few snippets (based on your code) to illustrate my thinking:

type SortResults struct {
	compareFuncs []compareFunc
	cfg          *config.Config
}

type compareFunc func(result.Issue, result.Issue) compareResult

func NewSortResults(cfg *config.Config) *SortResults {
        // single responsibility function  
	byName := func(first result.Issue, second result.Issue) compareResult {
		return compareResult(strings.Compare(first.FilePath(), second.FilePath()))
	}

	byLine := func(first result.Issue, second result.Issue) compareResult {
		return compareResult(first.Line() - second.Line())
	}

	byCol := func(first result.Issue, second result.Issue) compareResult {
		return compareResult(first.Column() - second.Column())
	}

	byRange := func(first result.Issue, second result.Issue) compareResult {
		return compareResult(first.GetLineRange().From - second.GetLineRange().From)
	}

	return &SortResults{
		compareFuncs: []compareFunc{
			byName,
			byLine,
			byCol,
			byRange,
                        ... --> we can add more type of compareFunc here
		},
		cfg: cfg,
	}
}

// Process is performing sorting of the result issues.
func (sr SortResults) Process(issues []result.Issue) ([]result.Issue, error) {
	if !sr.cfg.Output.SortResults {
		return issues, nil
	}

	sort.Slice(issues, func(i, j int) bool {
		for _, c := range sr.compareFuncs {
			if res := c(issues[i], issues[j]); !res.isNeutral() {
				return res <= less
			}
		}
		return false
	})
	return issues, nil
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to use CoR in order to avoid making logic complicated (cmon who what to see 3-5 blocks of different if statements ?), CoR helps to pass the request along a chain of handlers. More of that - I can test each of them. And logic isn't really simple here (what if linter don't provide column? or line?) I did try to cover it with tests.

func numericCompare(a, b int) CompareResult {
	var (
		isValuesInvalid  = a < 0 || b < 0
		isZeroValuesBoth = a == 0 && b == 0
		isEqual          = a == b
		isZeroValueInA   = b > 0 && a == 0
		isZeroValueInB   = a > 0 && b == 0
	)

	switch {
	case isZeroValuesBoth || isEqual:
		return Equal
	case isValuesInvalid || isZeroValueInA || isZeroValueInB:
		return None
	case a > b:
		return Greater
	case a < b:
		return Less
	}

	return Equal
}

I see what you have written, and got the idea, and I wish I have more time for opensource, to rethink the structure and implement all checks, write new tests, cover edge cases to pleasure community, and maintainers in order to keep code quality high. But my family wouldn't agree with me. I am really sorry for wasting your time with this PR. Please reject this PR if you think code quality is way lower the level you will accept.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you have written, and got the idea, and I wish I have more time for opensource, to rethink the structure and implement all checks, write new tests, cover edge cases to pleasure community, and maintainers in order to keep code quality high. But my family wouldn't agree with me. I am really sorry for wasting your time with this PR. Please reject this PR if you think code quality is way lower the level you will accept.

Appreciate that you spend time to send PR with a very good coverage 💯 . Yeah, I personally understand your point about OSS involvement. While only few are lucky to have OSS as paid job, the rest of us (including myself) are voluntarily doing it either during night, weekends or our own spare time. In my opinion, OSS PR process is a chance to have open discussion, and in turn learn from each other. Sometimes it can take a little longer than we expect.

I could be a bit opinionated, or even wrong on this subject. So, let me park it right here, we can see if someone else from @golangci/team has any comment/idea in general.

In the meantime, I can continue reviewing on the basis that CoR is used. Hope it is fine to you.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Appreciate this. Using your way to solve the issue will require additional time to think about implementation and tests, and actually require to throw my code out almost entirely.

next: ByLine{
next: ByColumn{},
},
},
cfg: cfg,
}
}

// Process is performing sorting of the result issues.
func (sr SortResults) Process(issues []result.Issue) ([]result.Issue, error) {
if !sr.cfg.Output.SortResults {
return issues, nil
}

sort.Slice(issues, func(i, j int) bool {
return sr.cmp.Compare(&issues[i], &issues[j]) == Less
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Common way is just comparing to zero. We can simplify consts as well.

Copy link
Member Author

@butuzov butuzov Jul 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same reasons as in res.isNeutral() related comment. Read it as a story: If the results of comparing a and b state Less, compare function receives true. I would keep names constants as it really simplifies the understanding of the code for a small price.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tricky one. Will it be "<=" instead of "==" ?

I didn't scan for all Comparator implementation to make sure that negative result will only be -1

})

return issues, nil
}

func (sr SortResults) Name() string { return "sort_results" }
func (sr SortResults) Finish() {}

type compareResult int

const (
Less compareResult = iota - 1
Equal
Greater
None
)

func (c compareResult) isNeutral() bool {
// return true if compare result is incomparable or equal.
return c == None || c == Equal
}

//nolint:exhaustive
func (c compareResult) String() string {
switch c {
case Less:
return "Less"
case Equal:
return "Equal"
case Greater:
return "Greater"
}

return "None"
}

// comparator describe how to implement compare for two "issues" lexicographically
type comparator interface {
Compare(a, b *result.Issue) compareResult
Next() comparator
}

var (
_ comparator = (*ByName)(nil)
_ comparator = (*ByLine)(nil)
_ comparator = (*ByColumn)(nil)
)

type ByName struct{ next comparator }

//nolint:golint
func (cmp ByName) Next() comparator { return cmp.next }

//nolint:golint
func (cmp ByName) Compare(a, b *result.Issue) compareResult {
var res compareResult

if res = compareResult(strings.Compare(a.FilePath(), b.FilePath())); !res.isNeutral() {
return res
}

if next := cmp.Next(); next != nil {
return next.Compare(a, b)
}

return res
}

type ByLine struct{ next comparator }

//nolint:golint
func (cmp ByLine) Next() comparator { return cmp.next }

//nolint:golint
func (cmp ByLine) Compare(a, b *result.Issue) compareResult {
var res compareResult

if res = numericCompare(a.Line(), b.Line()); !res.isNeutral() {
return res
}

if next := cmp.Next(); next != nil {
return next.Compare(a, b)
}

return res
}

type ByColumn struct{ next comparator }

//nolint:golint
func (cmp ByColumn) Next() comparator { return cmp.next }

//nolint:golint
func (cmp ByColumn) Compare(a, b *result.Issue) compareResult {
var res compareResult

if res = numericCompare(a.Column(), b.Column()); !res.isNeutral() {
return res
}

if next := cmp.Next(); next != nil {
return next.Compare(a, b)
}

return res
}

func numericCompare(a, b int) compareResult {
var (
isValuesInvalid = a < 0 || b < 0
isZeroValuesBoth = a == 0 && b == 0
isEqual = a == b
isZeroValueInA = b > 0 && a == 0
isZeroValueInB = a > 0 && b == 0
)

switch {
case isZeroValuesBoth || isEqual:
return Equal
case isValuesInvalid || isZeroValueInA || isZeroValueInB:
return None
case a > b:
return Greater
case a < b:
return Less
}

return Equal
}
182 changes: 182 additions & 0 deletions pkg/result/processors/sort_results_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
package processors

import (
"fmt"
"go/token"
"testing"

"github.com/stretchr/testify/assert"

"github.com/golangci/golangci-lint/pkg/config"
"github.com/golangci/golangci-lint/pkg/result"
)

var issues = []result.Issue{
{
Pos: token.Position{
Filename: "file_windows.go",
Column: 80,
Line: 10,
},
},
{
Pos: token.Position{
Filename: "file_linux.go",
Column: 70,
Line: 11,
},
},
{
Pos: token.Position{
Filename: "file_darwin.go",
Line: 12,
},
},
{
Pos: token.Position{
Filename: "file_darwin.go",
Column: 60,
Line: 10,
},
},
}

type compareTestCase struct {
a, b result.Issue
expected compareResult
}

func testCompareValues(t *testing.T, cmp comparator, name string, tests []compareTestCase) {
t.Parallel()

for i := 0; i < len(tests); i++ {
test := tests[i]
t.Run(fmt.Sprintf("%s(%d)", name, i), func(t *testing.T) {
res := cmp.Compare(&test.a, &test.b)
assert.Equal(t, test.expected.String(), res.String())
})
}
}

func TestCompareByLine(t *testing.T) {
testCompareValues(t, ByLine{}, "Compare By Line", []compareTestCase{
{issues[0], issues[1], Less}, // 10 vs 11
{issues[0], issues[0], Equal}, // 10 vs 10
{issues[3], issues[3], Equal}, // 10 vs 10
{issues[0], issues[3], Equal}, // 10 vs 10
{issues[3], issues[2], Less}, // 10 vs 12
{issues[1], issues[1], Equal}, // 11 vs 11
{issues[1], issues[0], Greater}, // 11 vs 10
{issues[1], issues[2], Less}, // 11 vs 12
{issues[2], issues[3], Greater}, // 12 vs 10
{issues[2], issues[1], Greater}, // 12 vs 11
{issues[2], issues[2], Equal}, // 12 vs 12
})
}

func TestCompareByName(t *testing.T) { //nolint:dupl
testCompareValues(t, ByName{}, "Compare By Name", []compareTestCase{
{issues[0], issues[1], Greater}, // file_windows.go vs file_linux.go
{issues[1], issues[2], Greater}, // file_linux.go vs file_darwin.go
{issues[2], issues[3], Equal}, // file_darwin.go vs file_darwin.go
{issues[1], issues[1], Equal}, // file_linux.go vs file_linux.go
{issues[1], issues[0], Less}, // file_linux.go vs file_windows.go
{issues[3], issues[2], Equal}, // file_darwin.go vs file_darwin.go
{issues[2], issues[1], Less}, // file_darwin.go vs file_linux.go
{issues[0], issues[0], Equal}, // file_windows.go vs file_windows.go
{issues[2], issues[2], Equal}, // file_darwin.go vs file_darwin.go
{issues[3], issues[3], Equal}, // file_darwin.go vs file_darwin.go
})
}

func TestCompareByColumn(t *testing.T) { //nolint:dupl
testCompareValues(t, ByColumn{}, "Compare By Column", []compareTestCase{
{issues[0], issues[1], Greater}, // 80 vs 70
{issues[1], issues[2], None}, // 70 vs zero value
{issues[3], issues[3], Equal}, // 60 vs 60
{issues[2], issues[3], None}, // zero value vs 60
{issues[2], issues[1], None}, // zero value vs 70
{issues[1], issues[0], Less}, // 70 vs 80
{issues[1], issues[1], Equal}, // 70 vs 70
{issues[3], issues[2], None}, // vs zero value
{issues[2], issues[2], Equal}, // zero value vs zero value
{issues[1], issues[1], Equal}, // 70 vs 70
})
}

func TestCompareNested(t *testing.T) {
var cmp = ByName{
next: ByLine{
next: ByColumn{},
},
}

testCompareValues(t, cmp, "Nested Comparing", []compareTestCase{
{issues[1], issues[0], Less}, // file_linux.go vs file_windows.go
{issues[2], issues[1], Less}, // file_darwin.go vs file_linux.go
{issues[0], issues[1], Greater}, // file_windows.go vs file_linux.go
{issues[1], issues[2], Greater}, // file_linux.go vs file_darwin.go
{issues[3], issues[2], Less}, // file_darwin.go vs file_darwin.go, 10 vs 12
{issues[0], issues[0], Equal}, // file_windows.go vs file_windows.go
{issues[2], issues[3], Greater}, // file_darwin.go vs file_darwin.go, 12 vs 10
{issues[1], issues[1], Equal}, // file_linux.go vs file_linux.go
{issues[2], issues[2], Equal}, // file_darwin.go vs file_darwin.go
{issues[3], issues[3], Equal}, // file_darwin.go vs file_darwin.go
})
}

func TestNumericCompare(t *testing.T) {
var tests = []struct {
a, b int
expected compareResult
}{
{0, 0, Equal},
{0, 1, None},
{1, 0, None},
{1, -1, None},
{-1, 1, None},
{1, 1, Equal},
{1, 2, Less},
{2, 1, Greater},
}

t.Parallel()

for i := 0; i < len(tests); i++ {
test := tests[i]
t.Run(fmt.Sprintf("%s(%d)", "Numeric Compare", i), func(t *testing.T) {
res := numericCompare(test.a, test.b)
assert.Equal(t, test.expected.String(), res.String())
})
}
}

func TestNoSorting(t *testing.T) {
var tests = make([]result.Issue, len(issues))
copy(tests, issues)

var sr = NewSortResults(&config.Config{})

results, err := sr.Process(tests)
assert.Equal(t, tests, results)
assert.Nil(t, err, nil)
}

func TestSorting(t *testing.T) {
var tests = make([]result.Issue, len(issues))
copy(tests, issues)

var expected = make([]result.Issue, len(issues))
expected[0] = issues[3]
expected[1] = issues[2]
expected[2] = issues[1]
expected[3] = issues[0]

var cfg = config.Config{}
cfg.Output.SortResults = true
var sr = NewSortResults(&cfg)

results, err := sr.Process(tests)
assert.Equal(t, results, expected)
assert.Nil(t, err, nil)
}
Loading