bytes: tests are slow on arm

[alexbrainman]

I was watching windows/arm build output, and I noticed that bytes package tests are pretty slow

https://farmer.golang.org/temporarylogs?name=windows-arm&rev=311d87dbebbb0238196d3aa13fd9a37f655e1fc3&st=0xc4704774a0

ok  	bytes	146.447s

and then I looked at linux/arm too, and it is slow too

https://farmer.golang.org/temporarylogs?name=linux-arm&rev=311d87dbebbb0238196d3aa13fd9a37f655e1fc3&st=0xc4576f58c0

ok  	bytes	121.089s

I just felt sorry for the builders, and decided to create this issue.

I run byte tests here, and they are fast

c:\>go test -count=1 bytes
ok      bytes   1.106s

I also did not notice any particular test that is slow than others.

Maybe there is bytes package code that can be made faster? Maybe some tests can be adjusted to make arm builders faster? Maybe nothing can be done? Then just close the issue.

Thank you.

Alex

[mvdan]

I'm also puzzled as to why some packages on some builders take a very long time to test. Thanks for opening the issue!

[tklauser]

Running gotip test -v bytes on linux/arm (on a Toradex Apalis T30) gives:

https://gist.github.com/tklauser/576094d1635c1d3d18793ab3fbef9abf

PASS
ok  	bytes	101.559s

There's one obvious outlier:

=== RUN   TestCompareBytes
--- PASS: TestCompareBytes (88.86s)

I suspect that's due to missing SIMD implementations of internal/bytealg.Compare for GOARCH=arm.

[alexbrainman]

There's one obvious outlier:

=== RUN   TestCompareBytes
--- PASS: TestCompareBytes (88.86s)

Thanks for testing.

I suspect that's due to missing SIMD implementations of internal/bytealg.Compare for GOARCH=arm.

That sounds like reasonable explanation. I know nothing about arm instructions, so I would not know if it is possible to make this faster. If not, we should make TestCompareBytes shorter on arm - 2 minutes is too long for one single tests.

Leaving this for others to decide.

Alex

[tklauser]

I suspect that's due to missing SIMD implementations of internal/bytealg.Compare for GOARCH=arm.

That sounds like reasonable explanation. I know nothing about arm instructions, so I would not know if it is possible to make this faster. If not, we should make TestCompareBytes shorter on arm - 2 minutes is too long for one single tests.

Unfortunately, ARM vector instructions are not yet available in The Go assembler (#7300). However, from a quick look at the current implementation in internal/bytealg/compare_arm.s it looks like we could at least get some speedup by doing word-wise comparison (for word aligned strings) instead of always using byte-wise comparison.

[gopherbot]

Change https://golang.org/cl/153557 mentions this issue: bytes: avoid reallocations in TestCompareBytes

[gopherbot]

Change https://golang.org/cl/165338 mentions this issue: internal/bytealg: use word-wise comparison for Compare on arm

[tklauser]

Testing cmd/compile/internal/gc also takes a substantial amount of time (see https://gist.github.com/tklauser/6c265a73eb9ef310998e29ff976552b6 for the full log):

ok  	cmd/compile/internal/gc	159.902s

one of the biggest contributors is

--- PASS: TestCode (77.99s)

It seems the main culprit there is the bytes.Contains call (to determine whether to test using softfloat): which is not surprizing given that there is no arm-specific optimization for Index/IndexString in internal/bytealg. I'll try to add one.

[alexbrainman]

@tklauser your hard work achieves results. Recent windows-arm builder run

https://build.golang.org/log/32d08cb2c389db61937c461cbcf64172bd00bc84

ok  	bytes	60.869s

is more than twice as fast as before. Thank you very much.

Alex

[gopherbot]

Change https://golang.org/cl/166677 mentions this issue: internal/bytealg: share code for equal functions on arm