Skip to content

Fix run command race #1470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 13, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 8 additions & 22 deletions modules/process/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,12 @@ package process

import (
"bytes"
"context"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only availably in 1.7+, we still need to support 1.6

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need to support 1.6?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because some developers (@strk for example) still run 1.6. Latest version in Ubuntu 16.04 LTS is 1.6 as well http://packages.ubuntu.com/xenial/golang-go

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1349 discussed before.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not liking it 😂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can vendor the package but the reason is less compelling since it doesn't affect end-users. Otherwise we can postpone the PR until Ubuntu catching up with the latest.

By the way, what keeps you (@bkcsoft @strk) using Ubuntu's Go rather than the official one? I'm curious if there's anything I could help.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next LTS is 18.04, which should include 1.7+. IMO we should wait for that until droping support for 1.6

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I've upgraded already, so don't think about me. Just think about random possible contributor staying safe in her latest debian/ubuntu/fedora stable system. We want to reduce the barriers of entry for any contributor.

We should also have Drone test all Go versions we want to support. For comparison Gogs is Travis-tested against Go versions from 1.5 to master.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree that strictly supporting old toolchains would reward the project's developers and users that much, if any. Just speaking for my own opinion though. ❤️

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

18.10 is released and have Go 1.8 so no need for this change.

"errors"
"fmt"
"os/exec"
"sync"
"time"

"code.gitea.io/gitea/modules/log"
)

// TODO: This packages still uses a singleton for the Manager.
Expand Down Expand Up @@ -101,7 +100,10 @@ func (pm *Manager) ExecDirEnv(timeout time.Duration, dir, desc string, env []str
stdOut := new(bytes.Buffer)
stdErr := new(bytes.Buffer)

cmd := exec.Command(cmdName, args...)
ctx, cancel := context.WithTimeout(context.Background(), timeout)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the timeout is negative? (Since the functions docs says to use -1 for DefaultTimeout (which we don't seems to honour... )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkcsoft The first line on this function has checked that. if timeout == -1 timout = 60 *time.Second

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want

if timeout < 0 {
    timeout = 60 * time.Second
}

?

defer cancel()

cmd := exec.CommandContext(ctx, cmdName, args...)
cmd.Dir = dir
cmd.Env = env
cmd.Stdout = stdOut
Expand All @@ -111,30 +113,14 @@ func (pm *Manager) ExecDirEnv(timeout time.Duration, dir, desc string, env []str
}

pid := pm.Add(desc, cmd)
done := make(chan error)
go func() {
done <- cmd.Wait()
}()

var err error
select {
case <-time.After(timeout):
if errKill := pm.Kill(pid); errKill != nil {
log.Error(4, "exec(%d:%s) failed to kill: %v", pid, desc, errKill)
}
<-done
return "", "", ErrExecTimeout
case err = <-done:
}

err := cmd.Wait()
pm.Remove(pid)

if err != nil {
out := fmt.Errorf("exec(%d:%s) failed: %v stdout: %v stderr: %v", pid, desc, err, stdOut, stdErr)
return stdOut.String(), stdErr.String(), out
err = fmt.Errorf("exec(%d:%s) failed: %v(%v) stdout: %v stderr: %v", pid, desc, err, ctx.Err(), stdOut, stdErr)
}

return stdOut.String(), stdErr.String(), nil
return stdOut.String(), stdErr.String(), err
}

// Kill and remove a process from list.
Expand Down
25 changes: 25 additions & 0 deletions modules/process/manager_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package process
import (
"os/exec"
"testing"
"time"

"github.com/stretchr/testify/assert"
)
Expand Down Expand Up @@ -31,3 +32,27 @@ func TestManager_Remove(t *testing.T) {
_, exists := pm.Processes[pid2]
assert.False(t, exists, "PID %d is in the list but shouldn't", pid2)
}

func TestExecTimeoutNever(t *testing.T) {

// TODO Investigate how to improve the time elapsed per round.
maxLoops := 10
for i := 1; i < maxLoops; i++ {
_, stderr, err := GetManager().ExecTimeout(5*time.Second, "ExecTimeout", "git", "--version")
if err != nil {
t.Fatalf("git --version: %v(%s)", err, stderr)
}
}
}

func TestExecTimeoutAlways(t *testing.T) {

maxLoops := 100
for i := 1; i < maxLoops; i++ {
_, stderr, err := GetManager().ExecTimeout(100*time.Microsecond, "ExecTimeout", "sleep", "5")
// TODO Simplify logging and errors to get precise error type. E.g. checking "if err != context.DeadlineExceeded".
if err == nil {
t.Fatalf("sleep 5 secs: %v(%s)", err, stderr)
}
}
}