Skip to content

Webhook not delivered for tag creation on mirror repo #13810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
2 of 6 tasks
dbowring opened this issue Dec 3, 2020 · 6 comments · Fixed by #13855
Closed
2 of 6 tasks

Webhook not delivered for tag creation on mirror repo #13810

dbowring opened this issue Dec 3, 2020 · 6 comments · Fixed by #13855
Labels
type/bug

Comments

@dbowring
Copy link

dbowring commented Dec 3, 2020

  • Gitea version (or commit ref): 1.13.0
  • Git version: 2.26.2
  • Operating system: linux (using gitea/gitea:1.13.0 from dockerhub)
  • Database (use [x]):
    • PostgreSQL
    • MySQL
    • MSSQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Yes (provide example URL)
    • No

Web hook: https://try.gitea.io/dbowring/gitea-tag-issue-test/settings/hooks/434
Not that an event was devlivered for the push, but not for the tag.

  • Log gist:
gitea 2020/12/03 08:03:18 ...n/webhook/webhook.go:697:NotifyCreateRef() [E] GetRefCommitID[v0.0.6[]: object does not exist [id: v0.0.6, rel_path: ]

Description

If you have a webhook on a mirror repository, events will not be produced when tags are created on the remote repository.
@lunny
Copy link
Member

lunny commented Dec 3, 2020

This should be fixed by #13556

@dbowring
Copy link
Author

dbowring commented Dec 3, 2020

@lunny That change fixed the event being skipped, but now (I think) it encounters an error while attempting to send it:

NotifyCreateRef() [E] GetRefCommitID[v0.0.6[]: object does not exist [id: v0.0.6, rel_path: ]

so the webhook ends up still not being sent.

@lunny
Copy link
Member

lunny commented Dec 3, 2020

Could you paste the complete error logs? I need to know which line it is in.

@manuelluis
Copy link
Contributor

We need to add the tag or branch prefix when parse the git output.
Also we need to notify the push, this is done in normal repositories

diff --git a/services/mirror/mirror.go b/services/mirror/mirror.go
index cfef55f2a..4bb0ef047 100644
--- a/services/mirror/mirror.go
+++ b/services/mirror/mirror.go
@@ -149,6 +149,11 @@ func parseRemoteUpdateOutput(output string) []*mirrorSyncResult {
 
                switch {
                case strings.HasPrefix(lines[i], " * "): // New reference
+                       if strings.HasPrefix(lines[i], " * [new tag]") {
+                               refName = git.TagPrefix + refName
+                       } else if strings.HasPrefix(lines[i], " * [new branch]") {
+                               refName = git.BranchPrefix + refName
+                       }
                        results = append(results, &mirrorSyncResult{
                                refName:     refName,
                                oldCommitID: gitShortEmptySha,
@@ -438,6 +443,21 @@ func syncMirror(repoID string) {
 
                // Create reference
                if result.oldCommitID == gitShortEmptySha {
+                       if tp == git.TagPrefix {
+                               tp = "tag"
+                       } else if tp == git.BranchPrefix {
+                               tp = "branch"
+                       }
+                       commitID, err := gitRepo.GetRefCommitID(result.refName)
+                       if err != nil {
+                               log.Error("gitRepo.GetRefCommitID [repo_id: %s, ref_name: %s]: %v", m.RepoID, result.refName,  err)
+                               continue
+                       }
+                       notification.NotifySyncPushCommits(m.Repo.MustOwner(), m.Repo, &repo_module.PushUpdateOptions{
+                               RefFullName: result.refName,
+                               OldCommitID: git.EmptySHA,
+                               NewCommitID: commitID,
+                       }, repo_module.NewPushCommits())
                        notification.NotifySyncCreateRef(m.Repo.MustOwner(), m.Repo, tp, result.refName)
                        continue
                }

@dbowring
Copy link
Author

dbowring commented Dec 4, 2020 •
edited
Loading

@lunny here's a log of pressing the "Synchronize Now" button after pushing a new commit (c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d) and tag (v0.0.2) to the remote repo, using this docker-compose.yml file (with - RUN_MODE=dev added to the server environment.

gitea     | 2020/12/04 11:53:27 Completed GET /user/events 200 OK in 1m47.3588391s
gitea     | 2020/12/04 11:53:27 Started POST /dbowring/gitea-tag-issue-test/settings for 172.29.0.1
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-list --count 7565fc855c0bbf17752efe64a3cbd8c9336aec92
gitea     | [git-module] stdout:
gitea     | 2
gitea     |
gitea     | 2020/12/04 11:53:28 Completed POST /dbowring/gitea-tag-issue-test/settings 302 Found in 94.7972ms
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= remote update --prune
gitea     | 2020/12/04 11:53:28 Started GET /dbowring/gitea-tag-issue-test/settings for 172.29.0.1
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= config --get commit.gpgsign
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= remote get-url origin
gitea     | [git-module] stdout:
gitea     | https://github.com/dbowring/gitea-tag-issue-test.git
gitea     |
gitea     | 2020/12/04 11:53:28 Completed GET /dbowring/gitea-tag-issue-test/settings 200 OK in 152.4428ms
gitea     | 2020/12/04 11:53:28 Started GET /user/avatar/dbowring/-1 for 172.29.0.1
gitea     | 2020/12/04 11:53:28 Completed GET /user/avatar/dbowring/-1 302 Found in 10.9324ms
gitea     | 2020/12/04 11:53:28 Started GET /user/events for 172.29.0.1
gitea     | 2020/12/04 11:53:28 Started GET /serviceworker.js for 172.29.0.1
gitea     | 2020/12/04 11:53:28 Completed GET /serviceworker.js 304 Not Modified in 2.471ms
gitea     | 2020/12/04 11:53:28 ...les/public/public.go:156:handle() [I] [Macaron] [Static] Serving /serviceworker.js
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-list -n 1 refs/tags/v0.0.1
gitea     | [git-module] stdout:
gitea     | 7565fc855c0bbf17752efe64a3cbd8c9336aec92
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= show-ref --tags -- v0.0.2
gitea     | [git-module] stdout:
gitea     | c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d refs/tags/v0.0.2
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= show-ref --tags -d
gitea     | [git-module] stdout:
gitea     | 7565fc855c0bbf17752efe64a3cbd8c9336aec92 refs/tags/v0.0.1
gitea     | c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d refs/tags/v0.0.2
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= cat-file -t c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     | [git-module] stdout:
gitea     | commit
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-list -n 1 refs/tags/v0.0.2
gitea     | [git-module] stdout:
gitea     | c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= show-ref --tags -- v0.0.2
gitea     | [git-module] stdout:
gitea     | c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d refs/tags/v0.0.2
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-list --count c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     | [git-module] stdout:
gitea     | 3
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-parse 7565fc8
gitea     | [git-module] stdout:
gitea     | 7565fc855c0bbf17752efe64a3cbd8c9336aec92
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-parse c4e82ce
gitea     | [git-module] stdout:
gitea     | c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= rev-list 7565fc855c0bbf17752efe64a3cbd8c9336aec92...c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     | [git-module] stdout:
gitea     | c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     |
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= show --name-status --pretty=format:'' c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= show --name-status --pretty=format:'' c4e82ce5ccd645f89a6f9c4cbf6926988409ca4d
gitea     | 2020/12/04 11:53:31 ...n/webhook/webhook.go:697:NotifyCreateRef() [E] GetRefCommitID[v0.0.2]: object does not exist [id: v0.0.2, rel_path: ]
gitea     | 2020/12/04 11:53:31 ...n/webhook/webhook.go:697:NotifyCreateRef() [E] GetRefCommitID[v0.0.2]: object does not exist [id: v0.0.2, rel_path: ]
gitea     | [git-module] /data/git/repositories/dbowring/gitea-tag-issue-test.git: /usr/bin/git -c credential.helper= -c protocol.version=2 -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= for-each-ref --sort=-committerdate refs/heads/ --count 1 --format=%(committerdate)
gitea     | [git-module] stdout:
gitea     | Fri Dec 4 11:52:39 2020 +0900
gitea     |

@lunny
Copy link
Member

lunny commented Dec 4, 2020

We need to add the tag or branch prefix when parse the git output.
Also we need to notify the push, this is done in normal repositories

diff --git a/services/mirror/mirror.go b/services/mirror/mirror.go
index cfef55f2a..4bb0ef047 100644
--- a/services/mirror/mirror.go
+++ b/services/mirror/mirror.go
@@ -149,6 +149,11 @@ func parseRemoteUpdateOutput(output string) []*mirrorSyncResult {
 
                switch {
                case strings.HasPrefix(lines[i], " * "): // New reference
+                       if strings.HasPrefix(lines[i], " * [new tag]") {
+                               refName = git.TagPrefix + refName
+                       } else if strings.HasPrefix(lines[i], " * [new branch]") {
+                               refName = git.BranchPrefix + refName
+                       }
                        results = append(results, &mirrorSyncResult{
                                refName:     refName,
                                oldCommitID: gitShortEmptySha,
@@ -438,6 +443,21 @@ func syncMirror(repoID string) {
 
                // Create reference
                if result.oldCommitID == gitShortEmptySha {
+                       if tp == git.TagPrefix {
+                               tp = "tag"
+                       } else if tp == git.BranchPrefix {
+                               tp = "branch"
+                       }
+                       commitID, err := gitRepo.GetRefCommitID(result.refName)
+                       if err != nil {
+                               log.Error("gitRepo.GetRefCommitID [repo_id: %s, ref_name: %s]: %v", m.RepoID, result.refName,  err)
+                               continue
+                       }
+                       notification.NotifySyncPushCommits(m.Repo.MustOwner(), m.Repo, &repo_module.PushUpdateOptions{
+                               RefFullName: result.refName,
+                               OldCommitID: git.EmptySHA,
+                               NewCommitID: commitID,
+                       }, repo_module.NewPushCommits())
                        notification.NotifySyncCreateRef(m.Repo.MustOwner(), m.Repo, tp, result.refName)
                        continue
                }

Could you send a PR to fix this?

@6543 6543 added the type/bug label Dec 4, 2020
@manuelluis manuelluis mentioned this issue Dec 4, 2020
Merged
@manuelluis manuelluis mentioned this issue Dec 5, 2020
Merged
@go-gitea go-gitea locked and limited conversation to collaborators Jan 18, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
type/bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix branch/tag notifications in mirror sync manuelluis/gitea
4 participants
@lunny @manuelluis @dbowring @6543