Skip to content

[usage] reduce possibility for races in ResetUsage #15006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 28, 2022

Conversation

svenefftinge
Copy link
Member

@svenefftinge svenefftinge commented Nov 28, 2022

Description

Fetch the cost center from db immediately before deciding to resetting the usage. So that the time window in which a parallel resetting for the same cost center could happen is minimized.

We saw eight cases in Nopvember where thsi happened. But teh likely hood was very high due to:

  • no resetting scheduler was active (so every cost center was outdated at some point)
  • we fetch the cost center first and do quite some computation before inserting the update.

Related Issue(s)

Fixes #14973

How to test

Release Notes

NONE

Documentation

Werft options:

  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-large-vm
  • /werft with-integration-tests=all
    Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh

@svenefftinge svenefftinge requested a review from a team November 28, 2022 13:18
@github-actions github-actions bot added the team: webapp Issue belongs to the WebApp team label Nov 28, 2022
Copy link
Member

@easyCZ easyCZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

/hold for improved log which can be used to correlate any future duplicates with logs

Comment on lines 411 to 418
cc := &db.CostCenter{
ID: db.NewUserAttributionID(uuid.New().String()),
CreationTime: db.NewVarCharTime(time.Now()),
SpendingLimit: 500,
BillingStrategy: db.CostCenter_Stripe,
})
}

err := conn.Save(cc).Error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use dbtest.CreateCostCenter.


now := time.Now().UTC()

logger.Info("Running `ResetUsage`.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.Info("Running `ResetUsage`.")
logger.WithField("attribution_id", id).WithField("cost_center", cc).Infof("Resetting usage for Attribution ID %s", id)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Added the cost center. the attributiond id was already included and I like to keep the message static, as it's a bit easier to filter on that way.

@svenefftinge
Copy link
Member Author

/unhold

@roboquat roboquat merged commit 7252b10 into main Nov 28, 2022
@roboquat roboquat deleted the sefftinge/ubp-duplicate-ledger-14973 branch November 28, 2022 14:02
@roboquat roboquat added deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: webapp Meta team change is running in production deployed Change is completely running in production release-note-none size/M team: webapp Issue belongs to the WebApp team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[UBP] Duplicate ledger reset entries
3 participants