Skip to content

Raise interrupt for used descriptors on tx queue and use edged triggered epoll. #1444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 4, 2019

Conversation

sandreim
Copy link
Contributor

@sandreim sandreim commented Dec 3, 2019

Reason for This PR

Fixes #1439, #1436 .

Description of Changes

  • Raise interrupt after TX queue is processed.
  • Use edged triggered epoll for the tap fd.

This change will degrade performance as it increases the interrupt rate. It will also improve emulation thread cpu usage.

I've run some benchmarks on a m5d.metal host running a microvm with 2vCPUs and 2GB of RAM.

Without this PR:

  • Guest RX, throughput: 13194.73 Mbps, retransmits: 106
  • Guest TX, throughput: 18468.89 Mbps, retransmits: 0

With this PR:

  • Guest RX, throughput: 12484.86 Mbps, retransmits: 102 -6%
  • Guest TX, throughput: 17103.22 Mbps, retransmits: 0 -8%

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license.

PR Checklist

[Author TODO: Meet these criteria. Where there are two options, keep one.]
[Reviewer TODO: Verify that these criteria are met. Request changes if not]

  • All commits in this PR are signed (git commit -s).
  • This PR is linked to an issue.
  • The description of changes is clear and encompassing.
  • The required doc changes are included in this PR.
  • Code-level documentation for touched code is included in this PR.
  • No API changes are included in this PR.
  • The changes in this PR have user impact and have been added to the CHANGELOG.md file.
  • No new unsafe code has been added.

@sandreim sandreim self-assigned this Dec 3, 2019
@sandreim sandreim added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Dec 3, 2019
@sandreim sandreim changed the title Net fixes Raise interrupt for used descriptors on tx queue and use edged triggered epoll. Dec 3, 2019
serban300
serban300 previously approved these changes Dec 3, 2019
acatangiu
acatangiu previously approved these changes Dec 3, 2019
serban300
serban300 previously approved these changes Dec 4, 2019
iulianbarbu
iulianbarbu previously approved these changes Dec 4, 2019
Add a unit test for this regression: firecracker-microvm#1436.

Signed-off-by: Andrei Sandu <[email protected]>
Remove register_tap_rx_listener and unregister_tap_rx_listener,
and set EPOLLET flag when registering the tap fd. Edged epoll
should work fine with the current code as it already tracks
unconsumed input from the tap device via *deferred_frame*.
When process_rx stops reading frames before EAGAIN it will
set deferred_frame flag. The process_rx function will resume
processing frames via RX_RATE_LIMITER_EVENT and clear
that flag.

Signed-off-by: Andrei Sandu <[email protected]>
@sandreim sandreim merged commit 7250788 into firecracker-microvm:master Dec 4, 2019
ioanachirca added a commit to ioanachirca/firecracker that referenced this pull request Aug 3, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 3, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
ioanachirca added a commit to ioanachirca/firecracker that referenced this pull request Aug 3, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit that referenced this pull request Aug 4, 2020
This is a regression test for
#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit that referenced this pull request Aug 4, 2020
This is a regression test for
#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 4, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 4, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 4, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
ioanachirca added a commit to ioanachirca/firecracker that referenced this pull request Aug 4, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
@ioanachirca ioanachirca mentioned this pull request Aug 4, 2020
9 tasks
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 4, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 4, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
ioanachirca added a commit that referenced this pull request Aug 4, 2020
This is a regression test for
#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 5, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 5, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit to dianpopa/firecracker that referenced this pull request Aug 5, 2020
This is a regression test for
firecracker-microvm#1444

Signed-off-by: Ioana Chirca <[email protected]>
dianpopa pushed a commit that referenced this pull request Aug 5, 2020
This is a regression test for
#1444

Signed-off-by: Ioana Chirca <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Awaiting review Indicates that a pull request is ready to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ratelimiter: Very high CPU usage when ratelimiter is throttling guest RX.
4 participants