Skip to content

Specify network namespace for firecracker process #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Specify network namespace for firecracker process #120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eecc586
specify network namespace for firecracker process
aaithal Mar 4, 2019
21948c4
start firecracker VM in non-default netns, when specified
aaithal Mar 4, 2019
71c9406
add netns flag to taskworkflow example.
aaithal Mar 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 77 additions & 19 deletions examples/taskworkflow.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,26 +33,41 @@ import (
)

const (
kernelArgsFormat = "console=ttyS0 noapic reboot=k panic=1 pci=off nomodules rw ip=%s::%s:%s:::off::::"
macAddress = "AA:FC:00:00:00:01"
hostDevName = "tap0"
kernelArgsFormat = "console=ttyS0 noapic reboot=k panic=1 pci=off nomodules rw ip=%s::%s:%s:::off::::"
defaultMacAddress = "AA:FC:00:00:00:01"
defaultHostDevName = "tap0"
)

func main() {
var ip = flag.String("ip", "", "ip address assigned to the container. Example: -ip 172.16.0.1")
var gateway = flag.String("gw", "", "gateway ip address. Example: -gw 172.16.0.1")
var netMask = flag.String("mask", "", "subnet gatway mask. Example: -mask 255.255.255.0")
var netNS = flag.String("netns", "", "firecracker VM network namespace. Example: -netNS testing")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the example should read "-netns testing".

var hostDevName = flag.String("host_device", defaultHostDevName,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd prefer to use - inside argument options instead of _, so this would be host-device.

"the host device name for the network interface, required when specifying 'netns'. Example: -host_device tap0")
var macAddress = flag.String("mac", defaultMacAddress,
"the mac address for the network interface, required when specifying 'netns'. Example: -mac AA:FC:00:00:00:01")
var kernelNetworkArgs = flag.Bool("kernel_nw_args", false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for hyphens rather than underscores.

I also would prefer something other than "nw" for network. Maybe kernel-ip-arg or kernel-net-arg?

"specifies if network params for the VMs should be included with kernel args. Example: -kernel_nw_args true")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this always true when netns is specified? Does it default to true when netns is specified? What about when ip is specified?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this always true when netns is specified? Does it default to true when netns is specified? What about when ip is specified?

It's true only when someone sets it. Depending on how your VM is configured you or may not need it. My rootfs has a dhcp client. Hence, I don't need to set this at all. The taskworkflow.md file demonstrates use-cases with and without it. Please let me know if it's still unclear and I'll try to document more.

log.SetFlags(log.Ldate | log.Ltime | log.Lmicroseconds)
flag.Parse()
if *ip != "" && (*gateway == "" || *netMask == "") {
log.Fatal("Incorrect usage. 'gw' and 'mask' need to be specified when 'ip' is specified")
if *kernelNetworkArgs && (*gateway == "" || *netMask == "" || *ip == "") {
log.Fatal("Incorrect usage. 'ip', 'gw' and 'mask' need to be specified when 'kernel_nw_args' is set")
}
if err := taskWorkflow(*ip, *gateway, *netMask); err != nil {
if err := taskWorkflow(*ip, *kernelNetworkArgs, *gateway, *netMask, *macAddress, *hostDevName, *netNS); err != nil {
log.Fatal(err)
}
}

func taskWorkflow(containerIP string, gateway string, netMask string) error {
func taskWorkflow(
containerIP string,
kernelNetworkArgs bool,
gateway string,
netMask string,
macAddress string,
hostDevName string,
netNS string,
) error {
log.Println("Creating containerd client")
client, err := containerd.New("/run/containerd/containerd.sock")
if err != nil {
Expand Down Expand Up @@ -93,21 +108,23 @@ func taskWorkflow(containerIP string, gateway string, netMask string) error {
task, err := container.NewTask(ctx,
cio.NewCreator(cio.WithStdio),
func(ctx context.Context, _ *containerd.Client, ti *containerd.TaskInfo) error {
if containerIP == "" {
if containerIP == "" && netNS == "" {
// No params to configure, return.
return nil
}
// An IP address for the container has been provided. Configure
// the VM opts accordingly.
firecrackerConfig := &proto.FirecrackerConfig{
NetworkInterfaces: []*proto.FirecrackerNetworkInterface{
{
MacAddress: macAddress,
HostDevName: hostDevName,
},
},
KernelArgs: fmt.Sprintf(kernelArgsFormat, containerIP, gateway, netMask),
// An IP address or the network namespace for the container has
// been provided. Configure VM opts accordingly.
builder := newFirecrackerConfigBuilder()
if containerIP != "" {
builder.setNetworkConfig(macAddress, hostDevName)
}
ti.Options = firecrackerConfig
if kernelNetworkArgs {
builder.setKernelNetworkArgs(containerIP, gateway, netMask)
}
if netNS != "" {
builder.setNetNS(netNS)
}
ti.Options = builder.build()
return nil
})
if err != nil {
Expand Down Expand Up @@ -151,6 +168,47 @@ func taskWorkflow(containerIP string, gateway string, netMask string) error {
return nil
}

type firecrackerConfigBuilder struct {
vmConfig *proto.FirecrackerConfig
}

func newFirecrackerConfigBuilder() *firecrackerConfigBuilder {
return &firecrackerConfigBuilder{
vmConfig: &proto.FirecrackerConfig{},
}
}

func (builder *firecrackerConfigBuilder) build() *proto.FirecrackerConfig {
return builder.vmConfig
}

// setNetworkConfig sets the fields in the protobuf message required to
// configure the VM with the container IP, gateway and netmask sepcified.
func (builder *firecrackerConfigBuilder) setNetworkConfig(
macAddress string,
hostDevName string,
) {
builder.vmConfig.NetworkInterfaces = []*proto.FirecrackerNetworkInterface{
{
MacAddress: macAddress,
HostDevName: hostDevName,
},
}
}

func (builder *firecrackerConfigBuilder) setKernelNetworkArgs(
containerIP string,
gateway string,
netMask string,
) {
builder.vmConfig.KernelArgs = fmt.Sprintf(kernelArgsFormat, containerIP, gateway, netMask)
}

// setNetNS sets the network namespace field in the protobuf message.
func (builder *firecrackerConfigBuilder) setNetNS(netNS string) {
builder.vmConfig.FirecrackerNetworkNamespace = netNS
}

func getResponse(containerIP string) {
response, err := http.Get("http://" + containerIP)
if err != nil {
Expand Down
28 changes: 26 additions & 2 deletions examples/taskworkflow.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,29 @@ After checking out the `firecracker-containerd` repo, you can build this
example using the `make examples` command.

### Running

#### Basic workflow (no networking)
For a basic workflow, without any networking setup, just run the executable
built with the command in the previous section as super user:
```bash
$ sudo /path/to/firecracker-containerd/examples/taskworkflow
```

#### Workflow to create container, with networking
For a workflow with networking setup for the container, create a tap device
for the VM by following [these instructions](https://github.com/firecracker-microvm/firecracker/blob/master/docs/network-setup.md).

This creates a tap device named `tap0`, in the local `172.16.0.1/24` subnet.
Since the example does not rely on a DHCP client running within the VM to
initialize the network interface, `gw` and `mask` flags should be used to
specify the gateway and subnet mask values.
specify the gateway and subnet mask values, along with the `kernel_nw_args`
flag.

The following example sets:
* The IP address to `172.16.0.2`
* The gateway IP address to `172.16.0.1`
* The subnet mask to `255.255.255.0` (`/24`)
* Kernel args to specify all 3 of these

** NOTE: This example will not work if you're running more than 1 container
on a host at the same time **
Expand All @@ -34,7 +39,8 @@ Now, run the example by passing the `-ip` argument:
```bash
$ sudo /path/to/firecracker-containerd/examples/taskworkflow -ip 172.16.0.2 \
-gw 172.16.0.1 \
-mask 255.255.255.0
-mask 255.255.255.0 \
-kernel_nw_args
```

You should see output similar to this:
Expand Down Expand Up @@ -75,3 +81,21 @@ Commercial support is available at
]
172.16.0.1 - - [11/Feb/2019:21:02:54 +0000] "GET / HTTP/1.1" 200 612 "-" "Go-http-client/1.1" "-"
```

#### Workflow to create container, with firecracker process in a netns
For a workflow with networking setup for a container, where network parameters
are being dervied from a network namespace, the `-netns` argument can be
used.

```bash
sudo /path/to/firecracker-containerd/examples/taskworkflow -ip 10.0.5.33 \
-host_device tap301 \
-mac aa:bb:cc:dd:ee:ff \
-netns vlan301
```

The above example command assumes that a vlan device with vlan tag `301` has
been setup with the `tap301` tap device in the `vlan301` network namespace.
Since `-kernel_nw_args` has not been set, it's also assumed that the rootfs
image for the VM has a DHCP client that runs on boot. The ip address for the
vlan device is assumed to be `10.0.5.33` as well.
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ require (
github.com/containerd/go-runc v0.0.0-20180907222934-5a6d9f37cfa3 // indirect
github.com/containerd/ttrpc v0.0.0-20181001154009-f51df4475b76
github.com/containerd/typeurl v0.0.0-20181015155603-461401dc8f19
github.com/containernetworking/plugins v0.7.4
github.com/coreos/go-systemd v0.0.0-20181031085051-9002847aa142 // indirect
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.13.1 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ github.com/containerd/ttrpc v0.0.0-20181001154009-f51df4475b76 h1:vUPO9S35+FvukX
github.com/containerd/ttrpc v0.0.0-20181001154009-f51df4475b76/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o=
github.com/containerd/typeurl v0.0.0-20181015155603-461401dc8f19 h1:gzdItdct+4eLnZxiZi1YcIXx3uo5QWa/xXKnsldEqY8=
github.com/containerd/typeurl v0.0.0-20181015155603-461401dc8f19/go.mod h1:Cm3kwCdlkCfMSHURc+r6fwoGH6/F1hH3S4sg0rLFWPc=
github.com/containernetworking/plugins v0.7.4 h1:ugkuXfg1Pdzm54U5DGMzreYIkZPSCmSq4rm5TIXVICA=
github.com/containernetworking/plugins v0.7.4/go.mod h1:dagHaAhNjXjT9QYOklkKJDGaQPTg4pf//FrUcJeb7FU=
github.com/coreos/go-systemd v0.0.0-20181031085051-9002847aa142 h1:3jFq2xL4ZajGK4aZY8jz+DAF0FHjI51BXjjSwCzS1Dk=
github.com/coreos/go-systemd v0.0.0-20181031085051-9002847aa142/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
Expand Down
Loading