Skip to content

Commit 66ea25a

Browse files
akutzakutz
committed
GoCSI
This patch provides a Go-based CSI client and server capable of supporting additional storage platforms at runtime via Go plug-ins.
1 parent a4b578a commit 66ea25a

File tree

466 files changed

+150680
-2
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

466 files changed

+150680
-2
lines changed

‎.travis.yml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
go_import_path: github.com/container-storage-interface/examples
2+
3+
language: go
4+
5+
go:
6+
- 1.8.x
7+
8+
env:
9+
global:
10+
- CSI_ENDPOINT=tcp://127.0.0.1:4210
11+
12+
before_install:
13+
- curl -LO https://github.com/google/protobuf/releases/download/v3.3.0/protoc-3.3.0-linux-x86_64.zip
14+
- unzip protoc-3.3.0-linux-x86_64.zip
15+
- chmod +x bin/protoc
16+
- export PATH="$(pwd)/bin:$PATH"
17+
18+
install:
19+
- go get -u github.com/golang/protobuf/proto
20+
- go get -u github.com/golang/protobuf/protoc-gen-go
21+
- go get -u google.golang.org/grpc
22+
- make -C gocsi goget
23+
24+
script:
25+
- make -C gocsi build
26+
- make -C gocsi test
27+
28+
after_failure:
29+
- if [ -e "gocsi/csd.log" ]; then cat gocsi/csd.log; fi

‎README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,6 @@
1-
# examples
2-
Reference plugin implementations
1+
# Container Storage Interface (CSI) Examples [![build status](https://travis-ci.org/container-storage-interface/examples.svg?branch=master)](https://travis-ci.org/container-storage-interface/examples)
2+
This project contains examples CSI examples.
3+
4+
| Name | Description |
5+
|------|-------------|
6+
| [gocsi](./gocsi) | A Go-based CSI client and server capable of supporting additional storage platforms at runtime via Go plug-ins. |

‎gocsi/.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.build/
2+
.DS_Store
3+
csd.log
4+
mock.so
5+
.vagrant/

‎gocsi/Makefile

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
SHELL := $(shell env which bash)
2+
3+
all: build
4+
5+
# load the go bits
6+
include go.mk
7+
8+
# configure and load the csi protobuf generator
9+
CSI_PROTO_DIR := csi
10+
include csi.mk
11+
12+
# the name of the program being built
13+
PROG := $(PKG_DIR_GO)/$(IMPORT_PATH).a
14+
15+
# the target for building the gocsi library
16+
$(PROG):$(CSI_GOSRC) \
17+
$(filter-out %_test.go,$(wildcard *.go)) \
18+
| $(GOGET) $(PKG_DIR_GO)
19+
go install -pkgdir $(abspath $(PKG_DIR_GO))
20+
@echo $@
21+
22+
# a list of sub-projects to make
23+
SUB_PROJS := $(subst /,,$(dir $(wildcard */Makefile)))
24+
25+
# prints a list of the projects to make
26+
projs:
27+
@echo . $(SUB_PROJS)
28+
29+
build: $(PROG)
30+
$(foreach d,$(SUB_PROJS),$(MAKE) -C $d $@;)
31+
32+
clean:
33+
go clean -i
34+
$(foreach d,$(SUB_PROJS),$(MAKE) -C $d $@;)
35+
36+
clobber: clean
37+
rm -fr $(CSI_PROTO_DIR) $(BUILD_DIR)
38+
$(foreach d,$(SUB_PROJS),$(MAKE) -C $d $@;)
39+
40+
goget: $(GOGET)
41+
$(foreach d,$(SUB_PROJS),$(MAKE) -C $d $@;)
42+
43+
test: build
44+
@rm -f csd.log
45+
CSI_PLUGINS=$$(pwd)/csp/moc/moc-csi-plugin.so csd/csd mock > csd.log 2>&1 &
46+
@until grep 'mock\.Serve' csd.log > /dev/null 2>&1; do sleep 0.1; done;
47+
csc/csc ls -version 0.1.0
48+
csc/csc new -version 0.1.0 \
49+
-o norootsquash,uid=500,gid=500 \
50+
-t ext4 -requiredBytes 107374182400 \
51+
-params color=purple,up=down \
52+
"My New Volume"
53+
csc/csc ls -version 0.1.0
54+
kill $$(ps aux | grep '[c]sd' | awk '{print $$2}')
55+
cat csd.log
56+
57+
benchmark: $(PROG)
58+
go test -benchmem -parallel 100 -bench .
59+
60+
.PHONY: projs build clean clobber goget vendor test benchmark

‎gocsi/README.md

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# gocsi
2+
This project provides a Go-based CSI client and server capable of
3+
supporting additional storage platforms at runtime via Go plug-ins.
4+
5+
| Name | Description |
6+
|------|-------------|
7+
| Container Storage Client ([`csc`](./csc)) | A command line interface (CLI) tool that provides analogues for all of the CSI RPCs. |
8+
| Container Storage Daemon ([`csd`](./csd)) | A CLI tool that serves the CSI services -- Controller, Identity, and Node -- on a TCP port or UNIX socket. The storage platform support for `csd` is provided via Go plug-ins, loadable at runtime. |
9+
| Container Storage Providers ([`csp`](./csp)) | A directory containing Go packages that can be built as stand-alone CLI programs capable of serving the CSI services via a TCP port or UNIX socket, similar to `csd`. Additionally, the `csp` packages can **also** be built as Go plug-ins that the `csd` program can load, extending its storage provider support dynamically, at runtime.
10+
11+
## Getting Started
12+
The example below illustrates how the above tools and Go plug-ins
13+
work together to provide a cohesive experience in line with the CSI
14+
specification.
15+
16+
Please note that the mock `csp` is used to fulfill the role of a
17+
storage platform. Also, because Go plug-ins are demonstrated this
18+
example must be executed on a Linux platform as Go plug-ins are not
19+
yet supported by other operating systems (OS).
20+
21+
```bash
22+
# get and install the sources
23+
$ go get github.com/container-storage-interface/examples/gocsi
24+
25+
# build the container storage client
26+
$ go install github.com/container-storage-interface/examples/gocsi/csc
27+
28+
# build the container storage daemon
29+
$ go install github.com/container-storage-interface/examples/gocsi/csd
30+
31+
# build the mock container storage provider
32+
$ go build -o mock.so -buildmode plugin github.com/container-storage-interface/examples/gocsi/csp/moc
33+
34+
# export the CSI endpoint
35+
$ export CSI_ENDPOINT=tcp://127.0.0.1:8080
36+
37+
# start the server (assuming $GOPATH/bin is in the PATH)
38+
$ CSI_PLUGINS=$(pwd)/mock.so csd mock > csd.log 2>&1 &
39+
[1] 19050
40+
41+
# use the client to ask for a list of volumes
42+
$ csc ls
43+
id=1 name=Mock Volume 1
44+
id=2 name=Mock Volume 2
45+
id=3 name=Mock Volume 3
46+
47+
# create a new volume
48+
$ csc new "My New Volume"
49+
id=4 name=My New Volume
50+
51+
# query the volume list again
52+
$ csc ls
53+
id=1 name=Mock Volume 1
54+
id=2 name=Mock Volume 2
55+
id=3 name=Mock Volume 3
56+
id=4 name=My New Volume
57+
58+
# kill the server
59+
kill -HUP $(ps aux | grep '[c]sd' | awk '{print $2}')
60+
61+
# view the server log
62+
$ cat csd.log
63+
2017/06/26 01:54:48 loaded plug-in: mock.so
64+
2017/06/26 01:54:48 registered endpoint: mock
65+
2017/06/26 01:54:48 mock.Serve
66+
2017/06/26 01:55:36 csd.ListVolumes
67+
2017/06/26 01:55:36 ...Volume.ID=1
68+
2017/06/26 01:55:36 ...Volume.ID=2
69+
2017/06/26 01:55:36 ...Volume.ID=3
70+
2017/06/26 01:55:47 csd.CreateVolume
71+
2017/06/26 01:55:47 CreateVolume.CapacityRange=<nil>
72+
2017/06/26 01:55:47 CreateVolume.Name=My New Volume
73+
2017/06/26 01:55:47 CreateVolume.Parameters=map[]
74+
2017/06/26 01:55:47 CreateVolume.VolumeCapabilities=[]
75+
2017/06/26 01:55:47 ...Volume.ID=4
76+
2017/06/26 01:56:04 csd.ListVolumes
77+
2017/06/26 01:56:04 ...Volume.ID=1
78+
2017/06/26 01:56:04 ...Volume.ID=2
79+
2017/06/26 01:56:04 ...Volume.ID=3
80+
2017/06/26 01:56:04 ...Volume.ID=4
81+
received signal: terminated: shutting down
82+
server stopped gracefully
83+
```
84+
85+
## Build Reference
86+
GoCSI can be built entirely using `go build` and `go install`. However,
87+
Make is also used in order to create a more deterministic and portable
88+
build process.
89+
90+
| Make Target | Description |
91+
|-------------|-------------|
92+
| `build` | Builds `csc`, `csd`, and all of the `csp`s. Please note that this target does not install anything to the `$GOPATH`. Instead each of the packages' own Makefiles ensure that all code is built in a directory named `.build` inside the package using the `go` tool's `-pkgdir` flag in conjunction with the environment variable `GOBIN`. This ensures builds that are easy to clean. |
93+
| `clean` | Removes the artifacts produced by the `build` target. |
94+
| `clobber` | Depends on `clean`. Removes all generated sources and vendored dependencies. |
95+
| `test` | Depends on `build`. Executes an end-to-end example using `csc`, `csd`, and the mock `csp`.
96+
| `bencmark` | Usses `go test` to illustrate the performance of the `PipeConn` used by `csd` to host the `csp`s as in-memory CSI endpoints. |
97+
| `goget` | Will scan the packages and execute `go get` for any missing dependencies. |
98+
99+
100+
## Frequently Asked Questions
101+
This section answers some of the common inquiries related to this project.
102+
103+
**What is the purpose of GoCSI?**
104+
105+
The purpose of GoCSI is not to provide a reference library or set
106+
of tools meant to be used by other, third-party or external consumers
107+
wishing to jumpstart CSI development. GoCSI is simply a series of small
108+
programs and packages that provide working examples and demonstrations
109+
of the CSI specification. If at some point the CSI working group wishes
110+
to build a set of reference implementations based on the contents of
111+
this project, then that is fine. If GoCSI stands forever as simply a
112+
series of examples, that's fine as well.
113+
114+
**Why vendor Proto and gRPC in the `csp`s?**
115+
116+
There are two answers to this question. The first is simply so that
117+
the examples included in GoCSI can be built/executed with standard
118+
Go patterns without the need for an external build system such as
119+
[Make](https://www.gnu.org/software/make/) or vendoring solution like
120+
[Glide](https://glide.sh/). The fact is that executing `make clobber`
121+
at the root of the GoCSI project will remove all of the vendored
122+
code as well as generated sources. Running `make` again will restore
123+
all of the removed code. It's simply by design to leave that code
124+
as part of the commit in order to make it as easy as possible to use
125+
the examples included in GoCSI.
126+
127+
The second answer is the result of weeks of research and trial and
128+
error. The short answer has to do with how Go manages packages, plug-ins,
129+
and the gRPC type registry. The long answer is summarized by
130+
[golang/go#20481](https://github.com/golang/go/issues/20481).
131+
132+
The purpose of the `csp`s is to demonstrate the triviality of creating a
133+
package that can be built both as a stand-alone CSI endpoint and a Go
134+
plug-in, capable of providing support for a new storage platform to
135+
a separate, completely unrelated host program. Therefore the `csp`s
136+
should have absolutely no relationship to any other GoCSI project,
137+
even `csd`, the program capable of loading the `csp`s as Go plug-ins.
138+
139+
Truthfully the matter is much more detailed, but it won't be repeated
140+
here. For that please see the aforementioned Golang issue. Suffice
141+
it to say, the only way to create Go plug-ins that are useful in
142+
production software is to 1) vendor everything inside the plug-in's
143+
package and 2) use only Go stdlib types and empty interfaces when
144+
communicating with the plug-in from the host program.
145+
146+
**Why generate Go sources from the CSI protobuf in multiple locations?**
147+
148+
The answer to this question is similar to the answer of the previous
149+
question. If the `csp`s referenced the `csi` package at the root of the
150+
GoCSI project then the `csd` program would not be able to load the Go
151+
plug-ins due to 1) duplicate gRPC type registrations and 2) a possibly
152+
different hash for the referenced package since the plug-ins are built
153+
separately and thus potentially from different sources.

0 commit comments

Comments
 (0)