Build and Test¶
Most binaries and packages are “pure” Go and can be directly go-installed and go-tested:
$ export GO111MODULE=on
$ go install go.gazette.dev/core/cmd/gazette
$ go install go.gazette.dev/core/cmd/gazctl
$ go test go.gazette.dev/core/broker/...
$ go test go.gazette.dev/core/consumer
Certain packages used by consumer applications, like go.gazette.dev/core/consumer/store-rocksdb
,
require CGO to build and also require appropriate development libraries for RocksDB.
Standard Linux packages are insufficient, as run-time type information must be enabled
in the RocksDB build (and it’s turned off in the standard Debian package, for example).
Continuous Integration¶
Gazette uses a Make-based build system which pulls down and stages
development dependencies into a .build
sub-directory of the repository root.
The Make build system offers fully hermetic builds using a Docker-in-Docker builder. Run CI tests in a hermetic build environment:
$ make as-ci target=go-test-ci
Continuous integration builds of Gazette run tests 15 times, with race detection enabled.
Package release Docker images for the Gazette broker and examples,
as gazette/broker:latest
and gazette/examples:latest
.
$ make as-ci target=ci-release-gazette-broker
$ make as-ci target=ci-release-gazette-examples
Deploy Gazette’s continuous soak test to a Kubernetes cluster (which can be
Docker for Desktop or Minikube). Soak tests run with latest
images.
$ kubectl apply -k ./kustomize/test/deploy-stream-sum-with-crash-tests/
Push images to a registry. If REGISTRY
is not specified, it defaults to localhost:32000
(which is used by MicroK8s):
$ make push-to-registry REGISTRY=my.registry.com
The kustomize
directory also has a
helper manifest for using
a local registry (eg, for development builds).
Dependencies¶
It’s recommended to use the docker-based build targets described above in most situations, since the
docker image will have all the required dependencies. If you want to execute build targets directly
instead of using as-ci
, then the following dependencies are required:
Compression library development headers (“-dev” packages):
libbz2
libsnappy
liblz4
libzstd
libbz2
Protobuf compiler:
libprotobuf
(development headers)protobuf-compiler
Etcd: The deb/rpm packages provided in many repositories are likely too old to work with Gazette. Gazette requires version 3.3 or later. Version 3.4.x is recommended, since that is used in Gazette CI.
Sqlite
libsqlite3
(development headers)- It’s also probably useful to have the sqlite3 CLI for debugging
RocksDB: On linux systems, this will be downloaded and built automatically. You’ll need to have a few things in order for this to work. Most systems will already have this stuff, but it’s listed here anyway just for the sake of being thorough
- A C compiler toolchain (on debian-based distros, the
build-essential
package will have you covered) curl
ca-certificates
(so that curl can validate the certificate of the rocksdb download server)tar
- A C compiler toolchain (on debian-based distros, the
Other Build Targets¶
If you execute these targets directly, then you’ll need to have all of the above dependencies installed.
$ make go-install
$ make go-test-fast
$ make go-test-ci
Building the Docs¶
To build these docs locally, you’ll need a few more dependencies. To start with, you’ll need
python
and pip
. Note that on some systems, these may be called python3
and pip3
.
Next you’ll need to install the following python packages using pip install --user <package>
.
sphinx
sphinxcontrib-programoutput
sphinx_rtd_theme
Once you have all those installed, you can change directory into docs/
and run make html
.
This will write the output to docs/_build
, and then you can open any of the html files in your
browser.