I’ve had the pleasure of working with several other gophers the last few months on a prototype dependency management tool named dep.
dep is part of the project started last year and is organized by Peter Bourgon. I was asked to join the team working on this project due to my involvement in a different tool, godep, the OG dependency management tool for Go, (inherited from Keith Rarick) and my work at Heroku serving our customers who use Go.
The other members of the team, aside from myself and Peter, are Jessie Frazelle, Andrew Gerrand and Sam Boyer. Andrew is part of the Go team @ Google. Jessie works at Google and is involved in large Go projects such as Docker and Kubernetes. Sam wrote and maintains gps, the constraint solver powering dep.
The team has published a bunch of info about our progress over the course of our work. To date, various other tool authors and concerned parties have also been involved in different ways.
NOTE: dep is pre-alpha, so anything below may or may not be true moments after this blog post is published. However, I will try to keep it up-to-date as development continues.
In the beginning…
Let’s take dep for a drive and pretend we’re writing a web application using github.com/gorilla/mux. Here is some pretend code to get us started:
*$GOPATH/src/github.com/freeformz/depExample/main.go*
The first thing to do after writing some code, or when using dep for the first time on an existing project is to run dep init.
dep init adds the current version of any of the project’s dependencies found in $GOPATH to the manifest.json file. Because I have github.com/gorilla/mux in my $GOPATH, the manifest.json file includes it. I recently ran go get -u github.com/gorilla/mux, so the version of github.com/gorilla/mux in my $GOPATH is at master. If the version in my $GOPATH matched a Semver compatible tag (ex: v1.2.3), that tag’s name would have been used instead.
dep works across architectures and go versions. Using github.com/gorilla/mux with older versions of Go (< 1.7.0) pulls in the github.com/gorilla/context package. When I last ran go get -u github.com/gorilla/mux I was running Go 1.7.5 so the github.com/gorilla/context package is NOT in my $GOPATH. Because this could be a dependency necessary to compile the project, it is included in the lock.json file. In situations like these, if a dependent project has a semver compatible release tag, dep chooses the latest release. In this case that is v1.1 of github.com/gorilla/context.
Because github.com/gorilla/mux does not include a manifest.json file dep don’t know if github.com/gorilla/mux currently works with github.com/gorilla/context @ v1.1. Barring an override (see below), dep honors the constraints found in dependencies’ manifest.json files.
dep init includes all dependencies, analyzed recursively and the exact versions being used in the lock.json file.
For the example app, this creates the following two files:
lock.json and manifest.json
Ensuring the project can build
After dep init is run, dep ensure should be run to populate the vendor/ directory with a copy of packages required to build your project. This ensures that any of your project’s dependencies are included in the lock file and vendor directory. Any time you want to ensure that you have all of your dependencies recorded, run dep ensure.
Adding another dependency
You don’t have to do anything up front to add another dependency, just start using it in your code. When it’s time to check in your work though you need to run dep ensure to update the lock.json file and vendor/. This will lock the project to the latest released version of each dependency.
In the sample app I added a sub package math that can be used to add, subtract or retrieve a named value and a HTTPHandler that can be used in main.go. After writing that code a dep ensure run produced these changes to the lock.json file and the vendor/ directory.
But what if the latest doesn’t work for me?
If you need to specify a version, you can use the alternate form of the ensure command like so: dep ensure github.com/gorilla/mux@^1.3.0. That command modifies the manifest.json file, constraining dep to use ^v1.3.0 of github.com/gorilla/mux, resolves the dependency tree and updates the dependencies in vendor/ to reflect any differences resulting from the change. On the example application, that looks like this because dep has already chosen the latest available version, 1.3.0. Future updates (dep ensure -update) however will no longer track the master branch and instead use semver tagged releases ≥ 1.3.0 and < 2.0.0.
dep mostly uses Rust’s cargo operators for selecting versions of dependencies. These include ^, ~ and =. To have a more restrictive, forward compatible match than ^ use ~. For example dep ensure github.com/com/gorilla/mux@~1.2.0 will match any version ≥ 1.2.0 and < 1.3.0. To lock to a specific version use the = prefix (e.x. dep ensure github.com/com/gorilla/mux@=1.2.0). In the future it’s planned that dep will default to ^ when no prefix is specified.
Keeping current
To keep a project’s dependencies up to date use dep ensure -update, which updates all dependencies to the latest versions allowed by the constraints in manifest.json, ignoring the contents of lock.json. New versions are written to vendor/ and the appropriate meta data updated in lock.json.
In the future it should be possible to dep ensure -update a single dependency.
Status
When a dependency is missing, dep status tells you which project and which packages contained inside are missing. For instance, here is what dep status shows after this commit to the example app:
When the project’s lock.json is up-to-date, the dep status command shows you a list of all dependencies, as projects, and for each:
- the constraint(s) applied
- the selected version
- the selected revision
- the latest version or revision available
- the number of packages used
After adding bolt to the sample project and running dep ensure to update lock.json and vendor/, dep status looks like this:
These two modes may be combined in the future.
Removal
dep remove <dep> removes the dependency from manifest.json, lock.json and vendor/, once it’s no longer used. Using the example app this commit removes the usage of github.com/gorilla/mux from the application and the next commit is the result of dep remove github.com/gorilla/mux. Because github.com/gorilla/mux is removed, github.com/gorilla/context is no longer needed and is also removed. If a dependency is still being used when dep remove is run, the command will fail. Removal can be forced with the -force flag, which results in the constraint being removed from manifest.json. However, since the dependency is still in use it’s still listed in lock.json and copied into vendor/.
In the future….
We’re still in an experimental phase with dep, there are many issues open and much work to do still. I hope this post clearly showed you some examples of how the tool is expected to be used. This usage may change over time via community feedback and pull requests. If you have some spare cycles and are interested in working on some Go tooling please drop by #vendor on gopher slack and pick up an issue or two to work on.
Thanks for reading!