Go Project Layout
Originally
ADR--0117-Go-Project-Layout (v1) · Source on Confluence ↗Go Project Layout
Context
Our current project layout has some limitations for cases where we want share functionality between projects.
Decision
Structure the Go code to support multiple use cases. With this structure we can deploy a standard GRPC service in the cloud and simultaneously support command line applications and importable libraries.
Project Layout
Our standard structure for a Go cloud service looks like:
go.mod
main.go
values.*
cmd
- root.go
- service.go
internal
service
- foo.go
- bar.go
other
- otherfoo.go
The key features of this structure are that there is a single main.go file at the root of the project, so this is a single application.
The cmd directory holds the code to bootstrap the service, but it doesn’t contain independent application code.
internal is a “magic” directory. Any code inside this directory is only accessible within the project. That means that if someone imported this project via go get they would only be able to access code defined in the non- internal directories.
To support both a GRPC service and a library, we’ll instead use the following structure:
go.mod
cmd
grpc
- main.go
- service.go
pkg
altimeter
- altimeter.go
internal
foo
- foo.go
In this project layout, there is no main.go in the root directory. Instead, any main.go files are pushed into the appropriate command under cmd. In the example above, there is only one command to run the GRPC server, but we could easily add additional operations under cmd to support a CLI tool in parallel.
The pkg directory will contain the library API designed to be imported by other projects. pkg is a standard name in Go projects for library code, but it doesn’t have any “magic” associated with it.
We’ll still use an internal directory so that we can control what code gets exported by the pkg directory. This way we can keep any utilities or implementation details separate from the library API. For instance, assume we have a function in internal/foo/foo.go named Convert. Convert calls some code in internal/bar/bar.go via importing the bar namespace and calling bar.BarFunc(). In our API, we want to expose Convert, but not BarFunc. We would accomplish this by adding a function in pkg/altimeter/altimeter.go named Convert that calls foo.Convert. Clients using the library would have access to altimeter.Convert but not foo.Convert.
Compilation and the Service Code
One issue to be concerned about is consumers of the library also carrying the wait of the GRPC service with all of its attendant dependencies. Fortunately, the Go compiler only pulls in code that is actually used by the client. This means that the binaries created by the consumers of the library will only depend on the library code and nothing from the cmd directory, or elements of the internal directly that aren’t called by the library will not be included.
Consequences
- Able to use the same code in library or application form as required.
Alternatives Considered
Another option would be to separate the library and the GRPC service into different repos. That would allow us to update the service and library separately so that clients of the library would know unambiguously which changes are relevant to them. In this model, we would update the service with the latest version of the library as the library changed.
The disadvantage is the separate maintenance of an entire repository. Since we don’t expect the service portion to change very much once the project is released the extra maintenance overhead wasn’t worth it.