This document details the official style guides for the various languages we use in the Thanos project. Feel free to familiarize yourself with and refer to this document during code reviews. If something in our codebase does not match the style, it means it was missed or it was written before this document. Help wanted to fix it! (:
Generally, we care about:
timeNow func() time.Time
mock.Some style is enforced by our linters and is covered in separate smaller sections. Please look there if you want to embrace some of the rules in your own project! For Thanos developers, we recommend reading sections about rules to manually apply during development. Some of those are currently impossible to detect with linters. Ideally, everything would be automated. (:
For code written in Go we use the standard Go style guides (Effective Go, CodeReviewComments) with a few additional rules that make certain areas stricter than the standard guides. This ensures even better consistency in modern distributed system databases like Thanos, where reliability, performance, and maintainability are extremely important.
In this section, we will go through rules that on top of the standard guides that we apply during development and code reviews.
NOTE: If you know that any of those rules can be enabled by some linter, automatically, let us know! (:
The coding style is not purely about what is ugly and what is not. It’s mainly to make sure programs are reliable for running on production 24h per day without causing incidents. The following rules are describing some unhealthy patterns we have seen across the Go community that are often forgotten. Those things can be considered bugs or can significantly increase the chances of introducing a bug.
It’s easy to forget to check the error returned by a Close
method that we deferred.
f, err := os.Open(...)
if err != nil {
// handle..
}
defer f.Close() // What if an error occurs here?
// Write something to file... etc.
Unchecked errors like this can lead to major bugs. Consider the above example: the *os.File
Close
method can be responsible for actually flushing to the file, so if an error occurs at that point, the whole write might be aborted! 😱
Always check errors! To make it consistent and not distracting, use our runutil helper package, e.g.:
// Use `CloseWithErrCapture` if you want to close and fail the function or
// method on a `f.Close` error (make sure the `error` return argument is
// named as `err`). If the error is already present, `CloseWithErrCapture`
// will append (not wrap) the `f.Close` error if any.
defer runutil.CloseWithErrCapture(&err, f, "close file")
// Use `CloseWithLogOnErr` if you want to close and log error on `Warn`
// level on a `f.Close` error.
defer runutil.CloseWithLogOnErr(logger, f, "close file")
Avoid 🔥 |
---|
|
Better 🤓 |
|
One of the most common bugs is forgetting to close or fully read the bodies of HTTP requests and responses, especially on error. If you read the body of such structures, you can use the runutil helper as well:
defer runutil.ExhaustCloseWithLogOnErr(logger, resp.Body, "close response")
Avoid 🔥 |
---|
|
Better 🤓 |
|
No globals other than const
are allowed. Period. This means also, no init
functions.
Never use them. If some dependency uses it, use recover. Also, consider avoiding that dependency. 🙈
reflect
or unsafe
Packages #Use those only for very specific, critical cases. Especially reflect
tend to be very slow. For testing code, it’s fine to use reflect.
Variable shadowing is when you use the same variable name in a smaller scope that “shadows”. This is very dangerous as it leads to many surprises. It’s extremely hard to debug such problems as they might appear in unrelated parts of the code. And what’s broken is tiny :
or lack of it.
Avoid 🔥 |
---|
|
Better 🤓 |
|
This is also why we recommend to scope errors if you can:
if err := doSomething(); err != nil {
// handle err
}
While it’s not yet configured, we might think consider not permitting variable shadowing with golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow
in future. There was even Go 2 proposal for disabling this in the language itself, but was rejected:
Similar to this problem is the package name shadowing. While it is less dangerous, it can cause similar issues, so avoid package shadowing if you can.
After all, Thanos system is a database that has to perform queries over terabytes of data within human friendly response times. This might require some additional patterns in our code. With those patterns, we try to not sacrifice the readability and apply those only on the critical code paths.
Keep in mind to always measure the results. The Go performance relies on many hidden things and tweaks, so the good micro benchmark, following with the real system load test is in most times required to tell if optimization makes sense.
Try to always preallocate slices and map. If you know the number of elements you want to put apriori, use that knowledge! This significantly improves the latency of such code. Consider this as micro optimization, however, it’s a good pattern to do it always, as it does not add much complexity. Performance wise, it’s only relevant for critical, code paths with big arrays.
NOTE: This is because, in very simple view, the Go runtime allocates 2 times the current size. So if you expect million of elements, Go will do many allocations on append
in between instead of just one if you preallocate.
Avoid 🔥 |
---|
|
Better 🤓 |
|
To extend the above point, there are cases where you don’t need to allocate new space in memory all the time. If you repeat the certain operation on slices sequentially and you just release the array on every iteration, it’s reasonable to reuse the underlying array for those. This can give quite enormous gains for critical paths. Unfortunately, currently there is no way to reuse the underlying array for maps.
NOTE: Why you cannot just allocate slice and release and in new iteration allocate and release again etc? Go should know it has available space and just reuses that no? (: Well, it’s not that easy. TL;DR is that Go Garbage Collection runs periodically or on certain cases (big heap), but definitely not on every iteration of your loop (that would be super slow). Read more in details here.
Avoid 🔥 |
---|
|
Better 🤓 |
|
The part that all Gophers love ❤️ How to make code more readable?
For the Thanos Team, readability is about programming in a way that does not surprise the reader of the code. All the details and inconsistencies can distract or mislead the reader, so every character or newline might matter. That’s why we might be spending more time on every Pull Requests’ review, especially in the beginning, but for a good reason! To make sure we can quickly understand, extend and fix problems with our system.
This is connected more to the API design than coding, but even during small coding decisions it matter. For example how you define functions or methods. There are two general rules:
Avoid 🔥 |
---|
|
Better 🤓 |
|
Avoid 🔥 |
---|
|
Better 🤓 |
|
This is a little bit connected to There should be one-- and preferably only one --obvious way to do it
and DRY
rules. If you have more ways of doing something than one, it means you have a wider interface, allowing more opportunities for errors, ambiguity and maintenance burden.
Avoid 🔥 |
---|
|
Better 🤓 |
|
It’s OK to name return parameters if the types do not give enough information about what function or method actually returns. Another use case is when you want to define a variable, e.g. a slice.
IMPORTANT: never use naked return
statements with named return parameters. This compiles but it makes returning values implicit and thus more prone to surprises.
There is a way to sacrifice defer in order to properly close all on each error. Repetition makes it easier to make an error and forget something when changing the code, so on-error deferring is doable:
Avoid 🔥 |
---|
|
Better 🤓 |
|
Always handle returned errors. It does not mean you cannot “ignore” the error for some reason, e.g. if we know implementation will not return anything meaningful. You can ignore the error, but do so explicitly:
Avoid 🔥 |
---|
|
Better 🤓 |
|
The exception: well-known cases such as level.Debug|Warn
etc and fmt.Fprint*
It’s tempting to define a variable as an intermittent step to create something bigger. Avoid defining such a variable if it’s used only once. When you create a variable the reader expects some other usage of this variable than one, so it can be annoying to every time double check that and realize that it’s only used once.
Avoid 🔥 |
---|
|
Better 🤓 |
|
Prefer function/method definitions with arguments in a single line. If it’s too wide, put each argument on a new line.
Avoid 🔥 |
---|
|
Better 🤓 |
|
This applies for both calling and defining method / function.
NOTE: One exception would be when you expect the variadic (e.g. ...string
) arguments to be filled in pairs, e.g:
level.Info(logger).Log(
"msg", "found something epic during compaction; this looks amazing",
"compNumber", compNumber,
"block", id,
"elapsed", timeElapsed,
)
else
#In most of the cases, you don’t need else
. You can usually use continue
, break
or return
to end an if
block. This enables having one less indent and better consistency so code is more readable.
Avoid 🔥 |
---|
|
Better 🤓 |
|
We use pkg/errors
package for errors
. We prefer it over standard wrapping with fmt.Errorf
+ %w
, as errors.Wrap
is explicit. It’s easy to by accident replace %w
with %v
or to add extra inconsistent characters to the string.
Use pkg/errors.Wrap
to wrap errors for future context when errors occur. It’s recommended to add more interesting variables to add context using errors.Wrapf
, e.g. file names, IDs or things that fail, etc.
NOTE: never prefix wrap messages with wording like failed ...
or error occurred while...
. Just describe what we wanted to do when the failure occurred. Those prefixes are just noise. We are wrapping error, so it’s obvious that some error occurred, right? (: Improve readability and consider avoiding those.
Avoid 🔥 |
---|
|
Better 🤓 |
|
_
#Blank identifiers are very useful to mark variables that are not used. Consider the following cases:
// We don't need the second return parameter.
// Let's use the blank identifier instead.
a, _, err := function1(...)
if err != nil {
// handle err
}
// We don't need to use this variable, we
// just want to make sure TypeA implements InterfaceA.
var _ InterfaceA = TypeA
// We don't use context argument; let's use the blank
// identifier to make it clear.
func (t *Type) SomeMethod(_ context.Context, abc int) error {
We use go-kit logger in Thanos. This means that we expect log lines to have a certain structure. Structure means that instead of adding variables to the message, those should be passed as separate fields. Keep in mind that all log lines in Thanos should be lowercase
(readability and consistency) and all struct keys are using camelCase
. It’s suggested to keep key names short and consistent. For example, if we always use block
for block ID, let’s not use in the other single log message id
.
Avoid 🔥 |
---|
|
Better 🤓 |
|
Additionally, there are certain rules we suggest while using different log levels:
msg
field. It should be used only for important events that we expect to happen not too often.msg
field. It can be a bit more spammy, but should not be everywhere as well. Use it only when you want to really dive into some problems in certain areas.msg
or err
or both fields. They should warn about events that are suspicious and to investigate but the process can gracefully mitigate it. Always try to describe how it was mitigated, what action will be performed e.g. value will be skipped
msg
or err
or both fields. Use it only for a critical event.Comments are not the best. They age quickly and the compiler does not fail if you will forget to update them. So use comments only when necessary. And it is necessary to comment on code that can surprise the user. Sometimes, complexity is necessary, for example for performance. Comment in this case why such optimization was needed. If something was done temporarily add TODO(<github name>): <something, with GitHub issue link ideally>
.
Use table-driven tests that use t.Run for readability. They are easy to read and allows to add a clean description of each test case. Adding or adapting test cases is also easier.
Avoid 🔥 |
---|
|
Better 🤓 |
|
time
package. #Avoid unit testing based on real-time. Always try to mock time that is used within struct by using, for example, a timeNow func() time.Time
field. For production code, you can initialize the field with time.Now
. For test code, you can set a custom time that will be used by the struct.
Avoid 🔥 |
---|
|
Better 🤓 |
|
This is the list of rules we ensure automatically. This section is for those who are curious why such linting rules were added or want similar ones in their Go project. 🤗
Never use print
. Always use a passed go-kit/log.Logger
.
Ensured here.
It’s very easy to forget to add Prometheus metrics (e.g a prometheus.Counter
) into a registry.MustRegister
function. To avoid this, we ensure all metrics are created via promtest.With(r).New*
and we disallow the old type of registration. Read more about the problem here.
Ensured here.
Standard Go vet is quite strict, but for a good reason. Always vet your Go code!
Ensured here.
golangci-lint is an amazing tool that allows running a set of different custom linters from the Go community against your code. Give it a star and use it. (:
Ensured here with those linters enabled.
Misspell is amazing, it catches typos in comments and docs.
No Grammarly plugin for this yet ): (We wish).
Ensured here, using golangci-lint / misspell.
All comments should be full sentences. They should start with an uppercase letter and end with a period.
Ensured here using golangci-lint / godot.
Overall try to NOT use bash. For scripts longer than 30 lines, consider writing it in Go as we did here.
If you have to, we follow the Google Shell style guide: https://google.github.io/styleguide/shellguide.html Ensured here using shfmt. We also use shellcheck to check any script errors.