Beware of Shadows

dev go philosophy

You’re a humble DevOps practitioner. You like Go because it’s beautifully C-like without requiring real smarts most modern DevOps tools are written in it. You’ve read Effective Go a few times, understand block scope and rely heavily on redeclaration.

One day you write this snippet (obfuscated to protect the guilty) which lints, compiles and passes all tests (clearly I could use better tests):

// in a caller far, far away...
ctx := context.Background()
res, err := someFunc(ctx)

// later in someFunc...
var things []foo.Thing
if len(thingID) > 0 {
	thing, err := getThing(ctx, thingID)
	if err != nil {
		logger.Fatal(err)
	}
	things = append(things, thing)
} else {
	things, err := getThings(ctx)
	if err != nil {
		logger.Fatal(err)
	}
	if len(things) == 0 {
		fmt.Println("no things!")
		return
	}
}

The code started only supporting collections. When adding support for single items, things was declared in advance and control flow added to assign as needed (real developers would keep the happy path aligned, a noble goal that’s orthogonal to this discussion).

The single case works fine since things is assigned with =. While you understood := to magically redeclare or reassign as needed, the else fails miserably because things is in fact not assigned as expected (it will create a new block-scoped things, with subsequent code operating on the originally declared things). You confirm this the only way a non-programmer knows how — lots of fmt.Printf statements checking the length and content of things. Yep, it’s an empty list.

In desperation your primordial DevOps brain rewrites… It looks sloppy, but works. You manage to sleep at night by convincing yourself this really is somewhat idiomatic (one nice thing about Go is not getting overly obsessed with DRY or trying to be clever at the cost of readability; “just writing more code” is often the idiomatic way):

var things []foo.Thing
if len(thingID) > 0 {
	thing, err := getThing(ctx, thingID)
	if err != nil {
		logger.Fatal(err)
	}
	things = append(things, thing)
} else {
	t, err := getThings(ctx)
	if err != nil {
		logger.Fatal(err)
	}
	if len(t) == 0 {
		fmt.Println("no things!")
		return
	}
	things = t
}

What gives? Re-reading the docs you see:

In a := declaration a variable v may appear even if it has already been declared, provided:

  • this declaration is in the same scope as the existing declaration of v (if v is already declared in an outer scope, the declaration will create a new variable),
  • the corresponding value in the initialization is assignable to v, and
  • there is at least one other variable that is created by the declaration.

I’ve read that many times and rely on it almost daily, but occasionally still find myself holding an incorrect mental model. My brain decides the first version feels right (function scope where the initial declaration occurs rules redeclaration), even though the declaration is scoped to the nearest block (the if).

This is so common in Go it has an official name, “shadowing”. Give it a search and the confusion is enlightening. Jon Bodner’s Learning Go even calls it out as a specific case to be aware of with a suggested fix:

Because := allows you to assign to both new and existing variables, it sometimes creates new variables when you think you are reusing existing ones. In those situations, explicitly declare all of your new variables with var to make it clear which variables are new, and then use the assignment (=) operator to assign values to both new and old variables.

Learning Go, Chapter 2, “var Versus :=”

Shadowing can happen in many contexts including “simple” cases with a single variable, multi-variable declarations and assignments (more common in my experience), and package imports.

You sit back, take another sip of coffee, and think about the epiphany… Nope, no epiphany here. Just another “WTF” moment where useful magic bites you in the ass. You know your developer friends will give you side-eye over the extra variable or not dropping the else entirely, but just as you start another rewrite the pager goes off. #devopslife