Finally solved a mystery this week after two different things happened. First… got an email from a coach telling me she was doing some kanban training with a hardware team. Second… had a chat with a couple guys about using scrum with a verification team. Put those 2 seemingly benign events together and voila… I finally find the right way to describe how I’ve approached functional verification the last couple years: it’s kanban with single item flow.

Here’s what that means…

Kanban is powerful but it’s not rocket science so if it’s new to you, don’t worry. In a nutshell, kanban is a way to visualize and manage your workflow. You define a series of steps on a kanban board, then you push all the items in your to-do list through each step, step by step. The arrangement on the board obviously depends on what you’re building. Thinking back, this how a kanban board would look for me, as a verification engineer.

The graphic shows a list of features down the left (i.e. my to-do list). I write my to-do list as a list of tests, a starting point that shouldn’t sound all that fancy considering most other verification teams I’ve seen build a testlist fairly early on in development. From my test list, I take 1 test and move it into the testbench support column. For testbench support, I build out the bare minimum infrastructure required to write and run the test. I use TDD to build out the testbench. When my testbench code is done and my unit tests pass, I move to the RTL test column where – you guessed it – I write and run an RTL test. When the RTL test passes, we go to the DONE column. In the above graphic, I’d have test 0 passing against the RTL and I’d be building the testbench features required for test 1.

In the past, I’ve used a diagram like this to show something similar, but with the focus being on TDD…

A few guidelines I like to follow with kanban for one:

• I like to limit the number of tests being worked on at a given time to 1. That’s why I called this kanban with single item flow. Maybe that doesn’t sound efficient if you’re used to using techniques like constrained random, but I’ve found focus to be surprisingly productive so I roll with it :).
• There’s only 2 steps and escape from both hinges on producing passing tests. The reason I don’t have more steps is that more movement can suggest false progress (i.e. “progress” that’s easily nullified down the road because of poor quality and/or wrong intent). Nothing is DONE until a test says it’s DONE.
• The toughest transition in this approach will be keeping a lid on how much testbench code you build for each feature. I’ve found keeping it to the absolute minimum, give or take a hair, to be the most productive. It keeps your solution space small and keeps you from debugging code that isn’t directly related to your immediate objective.
• If documentation is important to you, I would add a column for it before DONE treating it as a summary exercise before moving on to the next test. I wouldn’t add a planning or analysis column at the beginning though because planning is ongoing and intangible. Trying to nail down planning isn’t overly meaningful to me.

Like I said, kanban is step by step. No, it’s not rocket science, but there are huge implications. To summarize, the idea of mass testbench construction followed by test writing and running, followed by signoff (DONEness) goes out the window. Instead, you’re building and applying your testbench incrementally. I’ve written about this idea numerous times but this is the first time I’ve suggested the kanban link. I think kanban strengthens the idea of incremental development and application of a testbench considerably simply by quantifying the steps you’re going through and limiting work-in-progress.

Going back to why a conversation about scrum helped bring this on… the reason why I’m treating this as a bit of an epiphany is because kanban now feels to me like the right alternative for individual verification engineers or small verification teams working within a system that was not meant to support agile development. Kanban feels a little more straightforward than scrum does. You have the visual and you don’t have the self-imposed sprint deadlines that come with scrum so you have the breathing room to break old habits (note: I do acknowledge the counterpoint, that sprint deadlines can be effective in forcing you to break habits also). And if you’re really strict about maintaining single item flow it’s even more difficult because there’s no room for the old habits!

A few other points…

• Being interrupted with scrum means your sprint deliverables are jeopardized and I know people get concerned with that. With kanban it doesn’t seem to matter as much. Flow stops, of course, but the interruptions don’t have the same impact; you just carry on from where you left off.
• In the ideal scenario you’d share milestones with designers so you can incrementally build and test RTL, sprint by sprint. But if designers don’t buy into scrum, at least with kanban you can still soldier on feature-by-feature, changing priorities when necessary, assuming your designers have had the usual head start (i.e. you can verify features they’ve already coded and/or kindly ask them to prioritize their development in return for some fast turnaround test results).
• For a team of 1, there are no official roles like scrum master or product owner to fill (or pretend to fill) if you go with kanban. And for a small team of 2-4, there’s no urge to impose a bureaucracy that doesn’t make sense in your situation.

Before I go I better finish by saying I’ll never suggest people avoid scrum. There’s many different ways to get your feet wet with agile and ruling any of them out doesn’t make sense. If you’re using scrum and it works, you should carry on. I’ll also continue to advocate for scrum where people find themselves in the right circumstances. When you do have support for agile development that extends beyond you and the verification team (i.e. designers at minimum and hopefully managers have genuinely bought in), I think that can be the time to move ahead to scrum. You’ll have shared deliverables with a design and test component to each story (having both is critical), teamwide cohesion and collaboration will improve and you’ll be able to use your kanban experience within each sprint.

Until then, kanban for one can be the right place to start, especially for verification engineers.

PS: if you’re interested in a super nerdy way to build a kanban board with emacs, check out this post from Bryan from a couple years ago.