Portable Stimulus And Integrated Verification Flows

Posted on by nosnhojn

While a lot of information is produced to introduce and support individual verification techniques, methods for applying a variety of verification techniques in a complementary way are harder to come by. What’s here is a push in that direction; a rough guideline covering multiple techniques with suggestions for where each is most effective and how they integrate to verify a complete SoC.

The idea of building a verification flow isn’t at all new, but the possibilities for and constraints of such flows naturally change as new ideas emerge. The event that instigated this particular article was the release of the Early Adopter Portable Stimulus Standard. Long story short: with all the excitement around portable stimulus, I haven’t seen anyone paint a clear picture of how portable stimulus (a) integrates with or (b) replaces existing verification flows. To fill that void, I’ve mapped a set of possibilities for a generalized verification flow – one that includes portable stimulus – as a reference point.

Continue reading →

39% of Your Verification Budget Goes to What???

Posted on by nosnhojn

I used to drive a piece of garbage green 2000 Civic sedan. It sipped gas, was super reliable on the days it would start, and it didn’t matter how I abused it because it was already as bad as a driveable car could get. It was perfect for a guy like me who couldn’t care less about the vehicle he drove.

In November last year we bought a trailer. We needed a vehicle to pull it so I cashed in the crap-mobile (that’s what my kids called it) for a $50 tax receipt and upgraded to a 2004 4Runner. Somewhat unexpectedly, I’ve turned into a “truck guy” in the 9 months since and my dream vehicle has become a new 4Runner TRD PRO.

A 2017 Toyota 4Runner costs $44,440; that’s the TRD PRO model with the roof rack and alloy wheel options. Because my aversion to spending money is stronger than my love for dream vehicles, I’ve decided I can’t afford that. But I’d still like to drive one so I have to find a way other than paying for one myself. I think I found it… Continue reading →

The Time It Takes To Find A Bug

Posted on by nosnhojn

How much time passes between writing a line of RTL or testbench code and knowing that it works. I know it varies, but on average how long would it take? A line of code is written, time passes, then a test is run that either verifies it’s correct or discovers a bug. Does that time pass as minutes? As days? Maybe weeks? Months?

I’m preparing for a talk at DAC next week in the Verification Academy booth and in going through my material I’ve started asking myself that question. It’s the 3rd year in a row the folks at Mentor have invited me for a session in the booth. The last 2 years have been a lot of fun and I’m hoping for the same this year. This year’s session is called Add Unit Testing To Your Verification Toolbelt. As the name suggests, the focus will be on how we can use unit testing and test-driven development to improve the quality of the hardware we build.

I believe quite strongly in the value of unit testing and test-driven development because the quality of what I produce as a verification engineer is noticeably better because of them. I believe so strongly in TDD in particular that I know I get a little preachy at times. It’s the most important engineering skill I’ve ever learned and I don’t shy away from telling people about it.

I always think of rigour first when it comes to benefits. Testing code as you write it a few lines at a time is a rigorous process to say the least. But during the run-up to DAC I’m starting to think it’s timing that’s the real key. With TDD, the amount of time that passes between writing and testing a line of code is minutes, sometimes less. A design and test cycle measured in minutes is almost too short a time to think about anything else. I focus on getting exactly 1 thing right. There’s no thinking back, no back-and-forth with a teammate that found a bug, flipping through my logbook for clues as to what I’ve done or editing bug reports. In fact, with TDD there’s hardly any context switching at all. I write a line or 2, test them, if they’re broken I fix them, then I move to the next line. Most of the bugs I create are killed immediately; which coincidentally is the best time to kill them.

I’ve seen a lot of people talk about how bugs become more costly to fix the longer they live. For me, TDD has been the most effective way to respond.

If you’re at DAC next week in Austin and using TDD to kill bugs fast sounds good – or great! – I hope to see you at Mentor’s Verification Academy Booth at 11am on Monday. It’ll be a half hour presentation with lots of time for discussion. We’ll talk about TDD, unit testing, SVUnit and all the frustration that magically vanishes because of them!

Here’s a link to the session info for mine and others in the booth on VerificationAcademy.com.

See you at DAC!

-neil

Take The Um Out Of Scrum

Posted on by nosnhojn

Scrum is big in software and it’s slowly creeping into hardware development. To me this is good news because Scrum would be a transformative for hardware teams. In an industry that relies heavily on the big bang, iteratively coming to potentially shippable checkpoints, as a team would do with Scrum, would be a welcome change. It would give users and customers early access to product (or some version of a product depending on the technology involved), there’d be increased opportunities for feedback and the narrowed sprint-to-sprint focus on feature subsets would propel a significant improvement in quality.

Yes, Scrum would be great for hardware teams… unless, of course, we strip out the practices and objectives that make it effective and bend the rest around practices we already use without actually changing much. That would be less great. Continue reading →

Definition of (De)Bug

Posted on by nosnhojn

In 9 years of blogging, I’m guessing at least half of what I’ve written centered around code quality or debug. For all the writing and time I’ve put into these topics, it took me until yesterday, for whatever reason, to realize I’m without a solid definition for debug. Or even a definition for bug for that matter. I guess I was relying on a common understanding of what bugs are and what it means to debug code. Realizing that may not be the case, I came up with a definition that captures what bugs are to me…

Bug: A state where a feature is defined, implemented and/or expected to fulfill a specific purpose but fails to do so.

Continue reading →

I Can’t Hide From Unit Tests

Posted on by nosnhojn

There are times when I try and convince myself that unit testing my verification IP may not be necessary. Believe it or not, I can put together a pretty convincing argument. On a good day I shake it off and do the right thing. Other times I’m just convincing enough. I know it won’t end well, but I skip the unit tests and go straight to the code.

Here’s a couple arguments I’ve used on myself to get into trouble. Continue reading →

Agile Hardware Kickstart

Posted on by nosnhojn

I had a neat opportunity to do a lunch-n-learn with a large group of engineers a few weeks ago. It was organized by the IEEE Computer Society as part of a program they use to pair engineering teams with various speakers and topics. They found me through a referral from an agile software friend of mine. I pitched a topic that sounded interesting enough so it was all systems go. I flew in for a day the third week of november. We had 250 people tune in. Good realtime feedback and a great experience for me.

I’ve done webinars like this in the past but I’ve never felt like I left people with a solid feel for how they get started with agile hardware. I wanted this time to be different; I wanted to boil the mess of information that floats around in my head down to concrete possibilities; I wanted those listening to go back to their desks feeling like they had the gumption to get something started. Continue reading →