A Theory of Optimistic Performance

About 15 years ago my manager was quoted in an article where he said that software testers are inherently pessimistic people. The idea has resonated with me over the years, and it has shifted in my mind as I have shifted in my career from being a full-time quality analyst to becoming a test consultant specializing in performance testing. 

Initially, I had agreed with him, based solely on my experience in enterprise IT departments (and relatively little worldly experience.) Starting out as an internal quality analyst I spent much of my time learning how to break software and processes, learning how to find bugs. That’s what I got paid to do. But I also spend quite a bit of time being confused and frustrated in that role.

When I became a consultant, things flipped around. I spent more of my time focusing on the optimistic side of testing -- the benefits, the value, and the return on investment. Testing had to be a positive investment for my customers. And I realized that becoming a testing evangelist distanced me from those old feelings of frustration or disappointment, because in a consultative role, I had to be inspiring and persuasive.  

Then I went back to working as a tester again, at Microsoft, and I spent time every week with real customers engaged in the real world, one full of frustrating bugs, disappointing circumstance, and looming failures. This dose of reality brought me back down to earth. There it was again, that frustration and pessimism. But this time, during those short-cycled and intense performance optimization engagements, I saw customers change from a state of frustration to one of celebration. Reaching new peaks for performance with customers was exciting and positive, and often the result of  tireless efforts. 

And this new view on the experience led me to create my own theory about testers.

We are not inherently pessimistic people. Instead, we are actually relentlessly hopeful that we won't be disappointed. What I mean is, we aren't frustrated with the world as it is around us, and we also aren't longing for a more traditional approach to something. Testers don't lament about the past. Instead, when we are testing, we are all just temporarily frustrated, because our hopeful vision for an improved future just isn’t quite here. Yet.  

This is especially true for performance testers, where we attempt to manifest success for a system that’s limping along slowly and wrapped-up around itself, and we think “I know it’s just a few more tweaks and we’ll uncover the real power of this system!” 

So, the next time you get some complaint or push back on a performance defect where the developer is growing tired of your presumed negative or pessimistic attitude, be the optimistic tester you actually are. Remind them of your feelings about the future. Tell them about how you envision this bug -- and so many more -- being fixed. Let them in on the secret that you are not complaining about what they did in their code at some point in the past, but that you are thinking about how to celebrate the success that lies just around the corner.

Bucket full of Bottlenecks

For years I've seen a few super-guru performance engineers impress everyone by walking into the lab and with just 2-3 clicks through the code casually proclaim: “Sure, you just need to tweak the buffer setting in the max_connections_kernel_init.xml file to match the incoming parameters for the extension listing sequencer” with an “everybody-knows-that-one” attitude.  I figured my dream of becoming that cool could never happen. Or could it?

At one time or another in my own career as a performance engineer and tester, my self-perception was that I needed to be the guy who knows a huge list of all bottlenecks, and that would be my secret to success. But I've come to see there are 2 major flaws with this objective:

1)  there are a whole lotta of bottlenecks out there, and
2)  the list of bottlenecks is always changing and evolving

What I’ve learned from spending my working life encountering massive techno-churn, new platforms, and new usage patterns is that knowing where bottlenecks come from is far more important than knowing the bottlenecks after they exist. Understanding the engineering patterns that result in a performance bottleneck means that you can help to prevent performance bottlenecks from getting into the system in the first place.

In one of the performance engineering workshops I teach (“Performance Engineering for Product Owners”), I explain the importance of writing the specific performance requirements into the user stories or specifications.  Ideally, this means as the user story is evolving to become dev-ready there’s an opportunity to engineer performance into the thinking of the design, and that will impact how a developer responds to writing code or constructing the system to deliver on that story. And product owners can get ahead of bottlenecks, too.

So, the next time you are tirelessly tracking down the root cause of a performance bottleneck and some super-guru performance engineer comes walking in to pull a "miracle" solution from their bucket full of bottlenecks – remember the relative longevity of that fix, and the limits on such specific solutions.

STPCON 2012 Reflections on Excellence

If you’re like me, when you get a chance to attend a professional conference you look forward to meeting up with other people in “your tribe” (as some say).  I love these opportunities to connect with colleagues and peers who I’ve not seen for months or years sometimes.  Sometimes it’s people whom I have never had the pleasure of meeting in-person.  And always I’m meeting new people – with new ideas and new solutions that open my mind to new things.  My experience in the Spring of 2012 at the Software Testing Professionals conference (STPCON) in New Orleans (NOLA) was an excellent example of all these reasons.  Here’s a rapid-fire reflection on the folks I met and why it’s so valuable to be at this conference with our tribe!

For starters, I met up with Matt Heusser right away and that’s like diving into the deep end of the STPCON pool – he’s so intense, passionate and excited about testing it’s hard to believe.  During our late-night dining at Daisy Dukes, he invented an alcoholic milkshake made of warmed milk and Kahlua.  (Apparently not bad!)  Oh yeah, we also got caught up on work for future conferences and also giving a referral to a customer near his family in Salem, OR.  I got to chat with Jon Bach for 10 minutes before his presentation and introduced myself as a fellow responder to Alberto Savoia’s “Test is Dead” industry-bashing scandal.  Jon’s treatment of the subject in his keynote was a very professional response to the naiveté’ of Alberto and James Whittaker. 

Then, in my first track presentation on Conscientious Testing I noticed a familiar face, Ben Simo (@QualityFrog) in the back – lurking, but also smiling a nodding quite a bit at the review of how to be an ethically aware software tester.  At the same time my excellent friend Scott Moore was also in the session bringing up points of reference from his experience in performance test consulting from the last 20 years.  I stayed on for Scott’s session on Peak Performance Planning which was excellent!!  Later that day I had my second session on Performance Testing Metrics and Measures – which was exhausting, packed, standing-room only – and I got to sync-up with my mentor and peer Scott Barber.  Basically, if you have a job today in performance testing or as a load tester or performance engineer of any kind…you probably owe some thanks to Scott for his undying commitment to our discipline and industry.  That evening I got to hang out with the team from HP Software (John, Sylvia, Heather and Kristin) - chatting about the latest & greatest news from our favorite old ex-Mercury product vendor.  Scott Moore also taught us a new way to properly taste single-malt scotch.

The next morning I participated in Breakfast Bytes – taking an interactive survey of participants on the topic of Performance Engineering vs. Performance Testing titles and careers (to be summarized in a later blog post, coming soon).  I got to meet up with Lanette Creamer, who I recalled from the last STPCON in Fall of 2011 in Dallas, TX.  Lanette is a very sharp tester and presenter with a great sense of humor to keep you going!  At the reception that evening I got to have a marvelous chat with Fiona Charles about her own writing on ethics in software testing. She’s also worked closely with Jerry Weinberg and sent me a few references and readings to consider.

I finally met Dan Bartow from SOASTA in-the-flesh – and he’s really a great guy; enthusiasm for performance is not his weakness in any way.  Brad Johnson and Dan went with a group of us out to the Old Absinthe hotel and like true testing geeks we just hung-out and told stories about testing.  Sure we had some absinthe (the less-potent, legal kind) which was tasty actually.  As the conference continued, there were presentations from several other excellent testing guru’s out there like Kerry Fields, Griffin Jones and Brian Lynch – each of whom gave marvelous presentations, with depth and experience.

But there was a moment that really struck me at the very last day when I saw Ben Simo again and he responded to me:  “I realized – I need to do this.”  [referencing the idea of getting out and presenting, speaking and sharing and participating in the community].  I smiled and said, “Absolutely you should!“

What Ben was experiencing is the kind of buzz at the STPCON event, which is highly positive and uplifting, and that is being surrounded and immersed in testing excellence.  It’s so different from the conflict-ridden existence we typically face each day in our jobs as testers.  It’s different from the politics or the negativity we sometimes are forced to deal with in our work, in business or on our teams.  At STPCON all of that noise just fades into the background and you are surrounded by testers!  You’ll find (like me) that every conversation is filled with opportunity to learn and to share something excellent about software testing. 

See you in Miami – STPCON FALL 2012!!

PS – special thanks to Abbie, Fiona, Rich, Matt and the STP Crew!  This couldn’t happen with your excellent support.