BDD Without the Three Amigos: Maybe Talking To Yourself Isn’t So Bad
If you search online for articles on Behavior-Driven Development (BDD), you’re likely to find more literature on how everyone is doing it wrong versus what it really is and practical success stories in using it. I’ve read article after article explaining how teams are claiming to practice BDD but are falling short on the core intention of this process which is to communicate early in the development process to ensure the team is building the right thing.
BDD is ideally practiced by what is known as the three amigos: the business, the developer, and the tester. However, in the real world, it’s quite the uphill battle to get team buy-in to actually practice BDD in its pure form. Excuses I often hear from the business people are that they don’t have time to learn something new. Developers say they don’t want yet another meeting and don’t see why this is necessary. Which leaves the tester as the only willing participant.
So, without the three amigos, BDD purists argue there is no BDD! Yet, more and more testers are giving up on trying to get everyone on board and are focusing on one of the shining side effects of BDD: executable test scenarios! When I consult with teams who are simply using Gherkin for test automation purposes, I regurgitate what I’ve read in the literature:
- this is not proper use of BDD; communication is key
- you’re not getting the full advantages of pure BDD
- you’re adding an unnecessary layer of code to your automation framework
Shockingly, this has never been news to any of them. They are all well aware of the intentions of BDD and how it should be used. However, obtaining team buy-in was a battle lost, and they really, really want the executable test scenarios which can be written by their non-technical testers and plugged into a framework developed by automation engineers or developers. In addition to having more people contribute to the automation efforts, they also value the scenarios serving as documentation of how the system behaves.
I’ve seen multiple teams operate this way and they are very satisfied with what they have. I’ve waited for the bottom to fall out of their automation strategy, but it hasn’t. I’m not exactly sure if I’ve just not waited long enough, or if maybe using BDD testing frameworks (e.g. Cucumber, JBehave, Specflow, etc) for the sole purpose of test automation is not as horrible as the purists have made us believe.
This caused me to stop and think…why is everyone preaching against operating this way? In theory, it makes total sense. Heck, I’ve preached against it myself. However, I couldn’t find an answer to the why in any of the articles or in my own thoughts other than this is not how BDD was intended to be used and therefore is not BDD. But we all know that users don’t care about the intended use of software or processes; they use it to meet their needs. Maybe this is no different?
In discussing this topic on Twitter, I realized that I have never seen a tester successfully gain team adoption of BDD. The teams that I have seen doing BDD have all had the process pitched by a non-tester. So maybe instead of yet another “you’re not really doing BDD” article that assumes automation engineers are naively using these testing frameworks, we should put more focus on how to get team-level adoption to practice BDD in its pure form and obtain all of the wonderful benefits that the testers and automation engineers are fully aware they are missing out on.
Lisa Crispin
Hmm. My last team practiced ATDD (meaning, we didn’t use a G/W/T style format, we mostly used FitNesse column fixture tests) successfully for 7+ years. Of course, not every test case was written in a 3 amigos (or what Janet and I called Power of Three in our first book) meeting. However, we did have one or more devs and a PO and a tester get together to agree on at least high level acceptance test cases for each story. Definitely, it was us testers that dug down once the happy path test was running and added the boundary tests, edge cases, sad paths and so on.
I admit that on my current team, I have not yet been successful in getting them to adopt ATDD/BDD over the long term. We did BDD on one team for about 9 months and saw both our rejection rate and our cycle time go down dramatically because we had built that shared understanding of each story before coding began. But then the old people left the team and new people came on and they didn’t keep it going. To be fair, the team is much smaller, so we can have the conversations to get the shared understanding easily enough in the pre-IPM and IPM meetings. But I do think ATDD/BDD would still help.
I think the Cucumber devs would like us to understand that Cucumber is a test specification tool rather than an automation tool, though the automated regression tests are a nice side benefit. For my teams, it was important not to go overboard with specifying tests up front. We limited ourselves to tests that focused on the main goal of the story or the most important value to the customer. If we specified every case we could think of, the devs couldn’t see the forest for the trees. We still used our automation framework to pump in a lot of sad and mad path inputs, but we didn’t keep all of those tests in our regression suite.
I guess what I’m saying is it is GREAT for testers or BAs or whoever to specify all the test cases they can think up and automate them if they can do it with a good ROI. Getting shared understanding of each feature and story is kind of a separate deal. We could equally get that shared understanding with something like story mapping or example mapping and again, though I would prefer using something like Cucumber to capture the rules and examples and automate tests, it might be possible to succeed without doing that. But I think most teams still need automated regression tests to succeed long-term.
Pingback: Five Blogs – 9 January 2017 – 5blogs
Hemant Sathe
I believe unless you adapt specification by example, GWT is just another tool for test automation. Essential part of BDD is to have behaviour and hence alternate possibilities in a scenario. If we do not have examples which demonstrate the behaviour (which is responding based on the values in input), we just have simple CRUD kind of operations and the GWT just helps to validate the test is correct. Any tool implementing GWT is just a tool and I earlier used to think I can use it for unit testing to system testing depending on the abstraction I am doing. I did use it like that and found that it was easier to set up my unit tests since I did not have a good handle on NUnit/XUnit frameworks. But once I learned the unit testing frameworks, I found using BDD is overhead and no business value. (BTW many people read BDD as Business Driven Development). The only way three amigos can be involved is to do Spec. By Example workshops first and them move the identified scenarios to BDD if needed. Else simple tests are enough for automation.
Jason Koelewyn
Having been on teams with 1, 2 and 3 amigos, I can say that BDD is a very effective tool for automation, a great end in itself. However when you do have all 3 at the table, what you have is much more then a testing tool. As long as you understand what you have there is nothing wrong with just using BDD for automation.
But I agree that the better discussion is how to drive full adoption of BDD. That seems to be the point most teams struggle with, and it can be a major force multiplier for teams.
Dan Ashby
Nice post! I’ve thought about this alot in the past too, but I came to the conclusion that if it’s just the testers writing the scenarios as test cases to plug into automation, then GWTs aren’t very lean (the teams that I’ve seen do this typically had tons of lines of “and” after a given for test data setup)… Why do that, when you can write the expectations in any format and assert it in the automation. Is it just so they can say they are doing BDD (because as you pointed out, they aren’t) or using cucumber (which I think is great when you have that feature file already created as an output of BDD if you are doing that)…
A way that I started to look at it was that BDD is the conversations (purpose is to refine/refactor) our idea of the features), which has the output of feature files. If we have the feature files, then we can use cucumber to reuse them and plug them into our glue code and step defs to assert them. If we don’t do BDD (hence have no feature files) what’s the lightest way of writing our assertions for our automation? (my answer – probably not feature files… But I may be wrong).
I like fit tables like Lisa pointed out. They are leaner from my experience.
But overall, I like this post as I recently blogged about BDD from the “negative” perspective on BDD usage, so I need to learn from you!!
Angie Jones
very good (and true) point about the bloated feature files when not doing pure BDD. I’ve seen this quite a bit as well.
Stephanie DESBY
Very good post. Thank you.
I struggle to have full 3 amigos conversations throught the project lifetime. We have them when the project is not too big and specification are mostly functional (not too many technical needs), by using Example Mapping.
But most of the time, I’m alone writing my scenarios. I go after information and validation of my scenarios when I need too, I even email them to get validation but we usually don’t write the GWT together.
I use Cucumber for my automation framework and even if we don’t do BDD all the time, I noticed that the devs read my feature files when they need to remember what a feature, as I do. So I really see the feature files as our living documentation.
Noticing that, I started to enhance our feature files and take care to explain the feature and conditions in details in the files. For example, I have sentences in my gherkin files that haven’t got code under them. They are here only for the good comprehension of the feature.
So I think that having the GWT format even if we don’t do “pure BDD” is not a waste of time. It helps us to have a good view of what our API can do and why we did the feature.
Sorry for my English 🙂