Pros of Behaviour Driven Development

Living Documentation

BDD methodology creates a feedback loop that maintains the connection between documentation and tests. As a result, when tests fail because of an incorrect specification, you are encouraged to update the executable specifications in the documentation. In this way, the documentation remains fairly up to date. This ‘living documentation’ is a more accurate representation of what the software/product actually does.

Collaboration

BDD encourages collaboration between business users, developers and testers. These ‘three amigos’ come together to formulate an idea of the specifications and behaviour of the application/product that they can all agree on. Traditionally, this collaboration takes the form of separate meetings: business user talks to developer; developer talks to tester; tester talks to business user. BDD stipulates that all parties should meet together to ensure that they are all on the same page.

These meetings to define the specifications usually take place right at the start of the project, before any code is written. This helps to identify defects in the specifications early on in the life of the project. This can be a big adjustment for a team. The urge to start building something as soon as possible is always a strong force to contend with. Yet we all know that more time spent up front results in far fewer design and implementation issues surfacing during development.

Domain-Specific Language

BDD encourages the formalisation and usage of a Domain-Specific Language (DSL) that encompasses the terms peculiar to the domain of your business and, therefore, is useful for defining the project specifications with greater precision. This helps to improve communication, drive consistency in specifications and avoid misunderstandings. Having a clear DSL that is used by all team members contributes significantly towards fostering understanding.

Compartmentalisation

BDD encourages you to define specifications and acceptance tests in smaller, more manageable units. The act of defining individual scenarios in dedicated feature files encourages participants to work on specifications in neat chunks. This makes it easier to stay focused on delivering a particular chunk or feature that meets a clearly-defined set of acceptance criteria.

Discovery Workshops

The BDD discovery workshops are key in uncovering additional capabilities and complexities right at the start of the project. Because the specifications are so detailed, and described in terms of application behaviour, a lot more of the finer details and edge cases are scoped at the beginning of the project. Part of the reason for this is the collaborative nature of these sessions. Having more scenarios identified at the start leads to a lot less rework in the later stages of the project.

Project Scope

As a consideration for the developers, detailed specifications are written up front, and tests are written before the code is developed. This encourages developers to only work on implementing what needs to be built in order to make the tests pass. There is less scope for developers to start working on other features that they think might be necessary but aren’t actually within the predefined scope. This curtails the ‘scope creep’ that happens when other parties push developers to work on functionality that hasn’t been defined as part of the BDD process.

Tester Involvement

From a tester’s perspective, involvement starts far earlier in the project life cycle, in the analysis and specification phase. Typically, in other methodologies, like waterfall, testing was always a back-end task, carried out only when there was a physical product to work on. While everyone recognised that getting testers involved early on was worthwhile, it was difficult to do because testers were always firefighting to cover a backlog of testing just prior to release. BDD encourages, even dictates, that testers bring their analytical talents to bear on the project right at the start.

Product Owners

The product owner is likely to have a clearer understanding of what needs to be done to deliver a piece of functionality. Well spec’d out pieces of functionality that are broken down into more manageable pieces are easier to consume. This makes it easier to estimate and assess the overall complexity and, therefore, to make judgement calls based on a more accurate cost√¢¬Ä¬ìbenefit analysis.

Features Working Sooner

BDD encourages the production and delivery of working features as soon as possible. The fact that developers are working towards getting predefined tests passing encourages the delivery of features that can be demonstrated to the business users early on in the project life cycle. This, of course, assumes that the acceptance criteria and tests have been written having in mind the behaviour of the application from the user’s perspective.

Automated Test Creation

BDD drives the creation of automated tests. In traditional methodologies, automated tests are often an afterthought, only being written once the feature has been developed. BDD places a strong emphasis on creating the automated tests from the start rather than at the end. This leads to more investment in automated tests up front and a larger automated regression pack in the long run.

Automated Test Stability

When more thought is put into the specifications as the start, the automated test are likely to be much more stable. Of course, if the end user keeps changing their mind about what they want, it will be difficult to keep the automated tests in sync with what’s being developed. In general, though, the more stable the specifications, design and code, the easier it is to keep the automated tests running reliably. BDD encourages well-defined specifications and stable design from the start and, therefore, more stable automated tests as well.

Traceability

When tests are developed from the BDD scenarios, they should map back to the executable specifications. This means that traceability between tests and requirements is built in from the start. This provides simple coverage reporting at the requirements level, too. Assuming you have the right tools and setup, the BDD approach to traceability is much easier to implement and manage.

Refactoring

A knock-on benefit of this is that refactoring the code, at any stage in the project, becomes less risky because the automated tests are in place from the start, can be run more frequently, and will likely be more plentiful and have better coverage. If nothing else, the traceability and coverage information will help to show you where refactoring might be less of a risk.

Test Reuse

With the right approach and tools, it’s easy to reuse the test steps in different automation scenarios. This is easier to implement with some software stacks than others. In principle, though, once you’ve automated a step, it can be used in any automation scenario. While the principle is straightforward, in practice it’s a little more difficult to pull off. Your tests need to be well structured and the automation code needs to be neatly compartmentalised. If this is the case, then building a bank of automation scenarios from a selection of steps will deliver significant scalability in your automation capability.

Data-driven Testing

At its heart, BDD has parameterisation and data tables built in. You can leverage this to increase test coverage by driving each scenario with various data sets. Again, test data management in the real world can be quite complex, but if everything is well structured and compartmentalised this is another benefit that can help increase test coverage.

Targeted Tests

If tags are used effectively in the scenarios and specifications, then there is another benefit to be had. This gives you the capability to target test runs on specific functional areas. If your developers need a short, sharp, test run for the area of the application they are currently working on, they’ll be able to readily find the test related to that area. As the number of tests grows and the number of data permutations grows, this ability becomes essential to the development workflow.

Documentation Maintenance

In many test automation setups, analysing failed test results can present one of the biggest challenges. It’s a time-intensive exercise that can often only be completed by one or two people in the team, usually requiring the input of the person who wrote the automated test. This problem arises when there’s no supporting documentation to explain each test, and little to no traceability back to the functionality it is testing. BDD makes interpreting test results and investigating fails far easier because the test case is linked directly back to the executable specification that describes what it does.

With BDD, it is also much easier to keep the specification documents up to date because, having been written in plain English, anyone can update them. When you find you havenít looked at a specification for six months and someone decides that the related functionality needs to change, having understandable and up-to-date documentation counts for a lot.

Cons of BDD

Time Overhead

Creating and maintaining the feature files and scenarios requires an overhead investment of time and effort. In short and/or small projects it may not be worth it, but in projects that have many iterations you’ll definitely see a return on that investment. For smaller, one-off projects, it’s worth weighing up the project lifespan against the upfront investment in the BDD setup.

Retrofitting can be time-consuming

Retrospectively fitting BDD to an existing project can be a time-consuming and difficult process. Should you invest in creating BDD feature files and scenarios for legacy features, or should you just focus on whatís coming next? This would be an important consideration with any specification and testing approach so, while not a drawback related specifically to BDD, itís worth keeping in mind.

Structuring files

Structuring all your feature files, scenarios and executable specifications requires some careful planning. It’s not all that different to structuring a repository of traditional test cases. If you get the structure wrong with traditional test cases, though, there’s little knock-on effect. Getting it wrong in a BDD setup, however, can cause significant headaches for the developer writing the automated test code. It’s worth thinking through this structure up front and refactoring it at the first sign of any issues.

Communication

There needs to be a good amount of communication between the person writing the feature files and the person developing the automation code. The coder needs to accurately interpret these files and the scenarios in order to implement them as automation steps. If there isn’t a mutual understanding about the structure and approach being used, problems will arise as the scenarios become increasingly difficult to turn into working automated tests.

The Given-When-Then definition of scenarios can look deceivingly easy to work with. Although the concept is simple enough to understand, there is some skill required to structure scenarios in a way that facilitates the writing of automation code. You need to be able to simplify your ideas to fit the framework. It’s easy to head off in the wrong direction while thinking you’re on the right track. The trick is spotting when it’s wrong, when it needs adjusting and then committing to refactoring.

Data-driven Testing

BDD’s support for defining test data makes it easier to create data-driven automated test scenarios, but problems arise when you try to execute these tests in test environments that are never left in a known state. Dependencies on external data feeds often cause problems with the execution of the automated tests, too. You could define all of the setup conditions in the Given scenario statements, but this can quickly become complex.

Defining scenarios

The simplicity of the Given statement can lull you into a false sense of security. It’s easy to write a handful of Given statements that seemingly describe the exact condition the system needs to be in before the test steps are executed, but it can be much harder to convert those statements into setup instructions and scripts that really put the system in a known state before the When statements are executed.

Requires participation by all three amigos

BDD relies on the Three Amigos. If one of those amigos isnít quite the friend you thought they were and fails to turn up regularly, then your BDD project is going to suffer. You need buy-in from all three roles in order to create a complete shared understanding. This requires time, effort and commitment on everyone’s part. If your end user role is too busy in the business, then questions related to the business process will go unanswered. If the developer is too busy, then concerns about a particular feature being difficult to implement will not be raised. And if the tester is missing, you won’t get the benefit of analytical questions that identify edge cases and issues that might go unnoticed. Each of the three roles is critical in a BDD team.

Collaborative environment

Depending on the type of project you’re working on, setting up the collaborative environment between the Three Amigos may be easy or somewhat difficult. For a third-party development team, access to the business users might be restricted. An in-house development team may have easy access to end users. If you’re delivering consumer software, you may not be able to talk to any customers at all. Therefore, be aware of the communication channels that exist between these roles in your organisation when planning your BDD setup.

Documentation requirements

BDD does not mean you can do away with swathes of documentation. Just because you’ve created the user stories and described the behaviour of the application doesn’t mean you won’t need design documents or detailed lower-level specification documents. In many complex projects you may well decide that you need all the traditional types of documents as well. In this case, adding BDD documents feels like just another layer of documentation despite all the benefits.

Documentation Maintenance

BDD documentation needs to be maintained. You’ll need to commit ongoing resources (time and effort) to ensure that your BDD documentation is updated and maintained where necessary. Failure to do this will result in your automated tests starting to fail, and at some point you will no longer be able to rely on your automated regression tests.

More time writing automation code

You will need to invest more time and effort in writing the automation code and keeping that code in sync with the steps defined in the BDD scenarios. This will demand developer resources, which tend to be the pinch point for most projects. So, beware of overburdening your developers. Alternatively, you could hire a specialist automation engineer. They will need to be highly skilled because BDD doesn’t lend itself to record and play back script creation.

Interdependence within feedback loop

One of the major advantages of BDD is the feedback loop of: discovery sessions > executable specifications > automated tests > automated test execution > test results linked back to specifications. However, if any part of that loop breaks down you lose out on all the others as well. Each link in the chain depends on all the others. The advantages of BDD can only be realised if every part of the loop is in place and working properly. That takes commitment from all parties involved in the process.

Managing many feature files

The creation of many concise feature files can become a bit of a headache to manage. Good naming conventions and the organised use of tags definitely helps, but things can easily get out of hand. It takes a degree of dedication and frequent refactoring to keep so many files organised and under control. There are a number of tools on the market, like HipTest, that help significantly with this. Therefore, decide up front how you are going to effectively manage the specification files, before they become overwhelming.

There can be a temptation, especially in inexperienced BDD teams, to get sucked in to defining the implementation detail and design aspects of the software during the discovery sessions. This leads to feature files and scenarios that are less about the behaviour of the application and more about its inner workings. As a result, the developers and testers get pulled into developing and testing code for which there is no final business behaviour that can be demonstrated once implementation is complete. The goal of BDD is to implement features that are defined according to behaviours from the end user’s perspective. If you lose that focus, you’ll lose a lot of the benefits of BDD.

Writing tests up front

It’s more difficult to write automated tests up front, before you’ve seen the functionality you’re expected to test. The BDD process involves writing the tests first, have those tests fail, and then continuing development until they pass. Thatís not as easy or as quick as writing a test for an existing piece of functionality you can already see and work with. The BDD approach demands a bit more work to evolve those tests as the application grows.

The development of automated tests is no longer an independent process. While the tests themselves might be defined up front in the feature files, the actual automation code gets created, ideally, just before the feature is developed, or, in many cases, alongside the feature development. As the same developers are often responsible for both the feature and the automation, the independence of the test development can become compromised. Sometimes itís just too easy to tweak the automated test to get it to pass rather than questioning the validity of the test and/or feature.

In order to manage the definition of the features and specifications that are written around them you usually end up defining specifications in small, manageable, chunks. This leads to small and manageable automated tests, too, which is great in many respects, but in other respects not so great. There tends to be a lack of focus on creating end-to-end automated tests that cover integration and system test scenarios. You may have coverage of all the modules and features, but do you have coverage of the full end-to-end business process that the end user will be following?

Conclusion

In this article we’ve tried to present as many characteristics and implications for and against implementing BDD as possible. While there are many success stories of teams that have implemented BDD, there are also many teams that have failed with BDD. More common, though, is that teams succeed and see the benefits to begin with, only to give up after several months. There are many ways to avoid this, but one important way is to understand the issues you’re likely to encounter before you encounter them.

There can be many bumps in the road for teams to overcome as they implement BDD. Being forewarned, as they say, is forearmed. Having listed the many cons that weíve encountered in our BDD engagements, we hope that this list has successfully forewarned you of the difficulties you may encounter. Itís up to you to think through those potential difficulties and work out which of them might impact your project or organisation. From there, youíll be in a good position to plan ways to manage those difficulties before they have the opportunity to derail your well-intentioned BDD efforts.

There’s no doubt though that the mindset shift that comes from following the BDD approach can pay significant dividends over time. The benefits listed in this article will hopefully convince you of that.