Test-Driven Development At HR SoftCo
HR SoftCo use Scrum as a macro-process for software development. Scrum does not describe a day-to-day development process, however, so HR SoftCo has implemented Test-Driven Development (TDD) as outlined by Kent Beck. Test-Driven development forms part of the Extreme Programming process that is also used at HR SoftCo.
This is not an introduction to TDD. It is information about how we have implemented TDD for our global team. For basic information about TDD, please see http://en.wikipedia.org/wiki/Test-driven_development.
The important TDD processes are:
- Simultaneous QA Development
- Unit testing
- Acceptance testing
- GUI testing
- Manual testing
- Continuous integration
The specialist tools used are:
- MbUnit
- Fitnesse
- SWAT
- Cruise Control .Net
1 Why Did We Start TDD?
There were several factors involved here. The office I work in was once a small UK company. When HR SoftCo (a large US company) took over they were in the process of converting all of their teams to use Agile techniques. Our team already had an interest in Test-Driven Development, but had made no inroads to developing it further. We had all the usual waterfall related issues such as quality, knowledge distribution, late delivery, building the wrong software etc. With the support of HR SoftCo we were able to implement completely new processes and transform our work.
2 What Is Our Team Structure?
We have a classic small, cross-disciplinary team. The members are:
- 1 Scrum Master
- 2 QA (not enough!)
- 7 Developers
- 1 Product Owner
To make up the short fall in QA, everyone else helps out as we need to write / execute tests etc.
Outside of our team there are over a hundred other developers working in other Scrum Teams. We use a "Scrum Of Scrums" type system to manage communication between teams.
3 Simultaneous QA Development
The most important part of the TDD process is the continuous interaction between QA and developers. HR SoftCo has a strict procedure when a developer starts a new User Story:
- Mark the User Story with the developer and QA’s initials
- Mark the User Story status as “Started”
- Discuss and agree testing requirements with QA
The discussion with QA is critical to the development process. This discussion may take a few minutes if the development work is “business as usual” or could take more than an hour if new areas are being explored. The key decisions made during this discussion are:
- What types of testing are appropriate?
- Who will do the testing?
- How much effort is required for the testing?
This discussion is only the first step in the process. During development of the software the developer and QA team discuss progress and execute tests regularly. “Regularly” means whenever significant progress or changes have been made. In practice, the low current low level of QA resource on the team means that QA resource is sometimes not available until towards the end of the development process.
4 Unit Testing
Our team policy is to achieve 100% coverage of new code for unit testing. Where the code is completely new this goal is usually achieved easily. Where we are working with legacy code it is frequently not economically viable.
In a couple of the legacy web applications at HR SoftCo, there is no real business logic layer (although there is a data layer). This means that the business logic code is frequently in very large functions contained within the “code-behind” of forms (for ASP.Net). It would be possible to refactor this code into business logic classes to make it unit-testable, however this would take a very large amount of effort. There is agreement between the development team and the business that this would be a poor use of resources.
The key unit testing tool is MbUnit. We also use NCover to measure the coverage of the unit tests. We chose MbUnit at the recommendation of HR SoftCo. We did not research other tools such as NUnit or Team Foundation Server. At the time of implementing TDD we were also implementing Scrum, Extreme Programming, new servers etc and it was a blessing to have the advice of experienced teams when choosing technologies!
Developers run unit tests on the code they are developing frequently (every few minutes). Ideally the developers run the entire unit testing suite before checking code in to the source control repository. However, as the number of unit tests increases this becomes more time consuming and less efficient. The team has a Continuous Integration infrastructure (see below) to deal with this.
5 Acceptance Testing
Our long-term goal is to have our Product Owner devise automated tests as part of the requirements specification process. This does not involve writing code, but Fit tables of information about how code should behave.
For example, the HR SoftCo Performance Management product calculates a Total Performance Score based on a complex set of calculations. It is possible to construct a spreadsheet of the various inputs into the calculation and the expected output. The QA team can then write the necessary test ficture code to sit around the code written by developers.
At the moment this does not happen. The developers and QA tend to write these Fit tests between themselves. As the Product Owner becomes more experienced we hope he will take a more active role in the testing process.
This is the most under-used type of testing on the team at the moment mostly due to the amount of work that goes on with legacy products where it is very difficult to refactor the products into a testable state (see Unit Testing above). Developers and QA can run the Acceptance Tests at will. For work to be “Done” (“see Scrum At HR SoftCo”), all of the Acceptance Tests for the entire product must pass. Developers run Acceptance Tests as they develop code. When the Acceptance Tests all pass, the code is complete.
6 GUI Testing
The team engages in a large amount of GUI testing using the open source SWAT product. Although coverage on SWAT tests is poor, it is frequently the only economic method for testing legacy code (and changes to legacy code). SWAT is a web-based GUI testing tool based around Fitnesse.
Our GUI testing suite currently takes around six hours to execute, and it is therefore not included as part of our definition of “Done”, nor are these tests regularly executed by developers. There is a daily overnight execution of these tests. QA take responsibility for monitoring the results of these tests and reporting any breaks for fixing.
7 Manual Testing
There are times, especially on legacy code, where it is not possible to implement automated tests. A recent example was referred to above when the team was required to dispatch automated emails when certain events occurred in the HR SoftCo products. The GUI tests to be implemented were checking that emails were sent and worded correctly. Unfortunately, email is not always delivered in a timely fashion and it is not always easy to read emails that are dispatched.
On these occasions the team falls back to old fashioned manual test scripts. These scripts identify the test cases (usually devised by the Product Owner) and then relate the test cases to a sequence of actions to execute the test case. These scripts are written and executed by QA. Developers are kept up to date as changes are made to the scripts so that they can run the tests whilst developing the software to pass the tests. The large amount of work involved in executing manual scripts means that they are not executed regularly, but only in order to pass work as “Done” and just before release.
8 Continuous Integration
Continuous Integration (CI) brings most of the tests together. This ensures that the products hang together and that work done in one area does not break another area.
The team has an hourly build that:
- Gets the latest source code for products and tests
- Build the products and tests
- Executes the tests
The tests included are Unit Tests and Acceptance Tests. Cruise Control is employed to execute Nant scripts that take care of this.
The CI build produces a simple pass / fail light that the entire team monitors. As soon as a break is detected at least one team member stops work and fixes the build (usually the person who broke it). A broken build is ALWAYS the team’s highest priority.
The CI build also produces a report on the coverage of Unit Tests so that it is possible to see at a glance if the 100% coverage target is being achieved.
9 Related Topics
This paper has discussed how our team at HR SoftCo has implemented the Test Driven Development process. Our other activities will be discussed in the following articles:
- Scrum At HR SoftCo
- Automation At HR SoftCo
- Extreme Programming At HR SoftCo