Specification by example is a collaborative approach to defining requirements and business-oriented functional tests for software products based on capturing and illustrating requirements using realistic examples instead of abstract statements. Specification by example is also known as example-driven development, executable requirements, acceptance test-driven development (A-TDD), Agile Acceptance Testing or Test-Driven Requirements. I prefer A-TDD but will use both specification by example and A-TDD.
Examples as a single source of truth
A key aspect of specification by example is creating a single source of truth about required changes or new functionality from all perspectives. When business analysts work on their own documents, software developers maintain their own documentation and testers maintain a separate set of functional tests, software delivery effectiveness is significantly reduced by the need to constantly coordinate and synchronize those different versions of the truth.
With Specification by example, different roles participate in creating a single source of truth that captures everyone’s understanding. Examples are used to provide clarity and precision so that the same information can be used both as a specification, documentation and a business-oriented functional test. Any additional information discovered during development or delivery, such as clarification of functional gaps, missing or incomplete requirements or additional tests, is added to this single source of truth. As there is only one source of truth about the functionality, there is no need for coordination, translation, and interpretation of knowledge inside the delivery cycle.
When applied to required changes, a refined set of examples is effectively a specification and a business-oriented test for acceptance of software functionality. After the change is implemented, specification with examples becomes a document explaining existing functionality. As the validation of such documents is automated, when they are validated frequently, such documents are a reliable source of information on business functionality of underlying software.
Specification by example is very useful to projects with sufficient organizational and domain complexity to cause problems in understanding or communicating requirements from a business domain perspective. It does not apply to purely technical problems or where the key complexity is not in understanding or communicating knowledge. There are documented usages of this approach in domains including investment banking, financial trading, insurance, airline ticket reservation, online gaming and price comparison.
This brings me to my very successful experience with A-TDD within an actuarial modeling project. To be specific to the creation of a new MCEV model for a life insurance company.
MCEV Modelling
The Embedded Value (EV) of a life insurance company is the present value of future profits plus adjusted net asset value. It is a construct from the field of actuarial science which allows insurance companies to be valued. European embedded value (EEV) is a variation of EV which was set up by the CFO Forum which allows for a more formalized method of choosing the parameters and doing the calculations, to enable greater transparency and comparability. Market Consistent Embedded Value (MCEV) is a more generalized methodology, of which EEV is one example.
Depending on how you implement such a model one part of it is the future cash flow generation component. Input for this component is a coded feature vector of an insured person and his/her contract. Based on this information the component computes the expected cash flows for this insured person for t=0 (now) until t=40 (40 years from now). Very simplified you could say that the cashflow component computes premiums, benefits and total savings for each person in any given year. Afterward an ALS component computes Profit/Loss Statements and Balance sheets based on given Assets, Liabilities (the cashflows) and Scenarios (interest rates, market behavior, company behavior etc).
As you might already have noticed the cashflow component seems to be representable by input data, model, expected output data. This is exactly what we have done. We create one table containing input examples. We expressed business rules and model in tables to make them more comprehensible and assists in finding missing cases. And we defined an expected output table with one row for each input example. Based on this the model could be coded in in the modeling software of choice and tested automatically.
| PersonID | Date of Birth | Gender | Product | Start Date | Current Savings |
| 1 | |||||
| 2 | |||||
| 3 | |||||
| 4 | |||||
| 5 | |||||
| 6 |
| PersonID | Premium t0 | Benefits t0 | Savings t0 | … | Premium t1 |
| 1 | |||||
| 2 | |||||
| 3 | |||||
| 4 | |||||
| 5 | |||||
| 6 |
Implementation
So how did we got those tables? Since we used Scrum at this project we mapped the A-TDD steps on a Scrum iteration as described by the guys from LeSS.
Discuss in workshop – Before the detailed Sprint Planning, the team, Product Owner, and other stakeholders clarify the requirements collaboratively in a workshop.
Develop in concurrence – Tasks for implementing the tests/requirements are created in the detailed Sprint Planning and implemented during the iteration. All activities happen “at about the same time.”
Deliver for acceptance – The working product increment—the passing acceptance tests—are delivered for acceptance to stakeholders and discussed together in the Sprint Review.
Automation
Successful application of A-TDD on larger scale projects requires frequent validation of software functionality against a large set of examples (tests). In practice, this requires tests based on examples to be automated. A common approach is to automate the tests but keep examples in a form readable and accessible to non-technical and technical team members, keeping the examples as a single source of truth. We did this by using Excel for our Examples and SharePoint for versioning.
This process is supported by most test automation tools which work with tests divided into two aspects – the specification and the automation layer. The specification of a test which is in this case a CSV file and contains the examples, their results and auxiliary descriptions. The automation layer connects the example to a software system under test. When you combine this with a Continuous Integration tool that runs all tests with each build of your software/model you will improve the quality and potential delivery speed of your project a lot.
Conclusion
At the time I moved on to another project we had not automated the whole process 100% but came close. The whole process of A-TDD was received very positively by all team members. We got great cross learning and understanding between model coders, business analysts, actuaries and DB specialists. Besides that the next audit will be much easier since this kind of documentation is up-to-date, clear and detailed. Also regression testing of the model has become far much easier. So I can only recommend starting Specification by Example when you can code (parts of) your business in tables. It is definitely worth the effort.
