Risk is the possibility of a negative or undesirable outcome or event to occur. A specific risk is any problem that might occur, that would decrease customer, user, participant, or stakeholder perceptions of product quality or project success.
In testing there are two main types of risks. The first type is product or quality risks. When the primary effect of a potential problem is on the quality of the product itself, the potential problem is called a product risk. An example of a quality risk is: a possible reliability defect that could cause a system to crash during normal operation. The second type of risk is project or planning risks. When the primary effect of a potential problem is on the overall success of a project, those potential problems are called project risks. An example of a project risk is a possible staffing shortage that could delay completion of a project.
Not all risks are equal in importance. There are a number of ways to classify the level of risk. The simplest way is to look at two factors:
- The likelihood of the problem occurring
- The impact of the problem occurring
Likelihood of a problem arises primarily from technical considerations, such as the programming languages used, the bandwidth of connections, and so on. The impact of a problem arises from business considerations, such as the financial loss the business will suffer, the number of users or customers affected, etc.
In risk-based testing, the risk items identified during risk analysis, together with the level of risk associated with each risk item, are used to guide the testing. In fact, under a true analytical risk-based testing strategy, risk is the primary basis of testing.
Risk can guide testing in various ways, but there are three very common ones:
- First, product quality risks are identified, test activities are scheduled, test techniques are chosen and effort is allocated for each quality risk item proportional to the level of risk.
- Second, testing sessions are managed to provide risk mitigation and contingency.
- Third, test results and project status are reported in terms of residual risks.
Risk-Based Testing
Risk management includes three primary activities:
- Risk identification – what are the project and quality risks for the project
- Risk analysis, assessing the level of risk – typically based on likelihood and impact – for each identified risk item
- Risk mitigation – more properly called “risk control” because it consists of mitigation, contingency, transference and acceptance actions for various risks
These activities are sequential when they start. They are staged such that risk identification starts first. Risk analysis comes next. Risk control starts once we have determined the level of risk through risk analysis. However, because risk in a project should continuously be managed, risk identification, risk analysis and risk control are all recurring activities.
Because, everyone has their own perspective on how to manage risks on a project, including what the risks are, the level of risk, and the appropriate controls, risk management should include all project stakeholders.
For proper risk-based testing, identification of both product and project risks is needed. Both kinds of risks can be identified using techniques like:
- Expert interviews
- Independent assessments
- Use of risk templates
- Project retrospectives
- Risk workshops and brainstorming
- Checklists
- Past experience
- Special techniques (FMEA, Hazard Analysis)
In informal techniques, risk identification stops at the risk items. The risk items must be specific enough to allow for analysis and assessment of each one to yield an unambiguous likelihood rating and an unambiguous impact rating.
More formal techniques often look “downstream” to identify potential effects of the risk item if it becomes an actual negative outcome. These effects include effects on the system, as well as on potential users, customers, stakeholders, and even society in general. Failure Mode and Effect Analysis is an example of such a formal risk management technique, and it is commonly used on safety-critical and embedded systems. Other formal techniques look “upstream” to identify the source of the risk. Hazard analysis is an example of such a formal risk management technique.
Risk analysis or risk assessment is used to determine the level of risk. This determination often involves likelihood and impact as the two key factors.
The level of risk can be determined quantitatively or qualitatively. In quantitative risk analysis, numerical ratings for both likelihood and impact are used. Likelihood is a percentage and impact is often a monetary quantity. If the two values are multiplied, the expected payout or expected loss is obtained. The qualitative approach can also be used, but it depends on the project.
Unless the risk analysis is based on extensive and statistically valid risk data, the risk analysis will reflect perceived likelihood and impact. In other words, personal perceptions and opinions held by the stakeholders will determine the level of risk. That is why project managers, programmers, users, business analysts, architects and testers will have different perceptions and possibly different opinions on the level of risk for each risk item. To avoid the possibility of disagreements between stakeholders, the risk analysis process should include some way of reaching consensus.
Part of any management role, including test management, is controlling risks that affect the area of interest. How can risks be controlled? There are four main options for risk control:
- Mitigation, where preventive measures are taken to reduce the likelihood and/or the impact of a risk.
- Contingency, where a plan or perhaps multiple plans are developed to reduce the impact if the risk becomes an actuality.
- Transference, where another party has to accept the consequences of a risk.
- Finally, the risk and its consequences are ignored or accepted.
For any given risk item, selecting one or more of these options creates its own set of benefits and opportunities as well as costs and, potentially, additional risks associated with each option. Done wrong, risk control can make things worse, not better.
Analytical risk-based testing is focused on creating risk mitigation opportunities for the test team, especially for quality risks. Risk-based testing mitigates quality risks through testing throughout the entire lifecycle.
Post written by Cristina.C while attending ISTQB – Advanced Test Manager course.
Source: “Advanced Software Testing Vol. 2: Guide to the ISTQB Advanced Certification as an Advanced Test Manager” by Rex Black
