How would you know if the work done by your business analyst results in value? Perhaps one of the most difficult questions to answer, measuring the amount of value that projects deliver helps determine success. But how is this done? There are various ways of identifying the business value delivered by a product.
Agile business analysis typically uses story points to measure value. Although one of the quickest way for business requirements prioritization, accumulated story points used as a measure to determine project worth may fall short. That is why, other methods such as return on investment (ROI), internal rate of return (IRR) and net present value (NPV) are used. Other qualitative approaches are also recommended such as checking a product’s commercial, market, efficiency, customer and future value.
Now, to identify if your agile business analysis process is paying off, there are a couple of signs to look out for; there is also a method to systematically measure the amount of value produced by your IT business analysis endeavors.
Check customer behavior
Perhaps one of the most important measurements of business value is the rate of which a product helps a customer. This is evident on the willingness of new and existing customers to pay for a product or service; or a premium for new functionalities or software version. Off the top of head, these are good indicators that your Business Analyst (BA) captures the need of your market.
If you feel the need to systematically measure this, we can do what Eric Ries suggests: track your funnel metrics and do a cohort analysis. Funnel metrics refer to tracking customer behaviors that are critical to a company’s engine of growth. This could pertain to customer registration, application download, trial, repeat usage, and purchase. Once enough data is gathered, you can do a series of tests to know if the new functionality and consequently, your IT business analysis is performing as expected. This could be done through a cohort analysis. Rather than looking at the cumulative totals or gross number of your funnel metrics, a cohort will make you look at the performance of each group of customers that encounter the product independently. Also known as a cohort graph, it shows the performance of each group of customers such as the ones who registered but did not logged-in, the ones who logged-in but did not actively used the product, etc.
Check product efficiency
Another way to know if your agile business analysis efforts are paying off is by checking product efficiency. This pertains to how much reduction in terms of errors per task, your product is getting since doing an update. Another option is by checking how much response time has improved or if application crashes were diminished. These metrics will help you know if your IT business analysis works and if the requirements gathering tools used by your BA are worth-it. To scientifically measure the quality of your software update, let’s use this formula recommended by Scrum Alliance. Introduced by Harish Vasista, he says that this metric is useful to keep track of the defects reported by the customer per release. In addition, Vasista says ” this can be a negative value depending on the number of defects reported by the customer. Care must be taken to include only the defects that are relevant to the product. There will be cases where customer-reported defects may not be directly relevant to the product developed (examples: fiber cuts, power outages, problems due to other elements in the network, etc.)“. Per Vasista, a defect can be measured as:
Defect containment (DC) = [1 – (Dc/Di)] * 100%
Where:
Dc = Number of P1 and P2 defects detected by customers for a release
Di = Total number of P1 and P2 defects detected internally
Check Net Present Value
According to Alex Brown of Scrum Inc., although admittedly, there are a lot of ways to measure business value, using the Net Present Value (NPV) method works best. Anchored on cash flow analysis, NPV is known as the “time value of money”. This means that a dollar spent on your agile business analysis today is worth more than a dollar tomorrow. The same IT business analysis methodology is promoted by the Project Management Institute (PMI), the de facto standard certification for project management.
Using NPV is good to know if your IT business analysis helps maximize the value of your product. Scrum Inc. suggests calculating the NPV on the Epic level, using story points. The suggested formula can be seen in this presentation.
Using NPV for an agile project not only helps the business analyst to work on requirements with the highest value but also it encourages the team to see requirements from the perspective of its customers, helping deliver products with better features.
There are various signs that project managers can look at to know if their BAs are creating requirements that truly maximizes the value of the product they are trying to deliver. Although there are straightforward signs such as customer behavior, it is also useful to use metrics such as defect containment and NPV. Eric Ries’ funnel metrics and cohort analysis are also helpful to create a more scientific view of customer behavior that is also useful on incremental work unique to agile business analysis.
