Updated: May 30
The product backlog is where you prioritise your software development activities. Security and compliance are one priority, competing with reliability (bug fixes), and feature enhancements.
Software startup companies go through a series of transitions and increments in their practice maturity. One of those areas is in relation to how they determine the relative priority of its Engineering team or developer’s time. Determining which items on the product backlog are most important; this includes a balancing act between new features, improving existing functions, fixing bugs and non-functional requirements (NFRs). NFRs include security, reliability and scalability considerations to how the product is built.
In the earlier stages, the priority of change requests is more subjective. If there’s a product owner, or the CTO or CEO, will decide what they believe has the most impact on landing new sales, retaining valuable customers and creating overall product (and business) value. The greatest risk at the start of a company, is that the product doesn’t fit the market need or that the company will run out of funds pursuing traction in the market.
The priority of NFRs naturally falls behind the functional aspects of the product that bring in customers and funds in the shorter term. NFR’s often aren’t as significant or pressing, when there’s a smaller volume of users. At some point, these NFRs need to be made a priority. Once sensitive data is being collected, and particularly when there’s enterprise customers with a higher volume of users, the security, reliability and scalability of the platform are critical to long term company success.
The challenge with balancing priorities for functional and non-functional aspects of the system, is that they aren’t directly comparable. Sales and marketing teams and prospective customers favour flashy new features. Existing customers, and operational teams responsible for maintaining the system, resolving bugs and incidents, and addressing security requirements, will always favour NFRs.
So how do you implement a more measured approach? When is the right time to implement a formal process?
There’s broadly four main approaches, which may be used in combination or individually, to balance these priorities:
1. Product backlog ownership
In this approach, there's an owner of the backlog that is responsible for balancing multiple perspectives; including functional and non-functional. In order for this to work successfully the product owner's incentives need to be appropriately aligned. It’s common to see incentives geared towards sales outcomes or with product owners reporting to a CEO that has more of a sales focus. In that case, NFRs will naturally take a back seat role. If there’s performance metrics to account for NFR’s, or the product owner reports to others like the CTO, CSO/CIO, COO, it's likely to be more balanced.
2. Defined (scoring) criteria
Defined criteria are often used in incident, risk and vulnerability management. It's a way to define what constitutes a priority, urgency or rating system to determine the actions and timeline that follows. In the case of a product backlog, the scales are often based on the number of customers and the level of impact. When it comes to fixing product bugs; the more customers and the higher the impact of the bug, the higher the priority. On the flip side for new features, the more customers it will benefit (often with some preference to winning new customers) and the greater the benefit to customers, the higher the priority.
This approach is problematic for security and reliability focused NFRs that can't be measured against this criteria reliably as the impact is unknown. This can be addressed through a separate "dimension" or scale of the criteria that considers potential risks and the impacts or the longer term benefits of investing in NFRs. In the end, the rating needs to be directly comparable across all backlog items. It should recognise the importance of NFRs by the way it's defined.
3. Service level agreements
An SLA approach is where NFRs like security vulnerabilities have a maximum time period in which they need to be addressed. It's often based on their severity rating (eg. High, Med, Low). The use of SLAs may be required if the above approaches are insufficient, or just used as a way of simplifying the focus on NFRs. It takes out most of the judgement. This timeline still allows product owners to juggle other priorities to an extent but with an almost forced commitment to address NFRs in a reasonable timeframe.
4. Percentage allocation
The final approach is a slight variation on the last two. Instead of a direct comparison through criteria, or a defined timeline, it delineates the time spent on functional vs. non-functional requirements. For instance, 20% of each sprint may be dedicated to NFRs. Separately within the 80% and 20% allocations, the priorities of individual items are determined for what will make it into each sprint, while retaining those two backlogs separately for the purposes of prioritisation. This may work well where there is a product owner geared towards sales and functional requirements, and a security and operations team with an NFR focus. The main downside to this approach is that NFRs in the backlog can fluctuate in importance over time, and 20% (or whatever the allocation is) may be too little or too much in any given sprint.
The SOC 2 Perspective
This control area is often only represented in the Change Management section of the SOC 2 report (Common Criteria 8.1 below). However, it plays an important role in the linkage between identification of security vulnerabilities, correct action to remediate those vulnerabilities and implementation of ongoing security improvements to meet the company objectives (that should include security-related objectives).
Common Criteria 7.1: To meet its objectives, the entity uses detection and monitoring procedures to identify (1) changes to configurations that result in the introduction of new vulnerabilities, and (2) susceptibilities to newly discovered vulnerabilities.
Common Criteria 7.3: The entity evaluates security events to determine whether they could or have resulted in a failure of the entity to meet its objectives (security incidents) and, if so, takes actions to prevent or address such failures.
Common Criteria 8.1: The entity authorises, designs, develops or acquires, configures, documents, tests, approves, and implements changes to infrastructure, data, software, and procedures to meet its objectives.
AssuranceLab's Best Practices Series
AssuranceLab's best practices series, is about highlighting the "real operational benefits" that come from effective control practices. At best, they support your company culture, provide structure and clarity, and enable scalable growth. At worst, they tick the box of what your customers expect, reduce the reactive "firefighting" and time-wasting, and help you demonstrate your compliance with leading standards like SOC 2 and ISO 27001.