The Product Manager and the Agile Testing Strategy

On an Agile project, the application is being built incrementally with every sprint adding to the functionality. As a result of this approach, Developers are constantly adding to the codebase impacting the existing functionality. While Agile software development ensures faster time to market and greater transparency for the stakeholders, it also stands for finding defects earlier in the cycle and raises the bar for quality. Agile testing requires QA engineers on the Agile teams to test quickly, incrementally, and contextually while staying within acceptable defect limits. In this scenario, an Agile QA engineer employs "smart" testing strategies and tactics to meet the demands of an Agile project.

Smart Testing

Smart testing requires the application to be tested incrementally - meaning while the user story is being coded, the QA engineer plans for the test cases and scenarios in the immediate vicinity of the story and tests the application as soon as the developer is done coding. Individual user stories are tested in this manner and when the build is created for the sprint scope then QA accepts the build, regresses the system, and skims over the user stories of the sprint with greater focus on the impact on the surrounding areas of the functionality delivered in the sprint.

As you can well extrapolate, this approach can add to the quantum of build level testing that is done in successive sprints. To optimize on the testing scope vs effort, another facet of smart testing, which is testing prioritized system workflows is undertaken. Testing in this prioritized order while optimizing the effort does introduce some risk which can be handled to some extent by doing adhoc testing around the major workflows.

Product Manager's Contribution

The Product Manager's contribution to the smart testing approach is to validate the workflows that are being tested and ensure that the tests cover features that are indeed the major workflows at the customer end. This validation begins with the review of the Test Plan that QA comes up with at the end of the release planning or sprint 0. The validation continues on during the iteration while reviewing the scope of coverage of the test cases at the user story level. The validation concludes when looking through the prioritized list of major workflows that QA plans to test as the functionality scope increases. 

Additionally, during acceptance testing, the Product Manager can take into cognizance the scope of QA testing and review the product against the acceptance criteria. 

The Product Manager is able to collaborate with QA on the testing strategy by drawing on his/her knowledge of: how customers use the product, features that pre-Sales demo to prospects, the features that are obligatory requirements, and those user stories that effectively out-pitch the competition and find favor with analysts.

In case of a complex product, it may not be possible for a Product Manager to be involved with the Agile teams everyday. In this scenario, the Product Manager's contribution can be scaled by taking the help of a proxy Product Owner who works closely with the Agile teams.

In this way, a Product Manager can ensure that the Product is being built right by having a stake, albeit minor, in the testing strategy of Agile teams.

Challenges in conducting Agile Assessments

Some challenges that the Agile Center of Excellence could encounter when conducting Agile Assessments, including suggestions to move past the challenges.

1. When teams look at the assessment as an audit, there could be cover-up and manipulation. An Agile CoE can counter this by stressing on the role of the assessor as a Coach and not as an auditor. For his/her part the assessor should work with the team towards implementing the identified Agile areas of improvement so the assessment is a measure of collaborative effort on the part of the assessor-Coach and Agile team.

2. When a single Agile coach has to cater to the assessments of several teams, it spreads the coach too thin. I dont have a golden number which limits the team-coach ratio, but íf there is a situation where one coach has to work with over 4 teams, then the coach could have the Scrum Masters intervene on their behalf. This approach can help mentor Scrum Masters who can mature into coaches. The flip side would be the added delay and loss in transmission through the Scrum Master to the Agile teams.

3. Continuing on handling the coach-team bandwidth challenge and coupled with coach succession challenges, another approach could be to have a team of Agile assessors essentially members of CoE who are Scrum Masters do the assessments. This approach could help ease a single Coach's onus and build assessment abilities in other scrum masters or coaches.

The Assessor-Scrum Masters begin by observing the team in their work environment and ceremonies, followed by an assessment, and concluding with an action plan as agreed with the Scrum Master of the team being assessed.

The Assessor-Coach should stay with the team for at least 3 to 5 assessments so that they get an opportunity to implement Agile ideas and report to the CoE about the plans ahead for each team so that the expectations from the organization are met.

Agile CoE - Getting Started

In my role as a Program Manager I have played the role of a Change Agent at roll-out and post roll-out stages of Agile practice in an organization. I hope to share my experience in multiple posts on various aspects of setting up and managing Agile programs. This post talks about setting up a team of agilists who will sustain Agile, called CoE in common parlance.

After organizations have crossed executive buy-ins and Agile roll-outs across teams, sustaining and enhancing the Agile practice is the next milestone in an organization's journey. A team of agilists who are passionate about implementing agility and look for ways to sustain agility in a demanding work environment can step in to sustain and enhance the Agile practice.

One approach is to invite volunteers from across the organization who are passionate about Agile and don’t need an external push to take on this role of developing the Agile competency. I say developing the competency because there is a constant demand to derive an Agile approach to situations and issues that the Agile teams encounter. Failing to have a go-to forum for Agile queries could lead to teams slipping back to pre-Agile ways. Besides Agile may be blamed for the current issue and a general suspicion towards adopting Agile practices may ensue.

The disadvantage with a volunteer group of agilists is the fact that they are a virtual team moonlighting for Agile. When pressures from their primary role mount then they may not be able to sustain their energy towards the voluntary Agile evangelist role. Another reason could be the “whats-in-it-for-me” – when the entire Agile evangelism is considered pro bono by the organization. Third could be recognition and acceptance of intervention in Agile teams – which could be low in the absence of an organizational recognition of the team of volunteers.

The second approach is to put together a formal team of Agilists who are tasked with sustaining and developing the Agile practice. This team could come up with a creative moniker to move away from any negative connotations of Center of Excellence or Capability Center or Competency Center. Again a frivolous name in a formal culture could water down the effort of the formal team. So depending on the approach that would give greater acceptance, the formal team could choose its name.

For the purpose of this post and others related to the same topic I’d like to stick with CoE since this term seems to be commonly recognized.

Who comprises the Agile CoE?

At least one full-time member, Scrum Masters, and Product Owners

What should the CoE work on?

• Self-driven approach
• Engage in a visioning exercise that results in goals and roadmap for the CoE
• Inspect and adapt

What activities does a CoE undertake?

• Sharing learning and success stories
• Training or co-ordinating sessions on processes and tools
• Connecting with the Agile Community
• Evangelizing through active blogging and creative meetups
• Hands on coaching
• Measuring the value added by the CoE

How long should the CoE exist?

Until the organization believes that there is more to be done in the course of Agility, which I think is a continuous journey of incremental improvements

Getting Agile to Middle Managers

Middle managers are an essential layer in an organization; this layer comprises of leads, and managers who have greater proximity to the development teams and are responsible for project delivery. The middle management layer provides an incubation ground for senior management equipping managers with essential leadership skills for greater responsibilities. The Agile transition should include in its ambit, the ambitions and insecurities of middle managers  to ensure that the Agile rollout is successful, sustained, and enhanced to higher levels of productivity. Following are four areas which require an Agile flavor to win the support of middle management.

• Career progression – With Agile being an equalizer and tilting the balance in favor of the “doers” the key performance criteria of Lead and above, across disciplines has to be re-evaluated since tracking and assignment are no longer in the purview of managers. Managers and Leads may be threatened by the perceived power of Product Owners and the importance of Scrum Masters. Their responsibilities need to be recast into that of a coach and mentor, they could be encouraged to get back in touch with a hands-on approach, or perhaps take on a strategic role that determines the course of the development team.

• Clarity of role - Traditional responsibilities may overlap with the new SCRUM roles, Scrum Master and Product Owner. Some managers may fall back to the traditional way of managing teams and this could led to conflicts with the persons donning the Scrum Master and Product Owner roles. As part of the Agile practice, managers should have a forum to discuss the areas of conflict and resolve it in a manner that promotes Agile. Development teams may also require clarity on the roles so they can approach the right roles for support.

• Grooming development teams - Middle managers are also vested with the responsibility of reviewing and guiding the career progression of development teams. With Agile, Managers must remove extra status meetings and stay tuned to the team and individual's progress through daily standup, and information radiators. This being an indirect method could be a challenge for managers to assess and guide development teams. As part of the Agile Transition, middle managers should discuss with the Agile evangelists ways of evaluating individuals without jeopardizing the team collaboration.

• Frame of mind - Agile calls for middle managers, among other agilists, to adopt servant leadership which involves selfless service to build the development team. The middle management team needs to be coached and supported along their servant leadership journey.

Career progression, clarity of role, grooming development teams, and frame of mind are by no means an exhaustive list of areas that would require intervention during Agile Transformation. However these four areas are the bulk of the Agile Transformation plan for middle managers.

My Top 3 Areas that Product Managers Adapt with Agile

In a product development organization, product managers are placed at the center of the productizing process. In this capacity a product manager strategizes the product's path to development culminating in bringing value to customers and profitability to stakeholders and business owners. When development teams adopt Agile, the Product Management team should also be brought under the influence of Agile values and principles because the Agile team is dependent on Product Managers for direction with respect to "what to build."

Here are my top three areas which Product Managers must adapt to flourish in an Agile environment:

Roadmaps – Product Managers own roadmaps and this artifact in a Agile environment needs to be reviewed with the Agile team at the end of every cadence (release) so the team can align itself with the next prioritized item and at the beginning of every cadence in order to set priorities. For their part, Product Managers need to be aware of the release cycles of Agile teams and ensure  that the market research and other activities leading to executive buy-in and budget approvals are in time for the Agile team to start working on the release.

Increased Transparency -  User stories are based on the value that it brings to the user. This value is explicitly captured in every user story after the Product Manager talks with customers and analysts, and researches the market factors. Every person on the Agile team is encouraged to question the value of the user story which requires the Product Manager to have a granular understanding of the impact of a user story on customers and the market. 

Sales and Marketing Interactions – A Product Manager needs to train marketing and sales teams through demos, documents, and conversations on the products and its features so that there is effective messaging and lead conversion. In an Agile world, the Product Manager can prep the sales and marketing teams with confidence since he is aware of the nearly releasable features that are being churned out by the team sprint on sprint. 

Presented this topic at Agile Tour 2012, Chennai, India

Innovation Games Checklist for Distributed Teams

Image Courtesy: http://www.freeimages.co.uk

Use of innovation games in Agile teams and their ceremonies is all the rage currently. While innovation games are fantastic for bringing out the best possible results for co-located teams, they pose a challenge and perhaps an irritant for distributed teams. To address these challenges, thanks to fellow Agilists I found checklists and tools (like corkboard.me, video conferencing) that can ease the pain of facilitating the innovation game.

However if you find yourself in a situation where the remote members of a distributed team are available on teleconferencing only, then perhaps my learnings can help you tide over:

• Send a copy of the prop that you will be using with the local team, to the remote team. For example, if you are playing the Sailboat, then send the picture with the instructions ahead of the meeting so the remote team members can come prepared, and refer the instructions as you move forward with the game.
• Ensure that the remote team members have access to the prop and material like post-its, markers. If required, enlist the help of one of the remote team members to organize the material.
• In the local room, move the action area around the phone so its easier for the team to huddle and speak into the phone
• Get a list of names of the remote team members. Address each of them by name throughout the session to keep them more involved
• When you issue an instruction to the local team have the remote team do the same step
• When sharing the outcome of the exercise, have the remote team members talk through their comments. Have a scribe jot down comments of remote team members  and post them on their behalf.
• In addition to performing the activity, for example sticking the post-its around the positive winds area of the sailboat, have the local team members also announce or read out their post-it notes over the phone
• Before voting, group options so its easier to call out and have the remote members write down the options before they vote on them
• Ensure that a commentary of the happenings in the local room is ongoing - reminiscent of a bygone era of the radio :)
• Have the local and remote team members take a picture of the end result of their innovation game and share it
• Have a scribe note the action items and any other details from each location, merge, and circulate to all

Happy gaming!

Leveraging User Ecosystem for New Products

Here is a collection of opportunities that I have leveraged while trying to build a product that catered to target user requirements. Each project included leveraging one or more of these opportunities.

• While getting started:
• Build target personas - for a new product, you may not have existing customers whose input can validate feature sets, but what you do have as a product manager is the business goal which would illustrate the market segment. The users who comprise the target market segment can be developed into target personas around which workflows and features can be built.
• Research analyst reports and domain publications - to understand trends and players and   details on the facts and figures based on which you could understand the market in which you want to play.
• Network with field agents - Talk with colleagues who are Solution Consultants, Sales Engineers, Professional Services for input on what the market is asking especially if you are building a product that is related to the existing product portfolio. Strike up a chat, even one at the water cooler could be quite informative.
• Contact target users - Interview or survey target users to understand how they are currently meeting the requirement that your product aims to resolve. Use this input to align  product workflows with mental maps to shorten the learning curve for users and increase product acceptance. 
• During product development:
• Regular live demos - Build the product incrementally and demo the increments to channel partners, Solution Consultants and thought leaders to elicit feedback. Build a feedback loop into the development process so the team can incorporate the feedback with minimum disruptions and the provider of feedback feels heard and valued.
• Host recorded demos - with a short, crisp narrative and a simple way to provide feedback - perfect for target users who prefer to watch demos on their time.  
• Visible requirements - Publish the requirements gathered in a selective access portal that is visible to customers, partners, and target users so they can confirm your understanding of their requirement and can also be updated when the requirement moves up to be implemented
• Towards the end of a product development cycle
• Access to a sandbox - Provide access to a stable build through a sandbox so target users or their proxies get a feel for the real product. Such a provision at a Beta stage can be a source of confidence for prospects to know what they would be getting into and for the development team to know what prospects are looking for.
• Conduct usability tests - Specify a set of tasks that the user needs to accomplish using the product. watch how the works her way through the product to complete the tasks, areas that confuse, and areas that delight. The results can be a mine of information for feature design tweaks.
• Speak at user conferences - A perfect opportunity for cross-selling or up-selling the new product. Being able to pitch to multiple prospects while enabling prospects to leverage on each others' understanding of the products selling points could pave the way for a shortened sales pitch.