translate Automatic translation · View original
Conversations that should happen before implementing AI
Canal Ia Para Agilistas
When organizations adopt new technologies without first clarifying their purpose, they fall into "reverse thinking": starting from the solution instead of the problem. It is an approach that can generate superficial implementations that rarely produce sustainable value.
AI is no different. In fact, it can be worse. Unlike other enterprise tools, AI has characteristics that amplify the risks of a non-reflective adoption: it learns from our data (including our biases), it can automate decisions without clear human oversight, and it generates outputs that seem convincing even when they are incorrect or harmful.
However, the pressure to "not get left behind" leads many organizations to jump straight into AI pilots without having first discussed fundamental questions: What work do we want AI to do? What work do we want humans to keep doing? How will we know if AI is working well? What happens when it makes mistakes?
These are not technical questions. They are conversations about values, about how we understand knowledge work, and about what kind of organization we want to be.
Conversation 1: Why do we want AI?
The first conversation we need to have is brutally honest: why are we considering AI? The usual answers ("because everyone is doing it", "because the competition already uses it", "because it can make us more efficient") are warning signs, not valid reasons. If a team uses AI to write code faster but that code doesn't solve the right problems, have we gained anything? If we automate the generation of documentation that nobody reads, are we being more efficient or simply producing more noise?
Many organizations can confuse activity with progress. Adopting AI without clarifying the desired outcome is another way of falling into this trap. Before talking about tools, we need to answer:
- What specific problem are we trying to solve? Not "improve productivity" (too vague), but something concrete: "reduce the time developers lose writing repetitive tests", "help product managers identify patterns in user feedback", "speed up the generation of visual prototypes in discovery".
- Why is this problem important now? Is it genuinely limiting our ability to deliver value, or is it premature optimization of something that works well enough?
- What non-technological alternatives have we considered? Sometimes the real problem is not that we need automation, but that we have unnecessarily complex processes, a lack of role clarity, or skill gaps that would be better solved through training or organizational changes.
If the answer to "why do we want AI?" is not clear and specific, the rest of the conversations make no sense. And if the real answer is "because we are under pressure from above" or "because there is budget available", then the conversation we need to have is with those who are applying the pressure or allocating that budget, not with the technology.
Conversation 2: Who decides what to automate and what not to?
Once we have clarified the purpose, the question of control arises: who has a voice in decisions about which tasks are automated with AI and which remain human responsibilities? This is neither a technical nor a hierarchical question, but one deeply related to how we understand agile work.
Decisions about technological changes that directly affect teams' work, but are made without their participation, can erode trust and commitment. When AI is introduced top-down without consulting those who actually do the work, the implicit message is: "we don't trust you to make smart decisions about your own tools".
But involving teams doesn't mean simply "communicating better" the decisions already made. It means genuinely co-creating the criteria for deciding what to automate. Some useful questions for this conversation:
- Which tasks consume time without providing learning? Repetitive manual regression tests can be good candidates for automation. Exploring unfamiliar APIs, probably not, because the exploration process is where we learn.
- Where is critical human judgment needed? Automating code generation can speed up implementation, but automating prioritization decisions without human oversight can lead us to build the wrong things faster.
- Which tasks do we enjoy and that make us better professionals? Not everything that can be automated should be. If developers enjoy and learn from writing certain kinds of code, fully automating it may reduce their professional development and job satisfaction.
Empowered teams need autonomy to decide how they work. The introduction of AI should not be an exception to this principle. If we believe in self-organizing teams, then we need to trust that they can make smart decisions about when and how to use AI in their work.
This does not mean that each team does whatever it wants without coordination. It means that organizational policies on AI should be built with the teams, not imposed on them. And that the criteria for deciding what to automate should be explicit, debatable, and revisable.
Conversation 3: How will we measure success (and failure)?
If we don't know how we will measure whether AI is working, then we are adopting technology based on faith, not evidence. And in agile contexts, where we iterate based on feedback, we need clarity about what feedback we are looking for. Poorly designed metrics can generate perverse behaviors. If we measure the "success" of AI solely by speed (lines of code generated, tickets closed, time saved), we may incentivize the production of low-quality outputs that create more problems than they solve. The useful metrics for evaluating AI adoption in agile teams should capture multiple dimensions:
- Outcome metrics, not just output metrics. Not how much code AI generates, but whether that code reduces bugs, improves maintainability, or speeds up value delivery to users. Not how many meeting summaries it produces, but whether decisions are made faster or with better information.
- Quality and trust metrics. How often do teams need to review or discard AI outputs? Are they blindly trusting suggestions or validating them critically? A team that reviews all AI-generated code is using the tool differently (and potentially more safely) than one that accepts it without review.
- Team experience metrics. Do teams feel that AI is helping them or adding cognitive load? Are they still learning, or is AI replacing professional development opportunities? Team satisfaction is not a "nice to have"; it is a leading indicator of sustainability.
- Risk and failure metrics. How many errors introduced by AI have we detected? How many have we not detected? What biases are appearing in the outputs? A system without failure metrics is designed to hide problems, not to learn from them.
Beyond defining what to measure, we need to agree on when to review these metrics and what we will do if we don't see the expected results. Are we willing to withdraw or reduce the use of AI if the metrics show it is not working? Or have we invested so much politically and economically that going back will be impossible?
This conversation about metrics should not wait until implementation. It should be part of the pilot design. Because if we start without knowing what we are looking for, we will never know whether we have found it.
Conversation 4: What happens when AI makes mistakes?
All tools fail. AI is no different. But there is a critical difference between an error in a spreadsheet (which we can usually detect) and an error from a language model that generates code or documentation that seems plausible but is incorrect. Human experts develop an intuition for detecting when something "doesn't smell right". With AI, that intuition takes longer to develop, because the outputs are often good enough not to trigger our alarms.
Safe systems are not those that try to eliminate all errors (impossible), but those that are good at detecting and responding to errors when they occur. Applied to AI in agile contexts, this means we need explicit conversations about:
- Who is responsible when AI makes a mistake? If a model generates code with security vulnerabilities that make it to production, is it the responsibility of the developer who didn't catch it, the team that chose to use that tool, or the organization that pushed to increase speed without adjusting quality standards?
- How do we create safe spaces to report problems? If teams feel pressure to "successfully adopt AI", they may be reluctant to report problems or errors. We need to cultivate a culture where talking about what isn't working is valued, not penalized.
- Which review and validation processes are non-negotiable? Some organizations are establishing that certain types of AI-generated code (for example, code related to security, privacy, or critical financial calculations) always require review by an expert human. What are our red lines?
- How do we learn from mistakes without blame? An AI error should be treated as a system error, not as an individual failure. We need retrospectives that investigate not only what failed technically, but what pressures or incentives led to relying on AI in ways that turned out to be problematic.
This conversation is uncomfortable because it forces us to recognize that introducing AI brings new types of risk. But pretending those risks don't exist doesn't make them disappear; it only makes us worse at detecting and responding to them.
Conversation 5: What skills do we need to develop now?
The arrival of AI does not mean that certain skills become obsolete, but the mix of valuable skills does change. And teams need time and resources to develop these new capabilities. This conversation is especially relevant in agile organizations that value continuous learning.
Agile organizations need to constantly invest in developing their teams' capabilities, not just in delivering features. The adoption of AI is no different. Some skills that become more critical:
- Effective prompting. It is not simply "knowing how to use ChatGPT". It is understanding how to structure problems in ways that language models can usefully process, how to iterate on prompts, and how to detect when an output is plausible but incorrect.
- Critical thinking about AI outputs. The ability to quickly assess whether an AI output is correct, useful, or shows biases. This requires solid domain expertise, which it does not erode.
- Designing processes that combine human + AI. It is not about "AI or humans", but about designing workflows where each does what they do best. This requires systems thinking and experimentation.
- Ethics and responsibility in AI. Understanding implications for privacy, bias, and responsible use of data. In product teams, this is part of product thinking, not the exclusive responsibility of a centralized ethics team.
The question then is: are we giving teams time and resources to develop these skills? Or are we assuming that they will "learn on the go" while also being pressured to maintain the same delivery pace?
Conversation 6: What are we not going to do with AI?
This may be the most important conversation, and the one that the fewest organizations are having: which uses of AI do we consider unacceptable, regardless of their potential efficiency? Setting clear boundaries is not a sign of technophobia, but of organizational maturity. Some organizations are establishing explicit policies, for example:
- Not using AI for performance evaluation decisions. Although a model could theoretically analyze performance data, judgment about a person's performance must remain human and contextual.
- Not using AI to generate personalized communication with clients without human review. The reputational risk of an error or an inappropriate tone is too high. * Not using AI to make prioritization decisions without expert human validation. A model can suggest, but it cannot understand the full strategic context or the political implications of certain decisions.
- Not training models with sensitive client data without explicit consent. This seems obvious, but the line between "improving the product" and "violating privacy" is sometimes more blurred than we would like to admit.
These red lines are not universal. Each organization needs to define its own based on its values, its regulatory context, and the specific risks of its domain. But what is not acceptable is having no red lines at all, or having them implicit and never discussed. AI presents real risks: bias, errors, misuse, erosion of critical skills, impacts on employment. We need honest conversations about these risks, not optimistic narratives that minimize them.
What these conversations reveal
If you have made it this far and are thinking "this sounds like a lot of work before even starting with AI", then this article has served its purpose. Because yes, it is a lot of work. And that is precisely the point: the thoughtful adoption of any technology that impacts how we work requires conversations that go beyond "what tool do we buy?"
These conversations are not bureaucratic obstacles that delay the "real" adoption. They are the adoption. They are where we clarify values, align expectations, build trust, and design systems that truly work for the people who use them.
The organizations that are adopting AI most maturely are not necessarily those using it fastest, but those asking these uncomfortable questions first. And those willing to answer "maybe we shouldn't use AI for this" when the answers are not satisfactory.
In agile contexts, where we value collaboration over contracts, individuals and interactions over processes, and responding to change over following a plan, these conversations are consistent with our values. We are not being "anti-technology" by insisting on having them. We are being agile: starting with the problem, not the solution, and iterating based on real feedback.
Which conversation do you think is most urgent in your context?
Bibliography
- Cagan, M. (2020). Empowered: Ordinary People, Extraordinary Products. Wiley.
- Cutler, J. (2023). "The Messy Middle of Product Development". The beautiful mess. Available at: https://cutlefish.substack.com
- Dekker, S. (2014). Safety Differently: Human Factors for a New Era. CRC Press.
- Edmondson, A. C. (2018). The Fearless Organization: Creating Psychological Safety in the Workplace for Learning, Innovation, and Growth. Wiley.
- Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution Press.
- Gothelf, J., & Seiden, J. (2017). "Sense and Respond: How Successful Organizations Listen to Customers and Create New Products Continuously". Harvard Business Review Press.
- Heffernan, M. (2011). Willful Blindness: Why We Ignore the Obvious at Our Peril. Walker & Company.
- Klein, G. (1998). Sources of Power: How People Make Decisions. MIT Press.
- Martin, R. (2009). "The Design of Business: Why Design Thinking is the Next Competitive Advantage". Harvard Business Press.
- Perri, M. (2018). Escaping the Build Trap: How Effective Product Management Creates Real Value. O'Reilly Media.
- Torres, T. (2021). Continuous Discovery Habits: Discover Products that Create Customer Value and Business Value. Product Talk LLC.
Take part in the poll
This article includes an active poll in the Scrum Manager community.
poll View the poll in the community