It started with a single word in a pull request comment: “Tested.” The developer meant that the unit suite passed on her laptop. The reviewer, a former systems administrator, read it as “this ran against something resembling production”. The release went out on a Thursday, broke a payment callback on Friday, and by Monday the retrospective had turned into a quiet blame session. This is a story about that team, and about a fix for team misunderstandings in DevOps that turned out to have nothing to do with tooling.
The symptom: the same argument, every few weeks
The team in question was a mid-sized product group: a handful of developers, two operations engineers who had moved over from infrastructure support, a tester, and a product owner who sat with them twice a week. On paper, they were doing everything right. They had a pipeline, automated checks, code review and a weekly release.
Yet the same pattern kept returning. Something would ship, something would break, and the post-event conversation would circle around a handful of phrases. “I thought it was tested.” “It was ready.” “We said it was safe to deploy.” Each person was confident they had done their part. Each was frustrated that someone else had not.
The product owner described it well: “It is like we all speak the same language, but with different dictionaries.”
The first wrong diagnosis: process
The team’s first instinct was to add process. They introduced a longer review checklist, an extra approval step before production and a mandatory comment template. For a few weeks things felt more careful. Then the checklist became a box-ticking exercise, the approval became a rubber stamp, and the template filled with the same ambiguous words. “Tested: yes.” “Ready: yes.”
More process had not touched the underlying issue, because the issue was not a missing step. Everyone was following the steps. They just disagreed about what the words inside those steps meant.
The second wrong diagnosis: skills
Next came the suspicion that someone lacked skill. Perhaps the developers did not understand operations. Perhaps the former administrators did not understand modern testing. There was some truth in this. People who move into DevOps from sysadmin and support backgrounds often carry a strong, healthy instinct about production risk that developers may not share. Developers, in turn, often have a deeper feel for test design.
But training everyone in everything would take months, and it still would not explain why capable people kept misreading each other. Skill gaps made the misunderstanding more costly. They were not its cause.
The real diagnosis: one word, five meanings
The breakthrough came almost by accident. During a retrospective, the tester asked each person to write down, privately, what “tested” meant to them. Then they read the answers aloud.
There were five different answers:
- “Unit tests pass locally.”
- “The full pipeline is green.”
- “Someone has clicked through it in staging.”
- “It has run against production-like data and configuration.”
- “Acceptance criteria have been checked by the tester.”
The room went quiet. Every previous argument suddenly made sense. Nobody had been careless. Each person had done exactly what “tested” meant to them, and each was surprised that others had meant something else.
They tried the same exercise with “ready” and “safe to deploy”. The spread was just as wide.
Why this happens in healthy teams
It is worth pausing on why a capable team ends up here. Words like “tested” are inherited. Every engineer brings definitions from previous jobs, previous tools and previous incidents. In a team that has grown by hiring from different backgrounds, those inherited meanings overlap but do not match. Because the words sound the same, nobody notices the gap until a release falls into it.
The more cross-functional the team, the wider the gap tends to be. That is not an argument against cross-functional teams. It is an argument for making meanings explicit when people from different disciplines share a workflow.
There is also a social reason the gap stays hidden. Asking a colleague “what exactly do you mean by tested?” can feel like questioning their competence, so people avoid it. Each person quietly assumes the others share their meaning, and politeness keeps the assumption intact. It usually takes a neutral exercise, one that asks everyone the same question at once, to make the difference safe to talk about.
The solution: write down the words that matter
The team agreed on a modest experiment. They would pick the three words that appeared in every recent argument, write one shared meaning for each, and use those meanings for a month.
Choosing the words
They settled on “tested”, “ready” and “safe to deploy”. Other candidates, such as “done” and “urgent”, were noted but parked. Starting small kept the conversation focused.
Writing the meanings
Each definition followed a simple pattern: what must be true, how anyone can check it, and who owns the wording. Their version of “tested” ended up as:
Tested means the full pipeline has passed, including integration checks against the shared staging environment, and the change has been exercised with the configuration it will use in production. Local runs alone do not count. Owner: the tester.
It was not perfect, but it was specific. It also linked naturally to the team’s wider effort to build continuous testing into the delivery workflow, because now the pipeline itself could show whether the agreed bar had been met.
Assigning owners
Each word got a named owner. The tester owned “tested”. An operations engineer owned “safe to deploy”. The product owner owned “ready”, because it was really a statement about business acceptance. Owners could change their definition, but only by announcing the change in the team channel with a date.
Putting the words where people work
The three definitions went at the top of the pull request template and the release checklist. When someone wrote “tested”, the meaning sat a few lines above.
What changed in the following weeks
The first effect was a burst of friction. Several pull requests that would previously have been marked “tested” were now marked “unit tests only”, because the author knew the agreed bar had not been reached. That felt slower. It was also honest.
The second effect was calmer reviews. Reviewers stopped asking “did you test this?” and started asking specific questions that the definition prompted. The tone of conversations softened, because nobody was questioning anyone’s diligence any more.
The third effect showed up in the next post-event discussion. A minor problem reached production, but the conversation was short. The change had met the agreed definition of “tested”; the gap was in what the definition covered. So the team updated the definition, not the people. That shift, from blaming individuals to improving shared language, was the real win.
Lessons from the data world
When the team later described the experience at an internal meetup, a data engineer in the audience laughed in recognition. Data teams, she said, fight this battle constantly with words like “active user” and “complete dataset”, which is why the discipline puts such weight on shared, owned definitions. Her team had picked up those habits through DataOps practices for reliable data, and through providers such as DataOpsSchool, where data teams train in data quality and governance methods that start from agreed meanings.
How to run the diagnosis on your own team
If your team keeps having the same argument, try the exercise that broke this one open:
- Pick a word that keeps appearing in disputes or post-event discussions.
- Ask everyone to write down, privately, what it means to them.
- Read the answers aloud without comment.
- If they differ, agree on one shared meaning, an owner and a place to publish it.
- Use the definition for a month, then review whether it needs adjusting.
It takes about twenty minutes. In many teams, the spread of answers is the most useful thing anyone learns all quarter.
The quiet lesson
The team still ships weekly. They still have occasional problems. But the arguments that used to follow have largely disappeared, replaced by short conversations about whether a definition needs tightening. They did not need a new tool, a new framework or a reorganisation. They needed to notice that “tested” was carrying five meanings, and to agree, together, on one.