13 Traits of High-Performing Experimentation Teams
Experimentation teams are on a constant search to understand what works, and what doesn’t. This requires specialist skills and expertise. Experimentation teams have a high spirit of curiosity, learning and adaptability.
You may also enjoy these articles:
Introduction
Experimentation is all about learning faster and creating more value for your customers. If you do these things right it produces improved business performance.
When done well, experimentation can become a competitive advantage for an organisation. The faster you learn, the faster you can make decisions, the faster you can create and deliver value to customers.
If you can do this better than your competition, you avoid the risk of missed opportunities, with the costs of doing the wrong thing decreasing significantly.
Experimentation enables a business to proceed in the face of uncertainty by making lots and lots of smaller decisions with more confidence – nothing is ever 100% certain.
This got me thinking. What are the hallmarks and mindsets of high-performing experimentation teams?
Experimentation teams have a high spirit of curiosity, learning and adaptability. But what other traits are required to excel at experimentation?
In this article we’ll discuss 13 key characteristics and behaviours of high-performing experimentation teams:
Hypothesis driven
Structured learners
Express humility
Action orientated
Flexible and adaptable
Data informed
Curiosity
Customer obsession
Focus on impact
Not hierarchical
Excellent narrators
Entrepreneurial
Bravery
If you’re in the process of building out capability in an existing experimentation team, or establishing a completely new experimentation function, referencing these behaviours and traits during the hiring process will be invaluable.
This list is non-exhaustive.
I’m sure that you can think of other behaviours and characteristics that are also demonstrated by high-performing experimentation teams that you’ve worked with.
1. Hypothesis driven
Good experimentation teams think in terms of hypotheses, not ideas.
Ideas are a commodity, they’re everywhere, and everybody has got lots of ideas. Ideas are cheap.
“The problem with ideas is that they can manifest into false facts when they are not proven or disproven”.
If you’re not careful, untested ideas can become part of your organisational fabric and narrative.
A well-formed hypothesis is the cornerstone of a high-quality experiment.
Experimentation teams mine qualitative and quantitative research to generate hypotheses that are grounded in reality.
Hypotheses are inextricably linked back to improving customer and business value, ideas aren’t.
A hypothesis is testable and falsifiable, being able to be proven wrong.
2. Structured learners
Experimentation is underpinned by the scientific method of inquiry, the same approach that has produced countless advancements for humanity.
Experimentation teams think and work more like a scientist.
“They build models, test the models, then refine and iterate the models. These learning loops are repeated relentlessly until hypotheses have been proven or disproven”.
Designing and executing experiments is not so hard.
However, teams must work in a highly disciplined and structured manner to ensure sound experimental design and trustworthy evidence gathering.
The approach is focussed and intentional.
We’re just applying scientific principles that are hundreds of years old to products. This approach helps to solve business problems much faster.
3. Express humility
High-performing experimentation teams are high on humility. They have to be, otherwise their spirits would be crushed.
Imagine working on initiatives day-in, day-out, knowing that very few will succeed. It’s a great barometer of resilience.
Teams are comfortable knowing that they do not have all the answers. Experimentation is an ongoing process of search and discovery.
Even more important is being comfortable with being wrong, nearly all the time. Very few hypotheses are proven or supported.
We never really can know what will work. There is no such thing as a sure thing.
“Rather than expressing their thinking in terms of yes and no, right and wrong or black and white, experimentation teams think more in terms of confidence and probabilities”.
This involves navigating the “grey zone” of possibilities to understand what may “probably” happen.
This is a different thinking framework and mental model.
Information is an asset, not a threat. Experimentation teams are constantly being required to re-calibrate their beliefs.
The goal is to be advancing your knowledge and understanding, rather than being right.
4. Action orientated
Experimentation teams work to a different operating cadence than other business functions.
The operating horizon for experimentation is more near term, than future orientated.
“Rather than thinking in horizons of weeks and months, experimentation teams think in terms of hours and days, working in tighter, faster learning loops”.
If your strategy is sub-optimal, you want to find out immediately, not in two months time.
The riskier your experiment, the more critical it is to expose your assumptions to your customer base as soon as possible.
Doing something today, is always better than waiting until next week. Now always beats later.
5. Flexible and adaptable
It is common for experimentation to unearth unexpected insights and findings.
What you thought was going to happen, often doesn’t. We’re inherently bad at being able to predict the future.
Experimentation teams are flexible and adaptable, shifting investigation and inquiry in the direction that they are being taken.
“Good experimentation teams can detect and follow the impact of change”.
If a hypothesis is unexpectedly proven, they can flex in this direction, shifting organisational resources to support testing and validation of a new hypothesis, and line of inquiry.
Frequent course correction is required to discover the optimal path to your goals.
If you can become good at course correction, it’ll save you a lot on time and money in the long run.
6. Data-informed
Experimentation teams don’t blindly follow the data.
High-performing experimentation teams leverage multiple data inputs and insights from quantitative and qualitative sources to form a frame of reference.
Data is central to the job of the experimentation expert. It is used to support and bolster decision-making, inform new ideas, and ultimately create better customer experiences.
“Good experimentation teams look beyond the dominant, internal company narrative”.
They can spot the assumptions, opinions and strongly held beliefs of key stakeholders a mile off.
Data and statistics come naturally and are innate – or can be quickly learned.
Data is fuel for hypothesis.
The more informed you are, the more valuable your hypotheses will be, the more successful you’re going to be with experimentation.
Data, analysis, and insights are the experimentation raison d’etre.
Experimentation teams value objectivity, over subjectivity.
7. Curiosity
Experimentation teams are good at being able to sniff out a rat. They are naturally curious.
It’s all about the questions that you ask. High-performing experimentation teams can drill down into the “why’ by asking the right questions, at the right time.
“Nothing is taken on face value. If something seems too good to be true, it generally is”.
If the data or results seem questionable, further interrogation is undertaken to get to the bottom of potential anomalies or aberrations.
A good experimentation team is not shameful about re-testing experiments. This should be an expected process, to try and further understand cause and effect.
Organisational trust in experimentation data and insights is of utmost importance.
Experimentation is never viewed as a one-trick pony. A singular experiment never defines a strategy or hypothesis.
8. Customer obsession
The experimentation team will use structured methods of qualitative and quantitative research to learn from customers. They have a healthy obsession with customers.
Direct contact with customers is relished.
“Experimentation teams don’t deal in hearsay or proxies”.
They will actively seek to dive into the “why” to understand what is causing the “what” from experiments (I.e., the customer behaviours and actions).
They use structured research, insights, and observations from customers to establish a picture of reality.
9. Focus on impact
Experimentation teams deal in impact.
All experiments should be aimed at progressively advancing the business closer to its strategic objectives over time.
By staying intimate to the customer will ensure that your research is focussed on solving the right customer problems, having the biggest business impact.
“The experimentation team act as a default strategic guardian”.
When people get side-tracked, they need to take a leadership role to steer the boat back in the right direction.
Experimentation teams must have a razor-sharp focus on strategic objectives, customer problems, insights and jobs-to-be-done.
That’s why it’s important to hire people that are intelligent and intellectual, thinking about the business impact of the changes they’re making.
You don’t want people wasting time thinking about changing the colour of a button from green to blue.
10. Not hierarchical
This behaviour can be a little more challenging to live and die by, however, it is a key principle that underpins experimentation.
Experimentation teams don’t have a pecking order for whose hypotheses get tested. If it can be tested, it should be tested.
Everyone’s ideas should get tested. Senior Leaders aren’t absolved from having their ideas tested.
“There’s no executive path to experimentation”.
Every hypothesis is subject to gathering further evidence before any additional time or money is spent on it.
Having a “C” in your title does not make you any more successful at deciding what should be in your product.
11. Excellent narrators
Good experimentation teams are savvy at organisational storytelling.
You don’t need to be able to provide a presidential inauguration, but you do need to be able to convey experimentation insights and findings in a concise, compelling manner.
Experimentation is 50% designing and executing experiments, and 50% analysis and communications.
This is where the rubber hits the road.
“Experimentation is meaningless unless key insights and learnings are shared broadly within an organisation”.
Cross-functional information sharing is critical. This enables shared understanding of the customer to be established at all levels of business.
Impactful verbal and visual communications are key.
Faster learnings, means faster decisions, which means that you can create and deliver value to your customers faster.
This becomes a competitive advantage over time.
12. Entrepreneurial
Experimentation teams see the world through a different lens. They like to colour outside the lines.
They have a natural inclination towards creativity, innovation, and invention, being highly entrepreneurial at heart.
Resources for experimentation (people and money) can often be scarce. Experimentation teams need to hustle hard to have impact.
They’re motivated by changing the status quo and are always on the lookout for improvement opportunities.
“Satisfaction is derived from relentlessly improving business performance and customer experience”
Keeping things structured and orderly is not nearly as rewarding as navigating through chaos and uncertainty to discover new, novel insights.
Failure comes with the territory. Much of what experimentation teams work on day-to-day will end up on the editing room floor.
Most importantly, high-performing experimentation teams are lifelong learners.
Just like any entrepreneur, experimentation teams are searching to understand what works, what doesn’t work, what they learned and what the next decisions need to be.
13. Bravery
To work in experimentation, you need to be brave and fearless.
Each day you can easily find yourself flying in the face of conventional wisdom, and indoctrinated, legacy thinking.
“It takes great conviction and courage to stand up to the dominant business narrative”.
Sometimes, this will involve being a single, lone voice championing for change, a different direction, or new beginnings.
When you’re in that discussion with a member of the C Suite , or a Senior Leader, who are driving a counter-narrative to the customer insights and learnings, it takes strength of character to stand up and challenge constructively .
As a custodian of customer data and insights, it’s important to be able to confidently hold your ground and mount influential arguments that are grounded in reality – data and insights from customers.
Swimming upstream most days can be challenging. However, the rewards certainly outweigh any downside.
Conclusion
There’s a lot of specialist skills and expertise involved to do experimentation well.
High-performing experimentation teams straddle scientist and entrepreneur. Just like both pursuits, they’re on a constant process of search and discovery.
Experimentation teams are searching to understand what works, what doesn’t work, what they learned and what the next business decisions need to be.
This cycle is repeated relentlessly to prove or disprove hypotheses.
A high spirit of curiosity, predisposition to learning and a healthy dose of flexibility and adaptability are required.
Ultimately, experimentation teams need to hang their hat on the impact that they have. The business should always be advancing closer to its key business objectives.
Need help with your next experiment?
Whether you’ve never run an experiment before, or you’ve run hundreds, I’m passionate about coaching people to run more effective experiments.
Are you struggling with experimentation in any way?
Let’s talk, and I’ll help you.
References:
Before you finish...
Did you find this article helpful? Before you go, it would be great if you could help us by completing the actions below.
By joining the First Principles community, you’ll never miss an article.
Help make our longform articles stronger. It only takes a minute to complete the NPS survey for this article. Your feedback helps me make each article better.
Or share this article by simply copy and pasting the URL from your browser.