Follow us and stay on top of everything CRO

Forget Culture: Here’s How to Build an Experimentation Ecosystem

Duration - 45 minutes

Key Takeways

  • Experimentation should be a cross-departmental effort, including both online and offline aspects of the business, to drive a constant need to learn and optimize.
  • Advocate for a senior role, such as a Vice President of Experimentation, who oversees experimentation across all departments and is channel agnostic.
  • Identify and educate teams about opportunities for experimentation they may not be thinking of, thus building the need for it.
  • Understand that Conversion Rate Optimization (CRO) is a subset of the wider experimentation spectrum, which includes learning, exploring, and optimizing.
  • Recognize that shifting to an experimentation mindset is a significant change that needs to start from the top down, but awareness and initiative can drive this change.

Summary of the session

The webinar, hosted by Divyansh and featuring Manuel from Effective Experiments, focused on the evolution and challenges of conversion rate optimization (CRO). Manuel highlighted the shift from CRO being a part of marketing to becoming a specialized role, and now expanding to entire teams. However, he noted that these teams often lack the skills needed to drive a change in attitude towards experimentation.

Manuel emphasized the importance of understanding the current perception and awareness of CRO within an organization. He suggested an exercise for CROs to engage in conversations with senior stakeholders to gauge their understanding of CRO and experimentation. He shared that many stakeholders often perceive CRO as just running tests and generating revenue, but fail to understand the full scope of the role.

Manuel stressed the need to change this narrative. He pointed out that experimentation is not just about winning tests, but an exercise in learning that provides valuable insights, regardless of the outcome. He warned against the danger of stakeholders getting used to the idea of CRO as a revenue generator, which could lead to disappointment when the results plateau.

Manuel concluded by urging CROs to shift the narrative of experimentation to providing insights, thereby increasing its priority within organizations. The webinar ended with Devianj thanking Manuel for his insights and the attendees for their participation.

Webinar Video

Webinar Deck

Top questions asked by the audience

  • Would you say that experimentation can be applied across all departments in an organization? How to find out if there is actually a need?

    - by Timo
    This is a really good question. And I think, earlier I mentioned at this point that, you know, Sierra went through a phase where, like, we were thinking about redesigning it. And we should rename it a ...s experimentation. My current view about experimentation is, it's a discipline that is regardless of the channel. It's not I know VWO does it for web and apps and stuff like that, but experimentation can happen in multiple different teams, departments, and even business experiments.Right? There are different types of ways of experimenting, to find answers, to explore, and to optimize. So chances are if you have teams. If you look at digital, it is the easiest, to get going. Right? So you have your web test, you have your email test, PTC, social, SEO, and so on and so forth. But what about offline? If you have offline, stores, I mean, you know, sometimes you can optimize how people move through the store or find certain things. I know supermarkets do that a lot where they're testing product positioning to see, footfall and, and, purchasing. Yes. You know, experimentation should be happening across the business because that drives that constant need to learn and constant need to optimize. So the question about how do you find the need for that? Well, you look at the opportunities that are there. You can position the opportune fees that might be relevant to those departments or those teams. If we always advocate for our senior vice president or vice president of experimentation sitting right on top of the overseas experimentation across all the different departments and be channel agnostic. So they will get information about the web. They'll get information about email and so on and so forth. Because, ultimately, to orchestrate and unify all those teams together so they can learn from one another and drive the insights forward. Right? So the best way to go by finding the need is really building that need or seeing what opportunities they may not be thinking of and educating them about that.
  • I think a problem happens when people confuse CRO with experimentation. CRO was born to find small or medium discoveries to increase revenue and adjust knobs but experimentation has an even greater purpose of detecting opportunities and risks and qualifying their impact.

    - by Felipe
    Yeah. Absolutely. That's exactly the point I was trying to make earlier. Like, we conflate CRO and experimentation is the same thing, but the way I look at CRO is if it's a subset of the wider experim ...entation spectrum where you can either, you know, learn you want to learn something new. So you're experimenting to learn. You want to explore. You can use experimentation techniques as a way of exploring, what you want to optimize. You've got A/B testing as a methodology for optimizing and learning. Right? So that's the whole point of it. And there is a wider thing about experimentation that businesses need to be doing. This is why that culture of experimentation doesn't sit right with me because there's such a bigger potential out there. And for a lot of CROs, we are still, you know, you might still be doing web A/B testing. That's your remit. So how do you build that mindset when you're only working on the optimization part of it? So there's still a long way to go, but that's why in my opinion, it starts from the top down. That initiative starts from the top down, but the awareness might not be there. If people work towards their awareness, then the initiative starts, and that's where the change and all the shift will start happening.


Disclaimer- Please be aware that the content below is computer-generated, so kindly disregard any potential errors or shortcomings.

Divyansh from VWO: Hello, everyone. Thank you so much for joining the VWO webinar. We always try to upgrade and inspire you with everything around experimentation and conversion rate optimization. I’m your host, Div ...

All you Need to Know About Conversion Optimization Get the Guide