The financial impact of UX research: tying insights to revenue and research beyond product

Henrik Mattsson · 00:09
Hi everyone, and welcome to another episode of the UX Research Rundown podcast. My name is Henrik Mattsson. I am the CEO of Lookback and your host for this show, which is now in its second season. On this show we talk about my favorite problem, which is how to bring people along on the journey that is research. Because of course, the whole point of doing research is that we as a team are going to be able to build better experiences and everything. With me today, I have my guest, Claudia Natasia, who is the head of… yeah, no, director of insights at Highspot, and also known from the Learners UX Anywhere conference in New York, where you gave a fantastic talk about how to measure the impact of research financially, which is a very cool and unique and much needed perspective.

So I’m super happy to have you with us today. And we’ll start, as we always do, by getting a bit of your origin story. So if you could just start with how you got into research. When did you realize, hey, I’m a researcher? How did that come about, and what has your journey been?

Claudia Natasia’s path from finance to UX research

Claudia Natasia · 01:18
So I started in research… actually, when I’m thinking about my origin, it goes beyond that. My parents were entrepreneurs and they worked in banking. And so I grew up going to all of these different places like Citibank. While my mom’s finishing up a deal on the 24th floor, I’m sometimes with her, sometimes eating ice cream in the lobby. I’ve always been surrounded by the world of finance, and that’s what made me interested in finance, just because of exposure alone. And when I worked in finance in college and after college for a little bit, some of the clients I worked with were tech clients, and some of the people I talked to were in UX teams.

And so I naturally transitioned into UX, because I think one thing that finance did not have back then is the ability to interact with users, the ability to tell user stories. And as I learned more about the UX field, I realized that I’m drawn to that, and not just the numbers. And that’s what made me think also about how to integrate both worlds together with the hard numbers that we have. How can we use those numbers to tell user stories in a way that’s meaningful, even more meaningful to their lives?

Putting financial numbers on research

Henrik Mattsson · 02:38
Interesting. And I have to ask, because I found your talk on this, and also discussing it a little bit with you at the conference, so fascinating. I’m an economist by training, and I love this kind of stuff. But how do people receive this approach of trying to put numbers on the research?

Claudia Natasia · 02:59
That’s a great question. At first they are skeptical. And actually every single new person I’ve interacted with is always skeptical. Every time I join a new company and I join a new team, they’re also skeptical. Are you sure? I think somewhere along the line of conversations, after repeated interaction, and bringing them into the journey of research and doing this properly, they develop trust. And then they also become advocates. I’ve now even seen a friend of a friend, or a friend of a coworker, that has never met me, but then I meet them at a conference or an event and they say, hey, so-and-so said that they work with your team and that they have this new perception of research, and we’re also integrating it at our company.

And I’ve never talked to them before. So seeing that people are now becoming advocates when they started as skeptics has been amazing.

Research stakeholders beyond product and design

Henrik Mattsson · 03:53
Awesome. Yeah. As an economist, I’ve met this stereotype, I guess, where people think, oh, you don’t really care about people, you care about numbers and stuff like that. But when you talk to economists, it’s like, well, no, it’s people. That’s what it’s all about. It’s human beings doing stuff, and outcomes being produced. So I’m glad that you’re here and taking this perspective and bringing it to these conferences and everything. And as we were talking about before, I’d love to see at one point a model for calculating the financial impact of engagement, stakeholder engagement in research, of bringing people along on the journey.

So we’ll see if we talk a little bit about that in the podcast or not. But I’m asking all of my guests to just bring their experience and, if they have any, advice. This podcast is very much for people who are coming into research, or perhaps changing roles in research leadership. So could you share some of your initial thoughts when you think about bringing people along on the journey, and stakeholder engagement? What comes to mind? What are some of the challenges? What are some of the experiences that stand out for you?

Claudia Natasia · 05:21
The first thing I think about is the different types of stakeholders that we have. If you work in a UX research team, you’d immediately think about designers and product managers as your main stakeholders. And in the earlier parts of my career, I focused most of my impact on working better with product managers, on influencing the thinking of designers and product managers. But since then, I realized that the power of research shouldn’t just be limited to design and product. And the reason is, if we’re thinking about what companies care about right now, especially in this economic downturn, it’s building and growing efficiently.

Companies are going back to fundamentals and great accounting, to make sure that they survive this period of market uncertainty and thrive after. And if the company as a whole is solving this larger problem of better fundamentals, then the questions that researchers have to ask to remain relevant and insightful to the company extend beyond just what makes a good product or what makes a good design. It’s: how can I empower all of the organizations in my company to execute more efficiently? And so the questions my team have started asking are, for example, are we marketing the right content to people?

Can we collaborate with our marketing team to make sure that we are using the right content and targeting the right persona? In my past companies, we’ve run studies where we measure the effectiveness of messaging and the effectiveness of different persona groups, and we do segmentation alongside marketers as well. Going back to your question, then, it really depends on the types of people that we want to bring along on the journey, and also how we want to empower those people to use the insights in a way that’s most effective for them.

Whereas a designer you’d bring to, let’s say, a concept test to observe the study, for a marketer you’d want to make sure that they inherently understand why their positioning is not working and why something else would work. So they would be involved also in the design of the study, and so on. So it really depends on who the stakeholders are.

User-led growth: a story every team accepts

Henrik Mattsson · 07:48
Yeah. Yeah, totally. It’s interesting, because when I look back at the discussions I’ve had with research leaders, perhaps because they came mostly from very product-led companies and stuff like that, we tend to think about research as something that is more intimately connected to product and design than it should be. But the stakeholders that you’re talking about, the broader stakeholders: sometimes people can think, oh yeah, it’s product and design, and they should probably also get a foot in with leadership, because that’s good. But are there any specific things that you’ve noted with working with marketing, or some of these other groups, that are fundamentally different?

Or are they different in the same way that an engineer and a designer are different? Or is this a special group?

Claudia Natasia · 08:46
Yeah, I think every single group is special. And the one thing that I’ll say is, we often talk about a company being a product-led growth company or a sales-led growth company. The approach that I usually encourage my team to take is: how can we be a user-led growth company? And the reason why the user-led growth company is important, especially in this market, is that right now, if you work, let’s say, for a B2B or software as a service enterprise tech company, all of these companies are trying to cut down costs and reduce spending on tools.

So the best way to retain customers is to really make your customers your advocates. If it’s user-led growth, not sales-led growth, then the users are the ones helping grow your product. And because of that, I think that’s the narrative that aligns well with all of the stakeholders, regardless of where they are. If we are collaborating with engineering on, let’s say, a user research study… The most interesting one I’ve done, actually, is a user research study on Kubernetes. It’s an internal study to find out how we can make the deployment process more efficient for fellow engineers.

I mentioned the marketing study as well. So regardless of who the people are, if we fit the discussion under this narrative, how can we shift to be a user-led growth company, how can we help our users become advocates of our product and therefore drive revenue that’s more sustainable for our company, that narrative tends to be consistent and well received across all of the different teams.

Expanding research into other teams’ territory

Henrik Mattsson · 10:22
Super interesting. I was recording another episode of this season with John Cutler, who is a product manager, researcher guy, and he was talking about how one challenge for research teams is that there are already other teams in place doing some of these things that don’t see themselves as research teams. As you start expanding into the organization, and it sounds like you’re doing all sorts of internal stuff with engineers on deployment processes and everything, has that been your experience too? How is it received when you’re expanding your territory, so to speak, into the organization?

Claudia Natasia · 11:02
To be honest, it really depends. Originally, sometimes there might be hesitation. I was actually chatting with someone about this the other day. At the end of the day, we are humans, and we operate sometimes in a very territorial way, where we might think, hey, is someone stealing my job, or are you the right person to do this job? But at the end of the day, I think what really is a selling point for the research team, especially if they want to expand their impact, is the methodologies that we provide.

It depends on the type of research team you’re in. But if you build your user research stack with a combination of both quantitative and qualitative methodologies, and you encourage your research team to also act as shareholders of the company, and actually ask questions about company financials and company growth, then one, you speak the same language as these cross-functional partners, so you can relate to them and they know that you’re on their side. And two, with the unique methodologies that you bring, you provide a fresh perspective and an additional way to look at the data.

The way I think about it is like statistics. When you build a regression model, you want to try to find the right amount of input that ends up being representative of the world that you’re trying to model. If you only have one input, let’s say a Gartner report, that’s not enough. If you have too many inputs, the model becomes noisy. So researchers should really position themselves as a unique input that makes whatever model, whatever analysis, whatever insights you’re trying to deliver to the company a comprehensive representation of the world.

Prioritizing research against company OKRs

Henrik Mattsson · 12:50
Right. Oh, super interesting. I immediately start thinking about prioritization. In these discussions that I’ve had with people about engagement, there are two stories, depending almost on research maturity in the organization. One is that you’re starting with a pretty low research maturity, and you have to fight to even get engagement from your first stakeholder. Let’s say you’re starting in the product team or something like that. And then as you succeed on this journey, and that’s a whole episode in itself, exactly how to do this, it very quickly pivots to a situation where there’s so much demand for your research that you have to prioritize pretty hard.

And the people I’ve had these discussions with have not expanded into these other stakeholder groups in the company. So how do you manage prioritization across the limited resources that you have, when you take this ambitious approach?

Claudia Natasia · 13:50
That’s a really great question. I do it in two ways. The first is hiring. The way I hire is to hire researchers that specialize in particular areas. Right now we have this one amazing researcher who specializes in market research while also doing UX research. So her whole job is to integrate UX research methods into marketing and go-to-market opportunities. Now, I realize that hiring might not be the best advice, especially in this market, as companies cut costs. So with the limited resources that you have, the second way I do this is to align all initiatives not just to our own goals, but to the company-wide OKRs or the company-wide goals.

Doing it that way allows us to prioritize the same way other teams prioritize. We’re not a service organization. We’re not just responding to requests. We are also prioritizing our own initiatives against the company objectives. And a third that I’ve seen work as well is to develop a work cycle where the cycle of research is earlier than the cycle of development. Of course, for more evaluative research, we’d want a design or a product to actually test. But for generative work, doing it ahead and being part of the planning helps, because then all of the insights are ready before they are being used.

So create the cycle. I’m thinking about an illustration of cogs and wheels: you’re the one that’s really powering all of the other wheels and cogs spinning.

Henrik Mattsson · 15:33
Right, right. Interesting. So many questions. But I’m also very interested in the limits of research leadership responsibility vis-a-vis other things. I think you’re saying, and that’s right, that it’s not just a service function. Even if it does this internal thing, it also has this strategic responsibility as a function to do this prioritization. But that’s another layer of engagement and collaboration around that. Who are your stakeholders and partners in that strategic prioritization of research time? Who do you work with? Is that directly with leadership, or who is it?

You don’t have to name names, but you know what I mean.

Claudia Natasia · 16:28
Oh, I’ve always structured my team to work directly with leadership, in the sense that every single person in the executive team has an equal say in what the research team should work on. And me as the research leader, I also have an equal say in pushing back and saying, hey, while you think this is important, I actually think that if my team focuses on this, it could help forecast something else that would be even more important. The equal say part is super imperative when you want to build an impactful research function.

Because if one team, let’s say product management, has more of a say compared to design, compared to marketing, compared to even finance, then you’d always prioritize things in service of product, and you will never find time to do the things that would help serve other teams.

In Lookback Record what each executive needs to know in a stakeholder interview, and Lookback turns it into a research goal. Learn more

Breaking out of the usability testing box, step by step

Henrik Mattsson · 17:29
Oh, super interesting. Oh, so many questions. How to prioritize? This is very meta, because that’s what we’re talking about. I think when people hear this, about breaking out… not that there’s anything wrong with the product research part. That’s not what we’re saying here at all. But to break out into this broader function, I think that’s a pretty exciting future for research. And I’ve talked to other research leaders who’ve said that’s the future. We need to stop being typecast as doing usability testing on product stuff, and actually be this partner.

Where do you start? Let’s say you’re not at that level yet, where this is something that you do. Can you show us the step-by-step roadmap towards breaking out of the usability testing box and becoming this broader thing?

Claudia Natasia · 18:37
That’s a really great question. I want to caveat that by saying that it’s like a learning curve, or a hockey stick chart, where the first few months, or sometimes even a year, of trying to break out of an original process will be very difficult. In my experience, getting stakeholders comfortable with what more you can do is the first step. While you’re maintaining your original process of running usability tests and concept tests, begin proving to stakeholders that there are other valuable research methods. A great way to do this is, when you do a usability test, start tying a usability insight to a financial result.

Let’s say you’re doing usability tests on a change in the UI of a particular feature. And you find out that if you do change that UI, let’s say 20% more users can actually do the action that you want them to do. What does that 20% of users translate to from a financial perspective? Maybe that 20% of users translates to, let’s say, a 50K increase in average revenue per user. Even putting those tiny, tiny tidbits of information in the report starts to build the understanding that the team can do so much more, and starts to build that trust.

So that’s phase one: integrating small elements into your report and your current workflow. Phase two is starting to suggest studies, and citing what you did. Hey, remember when I added that thing to the report? We can actually do so much more if I can collaborate with marketing on this go-to-market initiative that they’re working on, and we can test it earlier. And then proving the value that way. So: incorporating small methodologies and expanding impact to other teams. And the third is finally when you can actually change your process and dedicate actual resources in the team, or actual time in your team’s quarterly roadmap, to work on these types of initiatives.

The most important thing is really starting with those small steps and proving, one, that you can do it, two, that you’re the best team to do this, and three, that it won’t impede on the work that people are expecting you to do as well.

Planning research two quarters ahead

Henrik Mattsson · 20:55
Yeah, yeah. I think that last part is interesting, because when I talk to research leaders, there’s always that trade-off. There’s the strategic initiative, which is what you’re doing here, and I think a lot of leaders and managers, if they’re good and they don’t have a big ego, would welcome that. They would be like, oh, finally someone who’s thinking about all of these things. So that’s one part of what you want to do. Building relationships and taking these steps, that’s another part. And training your team and hiring, that’s another part.

So how do you prioritize as a research leader, not as a team, but you yourself? Do you have a process for this, like in the morning where you remind yourself of the different things? What does that process look like for you?

Claudia Natasia · 22:00
The process really starts at the end of each quarter. A month before the end of the quarter, which is really considered mid-quarter, I begin to think about what would be most impactful, not for the next quarter, but the following quarter. I take my forecast even further in advance and begin planning for the following quarter. And then I have one entire quarter to rally my team, to rally the cross-functional partners, to tell them that something is up and that this will influence their future planning. I think the part of my process that has been most helpful is just really thinking ahead, and planning two quarters ahead on behalf of my team.

Another thing that I’d mention, which is somewhat related to your question: my biggest learning from expanding the impact of my teams beyond just product-specific and design-specific research is that the only way we can do this is if we also make research more accessible to everyone. If researchers’ time is solely focused on running small usability studies, or solely focused on writing reports, and we don’t make reports easily accessible to everyone, and we don’t make data easily accessible to everyone, then it’ll take up so much of their time that, regardless of how many people I hire, they will not have time to actually do more strategic work.

So part of it is finding tools and identifying new processes that will free up more time for researchers to also focus on more strategic work.

In Lookback Share a single finding with a public link, including with people outside your Lookback organization. Learn more

What research tools are for

Henrik Mattsson · 23:41
Right. Oh yeah, I think that last part is so… I’ve been reflecting on this myself lately, because I come from the tooling side of this, obviously. It’s clear to me that tooling is just a small part of it, but it can also be a very important part. And I even started questioning what the role of the tool is. Is it to help teams produce insights quickly, or is it to help teams work in a way where everyone is using their time and collaboration in a scalable way?

It’s almost like we’re moving into this new… it’s like Figma or something like that. I don’t know if this makes sense, perhaps I’m just a tool nerd. But we’re moving into a place now where technology and products are at a place where the tool doesn’t help collaboration towards an end, like an insight or a deliverable or something like that, but actually just fosters collaboration by the way the tool works. And I wonder if that’s what we need to do to manage to scale up to this very ambitious level that you’re talking about, which I agree is what we should be doing.

These are just some random thoughts floating around in my head, and I’m not there yet. But what are your thoughts?

Claudia Natasia · 25:07
Yeah, I completely agree with that. The way I see it: if anyone here has taken a statistics class, you remember having to open the Z table at the back of the textbook to find the exact z-score, and then calculating it manually with a formula on your sheet. That’s so annoying. And now we have a tool like R. It’s completely free. You can type two lines of code and it does all of that for you. But you, as the person with a background in statistics, know how to interpret it.

You know the ethics of using it. And you also know how to manipulate it. Hopefully not in a negative sense, but in a more informative sense. And I really think that the power of the tool, or any tools that we use, is to make our jobs more efficient, so that we can do more of the interpreting, more of the influence, more of the strategic thinking beyond the insights that we gain. And as a leader, the type of tools that I look for are the ones that can promise me, and have tangible proof, that they do help make my teams operate more efficiently.

Why researchers should think like shareholders

Henrik Mattsson · 26:17
Right. But on the collaboration part, if I dig a little bit more into that. How do I ask this question? I love the way that you’re talking about this stuff and the way you’re thinking about it, because it’s ambitious and it’s big, and I’m sure it has impact. I also don’t think this is a very common way of working. And I think one of the reasons it’s not common is because it requires quite a lot of this kind of thinking that you’re bringing to the table, and all of this planning and patient stuff.

So do you think there will be tools that help research teams think like that and operate like that in the future? Or do you think this is just something that humans have to learn? If that makes sense, this question.

Claudia Natasia · 27:15
Yes, it does. I think there will be tools, but I also think humans have to learn. And I am a fundamental believer, and this may be an unpopular opinion, so if anyone’s listening and you think otherwise, I’d love to hear your perspective as well. I have the opinion that tools can exist, but whether people use them or not, there’s always an element of humans that have to really govern the tools, have to encourage people to use the tools, have to learn how to use the tools effectively.

And so I think just having a tool will not help humans or leaders or researchers become impactful. Researchers have to take the first step, and even learn, let’s say, basic accounting phrases. What is a cost? What is ROI? How do you drive impact? Those are the types of things that you have to learn how to talk about. And you have to change your perspective to want to influence the business, as opposed to just your craft. That part is really important, and it’s what I honestly see missing currently from our industry.

Because a lot of researchers think about influencing design and product, but they don’t think like a shareholder. If you’re a shareholder, you want the company to succeed. You want the company to generate revenue, you want the company to minimize costs. And so the type of research work that you do has to target all of those bigger picture questions.

Research as a hedge in a downturn

Henrik Mattsson · 28:54
Yeah, I mean, with the financial crisis now, it’s interesting. I had a conversation with one of my researchers, and she was, as people do now, worrying about how things are going. And she just asked me about it: hey, do I need to worry about my job? And it was so interesting to me, because it was so obvious to me that no, I need you now more than ever. Even if things go badly, you’re going to be fine, because I need you. Because the harder it is to predict the landscape and what’s going to happen, and the more cramped your resources are, the more important it is to make better decisions.

And to make better decisions, you need research. So clearly there’s this thing there. But that doesn’t seem to be the way that a lot of companies are thinking about it. And I wonder why that is. Is this just something we’re going to get to? What’s your view?

Claudia Natasia · 30:01
I am hopeful that we’ll get there. I think right now a lot of companies follow each other’s models when they grow and scale. When we think about how we survive a recession, companies are following each other’s models of laying off, reducing costs, reducing tooling. I actually do some startup advising on the side. And when I talk to these startups, I encourage them to think like a hedge fund. You want to get ahead of the game and hedge against opportunities, as opposed to just following what the masses are doing.

And I think research is that hedge. It’s something that a lot of companies don’t notice yet. But when research is done properly, when we have bigger impact, when, to your point, we focus on de-risking our investments and we structure our research teams that way, that’s the type of research that can really hedge against market uncertainties and help gain revenue faster, help create great product-market fit faster. It’s not a widely adopted thought yet, but I am hopeful that it will be soon. And I really think that’s the key to building more sustainable companies in the future.

The future research team and research leader

Henrik Mattsson · 31:11
Yeah. I’ve been in this industry for a while now, and I remember when we started out, let’s say eight, nine years ago, most people didn’t have research, and they’d speak to leaders who were like, oh, I don’t need that, it’s going fine. And then there’s this evolutionary process happening, where people that think like that are just not going to build relevant experiences and products. And I think we’re getting there. So I do think that this thing will win evolutionarily at one point, because the other ways won’t produce as strong results.

But I’m hopeful, as a human being and especially for research, that we don’t have to go through the harsh evolutionary journey. We can perhaps just learn this and get ahead of the game and actually do something about it. So I hope so. I have another question, and this might be my last question, unfortunately. It’s a little bit of a departure from this. If we start doing more of this, and by this I mean your ambitious, big thinking for research in organizations, do you think that research will be kept together as a discipline that does all of this?

Or do you think it’s going to fragment into specialized teams that each do these things? Will there be an internal research team that does all the strategy and marketing stuff, and then one that’s still going to do product, and they’re not going to be the same? Or do you think we’ll keep it together as we make this journey?

Claudia Natasia · 32:54
I think the organization in itself will stay together, but I think the types of researchers that we hire will be different. I’m starting to see that as the case already. The backgrounds of researchers ten years ago used to be HCI, and now we have people coming from all types of backgrounds that are providing so much value. I’ve even had a lot of chats with PMs who want to transition to research, or data scientists who want to transition to research. And the skills that, for example, data scientists bring from their quantitative modeling into the UX field are amazing, because then you have all of these cool things where you build telemetry against usability.

So I think it’ll fall under one organization, but it’ll have more specialized functions than we have right now. And ten years in the future, I’m really excited to attend a research conference and learn from people doing cool things, let’s say on Web3, with a completely novel approach that isn’t how we currently define user research to be.

Henrik Mattsson · 33:55
Awesome. And what will the research leader of the future look like? What will be their core set of skills and defining features?

Claudia Natasia · 34:08
That’s a really great question. I think it will be less focused on the right methodology you need to use, and more focused on how we drive financial impact. And I’m not just biased because that’s my background. I really think the research leader will have an equal seat to, let’s say, a revenue leader or a product leader. And the discussion you’ll have is not pushing a research agenda, not saying, hey, so-and-so in my team did this amazing work. The discussion the research leader will have is: hey, we want to grow ARR by 20%, this is how we do it.

Henrik Mattsson · 34:48
Yeah, love it. That would make my job easier. So I hope this will be the future. Great. Thank you. You never know what to expect. That’s what’s so cool about having people as guests. I never know where these conversations are going to go, but I’ve learned a lot, and it made me think bigger about these things. And I hope that the audience has had a similar experience. So thank you so much for coming on the show and sharing these perspectives. I’m looking forward to hearing that talk on the impact of engagement in the future.

So thank you so much. Happy researching, everyone. Thank you.