UX research and growth: owning metrics, counter metrics and asking leaders for what you need

Henrik Mattsson · 00:09
Hi everyone, and welcome to the fourth season of the UX Research Rundown podcast. My name is Henrik Mattsson. I am the CEO of Lookback and your host for this podcast. And with me today I have a guest, Zoë Glas, who is a UX researcher, a strategist, and a collector of tiny things. And for this season, we’re going to talk about the upcoming UX Research Week this summer in… Or, it’s actually just called Research Week. Sorry, Alec. The next iteration of the great Learners franchise. And it’s going to be an entire week of fantastic events this summer in San Francisco.

And UXR Conf is still there for a couple of these days, but then we also have other days about other things. So we’ll talk a little bit about that and, you know, get too excited about a year in research. So welcome, Zoë. What’s up?

Zoë Glas · 01:08
Thank you so much. It’s great to be here. Going really well, enjoying this rainy weather compared to your snowy weather, but hoping that turns around soon.

Henrik Mattsson · 01:15
Yeah, you never know what you’re going to get in the valley, I guess. The fog. Awesome. So, as per ancient tradition, in this fourth season… It’s getting better now, because it used to be an ancient tradition in the second season, too. But now we’re getting pretty serious here. We always start the podcast with kind of digging into the origin story of our guests. And the reason we do that is because our field is full of people who are joining from all over the place and, in their first, their second or even third year of research, are always asking themselves, well, do I belong?

Do I have the right background? And for us who’ve been here a while, the answer, of course, is yes. You can come from almost any background. But it’s always great to get those stories about how people get into this. So could we start, please, by you just giving us the origin story of Zoë Glas?

Zoë Glas’s Origin Story

Zoë Glas · 02:11
Yeah. Origin story. So I’m a wildlife biologist, and I actually still often introduce myself that way. It tracks to how I think about things. I’ve given talks on how I still use wildlife management frameworks in determining where my product should go, and how I talk to people and think about these things. And I love it for these kind of questions, because I feel like it’s an off-the-wall answer. But I studied wildlife biology. I studied avian biology in my undergrad, mammalian biology more in grad school.

I actually trained skunks to run on treadmills, which is about as easy as you’d think. They bite.

Henrik Mattsson · 02:47
For fun, or…?

Zoë Glas · 02:48
No, actually. So we were looking at the exercise efficiency of skunks. So you can basically look at how much CO2 they produce and how much oxygen they intake as they’re working out on a treadmill, basically to look at efficiency metrics: how far could they move, how far could they get, and what does that teach us about skunks being skunks. In specific, skunks and a lot of these kind of mid-sized predators, foxes and things, are going to expand their range with climate change, because they’re not as limited by temperature.

So we kind of want to see how far can they get. So yeah, that’s where I started, totally different. And within that, I started to switch to what’s called the human dimensions of wildlife management, which is how do we manage the people doing things. So how do I get you to use a bear box? How do I get you to not try to befriend the skunk or the fox in your backyard, or make a pet of a raccoon? And I really, really loved that aspect of it, which is unique in the wildlife world, where most people get into it because they hate people.

I was actually at a conference once, and somebody asked the whole crowd of like 500 who was interested in the human dimensions part, and I was the only one who raised my hand. But I’ve loved it ever since and was doing that. I was working for the state of California as what’s called an R3 coordinator, so getting people to hunt and fish more, and loved it. Didn’t love the three-hour each way commute and very, very low pay. And a friend was like, hey, you know, you should think about UX research.

I think you’d really like it. You like asking questions, you like pushing on people. It’s necessary for a good researcher. And, yeah, the rest is history. I talked to a guy named Rob Youmans, who now runs research for Workspace, and he was kind enough to see something in me and took a shot on me. And now I’m a staff researcher at Google. So here we are.

Henrik Mattsson · 04:39
Wow. Well, I want to pivot this entire season to just talk about wildlife stuff. I mean, I don’t know if you can see behind me here, but that’s the Mountain in Montreal, Canada. And if you live close to here, you have a lot of skunks and raccoons and squirrels and all kinds of stuff. And it’s a really interesting dynamic, even if you’re in the middle of a very large modern city.

Zoë Glas · 05:02
So in some ways it’s more fascinating, because it’s how it all interacts. I’ll send you some papers. Anybody that’s listening, send me an email. But foxes, they don’t eat your cat. They’ve trained themselves to eat trash, which is super weird.

Henrik Mattsson · 05:15
Yeah, yeah. Well, you know, I’m sure when you start digging into it, it’s like, oh, okay, that makes sense. Really interesting. Awesome. Well, I would actually like to talk for hours about that, but let’s go back to UX research, which is where we all ended up. Cool. So now you’re the first guest for this season, so I’ll do a little bit more context here, just mostly for the listeners. Alec at Learners… What I love about Learners and the UXR Conference and UXR Week and everything that is happening now is that the team there never settles.

Even if things have been a great success, they never want to redo it in exactly the same way. And they always want to kind of expand on things and try new things and try new concepts and everything, which is fun. And except for just expanding the whole UXR Conference to a full research week now, which I think makes a lot of sense, we were also talking about ways to kind of… Instead of having everything happening in a short period of time, and people are in there and it’s exciting and everything, but it’s so intense for one week, and then it’s gone, and it takes another year.

We wanted to kind of provide an opportunity to introduce basically the organizers and speakers to the audience way ahead of time. So that’s what I’m going to try to do. So you are speaking at UXR Conf, but you’re also organizing a day focused on UXR and growth. Is that right?

Zoë Glas · 07:06
How do we make things bigger, better, cooler with growth?

Henrik Mattsson · 07:09
Awesome. That sounds very relevant. Were you involved in coming up with this? And how did that… Why should we, you know, have a day about that?

The origins of Zoë’s UXR and Growth session at Research Week

Zoë Glas · 07:21
Okay. There’s two answers here. So I’m lucky enough… I pitched a talk to Alec years ago. It was, I think, the first one out of the pandemic, that I still to this day can’t believe he took, and I’m so happy about it, which was why we should be thinking about religion and UXR. And we became pretty good friends after this talk. I think we jibed. We enjoyed the debate of does this talk even make sense and how it got there. So we talked pretty regularly, and he had reached out and said, hey, there’s something going awry with one of my days.

But I keep thinking about it, and I think it actually should be growth anyway. Do you want to take it? So I unfortunately don’t have the ability to say I’m so brilliant. That does fall fully on Alec. But why should we do it? And why should this be on the top of everyone’s mind? UXR is in a weird state. We’re seeing a lot of articles right now: it’s the end of our field, AI is going to take our jobs, democratization is going to take our jobs.

None of it’s positive. None of it’s like, this is going great. Even the state of design recently was like, everything’s over. But I don’t really think that’s true. I do think we’re in a really weird state of transition where, for a very long time, UXR was strictly usability research, right? Does this thing work? Will the chair tip over? It was pretty straightforward, and researchers were listened to in a straightforward way. Either you changed the button or you didn’t. Either you fixed the wobble in the chair or you didn’t.

That’s developed with time to being the strategic partners that we are today, to integrating within product spaces more directly. We tend to see fewer dispersed research teams and almost entirely integrated research teams. But that has happened at various places in various fields with various companies to varying degrees of success. When the pandemic hit, we had this opportunity to really reset and say, hey, we could hire people anywhere, we could do this research anywhere. And I think that provided an opportunity, in some ways really positively, to bring a ton of diversity of thought and training and skill into the field.

And at a time when labor was cheap, right? People were getting loans for nothing. So there was no risk to bringing that in. But those two things, in my mind, combined in kind of a toxic storm, where our value was changing and we didn’t actually know how it was changing or where it was going. And then we, positively, but still chaos, introduced a bunch of chaos into the system of different training and different people. And now we’re sitting at the other end of that going, what is research about?

What’s the point of this? Where are we going? How do we judge value? How do we judge merit beyond, did I tell you to change the button, and you changed the button, and a metric moved? And when we think about how we redefine that and how we re-solidify ourselves, and then hopefully specifically train the next generation of researchers coming up, growth to me is where that comes in. How do we own our metrics? How do we think about our value? How do we connect to the product across multiple dimensions and axes, and then continue to prove out that value, such that we kind of know where we’re going and we know where we’ve been, and we can bring people along with us, other researchers, our partners, etc.

So I’m super excited about the day. We’ve had a lot of diversity of thought coming into that. Everything from how do we look at metrics and measure these things? How do we think about 0 to 1 products? How do we really drive these big changes? But fundamentally, I do see this as, not to toot my own horn by any means, the answer to this: where are we going, and how do we continue to be here in a way that we want to be here?

Henrik Mattsson · 10:56
Yeah. Oh, that’s so much to unpack there. Awesome. I’m definitely going to go watch this thing. It’s interesting, this thing, because I’ve been in this field since 2015 or something like that. And as you know, running a startup in this field, I remember when we started, everyone was telling us, oh, you know, this market is too small. You should sell to designers or product managers instead, because there’s more of those. There aren’t enough researchers. It’s not big enough. But it was pretty obvious that there was something there.

Well, we were looking into this. It’s so obvious that it is, for me. I mean, you shouldn’t use “obvious”, because it’s kind of like either it’s redundant or it’s insulting, right? But to me, it was obvious that there’s a very strong connection between good research and growth, right? Especially now that it’s so easy to build product, and that’s got even easier now, and now it’s going to get cheaper too with the models and everything. So one of the few things that you have left to compete on, especially in the early phases, are just the experience of the thing, and there’s no way to access that if you’re not actually looking into it.

And when we started, all the big super successful companies that everyone was trying to be, they were doing research, and then no one else was doing research, right? And then we went into this super growth phase where now it’s like every type of company… There’s no industry now that doesn’t have a presence in the digital space, right? And now we’ve had kind of a dip, but the fundamentals haven’t really changed, right? So I think it’s very true that there’s something there. But at the same time, people haven’t…

How do I put this? There’s going to be a question here at the end. When you speak about this, you sound convinced that there’s something here. When people like me have been looking at this for a long time and trying to support these processes, I also see that there’s something there. I think in the last three seasons here, I’ve talked to a lot of research leaders, and I don’t think anyone would question that there’s something here. But at the same time, this is not uncontroversial.

What you just said, and what I just said, it’s not uncontroversial in companies. Is this something that you’ve seen too? And what do you think the resistance says, if that’s… Perhaps that’s not the right word, but…

What is the resistance to UX research?

Zoë Glas · 13:38
Yeah, no, but I totally know where you’re coming at with this. Where is this coming from? I mean, I think there’s a couple pieces of this. One, we rarely define what we are. You’re going to see a million articles about what we’re not. We’re not an iterative thing. We’re not just a survey specialist. We’re not an end of the road. Very rarely are we saying what we are and what that looks like, right? Hey, a really healthy relationship with design and with eng looks like this, and here’s what we need to get there, by the way.

And that’s going to vary by company and area. I work at a part of Google that is very technical. We’re an internal compliance team that builds tooling to ensure that we’re using the data that you produce in the way that you want it to be used. It could not be more technical and engineering heavy. And as a result, our products tend to be really technical and engineering heavy. So it is a little bit of a fight to get in that room and to prove you have value, especially when we can say, well, it just has to meet these regulations, and it feels very binary.

Either you can meet the regulation or you can’t meet the regulation. Now, the piece that I’m always pushing on is, yeah, but we should make that easy for our engineers to do. We should make that easy for our users to be able to control. And the easier it is, the more they’re going to want to do it, and the more they’re going to be connected to these things. And that is ultimately positive and meets that fundamental goal, keeping things in compliance. A piece of that, I think, is we’re not very good at actually forming what our goals are.

And I think one of the biggest indicators of this is that we fundamentally change our OKRs every year within companies, and they will be so drastically different. And something that isn’t a key metric, a key result that we’re driving, should be pretty similar, at least in theme, right? We should not be changing the direction of our teams. Now, research is going to solve that piece, but there should be something that’s bringing that along. And we often don’t have that piece in research. I think it’s stuck in that because we are reacting to whatever is in front of us, rather than these bigger pieces.

Now, the other piece I think that we’re up against is something you alluded to: we have a ton of data now, and it actually can be really cheap to get data, right? I can distribute a survey. It might not be any good. I can get product feedback in app really quickly, and I can even A/B test things. And we hear this all the time now: oh, just A/B test it. But there’s a couple of issues with that. And I think the biggest one is, if you’re not going the right direction and you’re A/B testing, you’re going to get the best of whatever shit direction you were already going in.

We were talking about running before this. So if I told you, hey, you should run towards the Golden Gate Bridge, and you run down Market Street, you might get there eventually. It’s definitely not the best route from where you were starting. And then you’re going to go, do I go left or right? And maybe you go left, and you’re going to end up out in the Presidio in the sunset, like five miles from where you want to be. And eventually you might get to that bridge.

But if you’re always going into this A/B, you might go the other way. You also might never get to that bridge. Or you might think, I got to a really good coffee shop today, and that was a great outcome, and you forgot that you wanted to get to the bridge, or you built in some other metric, or it’s the next year, so now your key result is coffee shop based. If we don’t have a clear sense of, this is why we should be going somewhere, this is our North Star, we can thrash around all the time, and we can continue to make things better of whatever poor direction it is.

By way of example, I was talking to a PM at a major social media company, and they said one of the ways that they got promoted once was that they would take an ad and make the text really big for six months, and then everybody would click on it because the text was really big. And then they’d stop clicking on it, so they’d make it really small again. And people started clicking on it again because it looked different. And each time they’d bump their metrics right around perf, right?

And they would just complete this cycle for years and years and years before they finally decided they were bored with that and changed company. It was objectively improving that metric for a short time. It did what it was supposed to do. Is that the right thing? Now, there’s probably a better ad out there. There’s probably a better approach. So that’s, I think, where research needs to come in, and where we end up getting stuck is choosing between right and left. So the more we can say we’re not choosing between right and left anymore, we’re really thinking about what that bigger thing is, what is our growth goal, how do we confirm against these things, the better position we’re going to be in.

Henrik Mattsson · 17:58
Right. Very, very interesting. Because I think what you just said is very… You know, without going into a long rant about it, we’ve done research on researchers for a very long time now. And we see what very successful research teams have in common and what not very successful research teams have in common, and getting what you just talked about right, and being proactive in that and working with leadership on that, is definitely one of those things that successful research teams do. So then, of course, there are people who feel like, well, my research team could be more influential, could have more of an impact, be more successful, impress leadership and stakeholders more.

What are some of the steps you think that researchers can take to bring their practice closer to this kind of ideal that we’re talking about?

What steps can research leaders take to optimize their practice?

Zoë Glas · 19:06
Yeah. I mean, so I’ll give a hint at my other talk on this, which is actually called “Care Less: The Secret to Success in Research”. So I want to put out first that I don’t think we should be in a position where researchers have to become PR experts of their field and constantly be like, I’m important, pay attention to me. One, that’s going to cause burnout. Two, sometimes it’s just totally out of your control, especially if you’re listening to this and you’re at the beginning of your career.

You should be following people that are more senior than you to drive those things. Don’t try to boil the ocean. Don’t try to solve the world if you’re not in a strong position to do so. Do the best with what you have in front of you and within your team, and change the leadership within your team. But set reasonable goals. We’re seeing burnout rates in research that are just way too freaking high. And I think a piece of it is because we’re telling people they can be responsible for changing things fundamentally when maybe they can’t be, and that’s just a little bit too big of a task.

Now, with that said, though, everything you can do from any level can track towards that. So set some North Star goals for yourself, for your team. What are the fundamental metrics? One of the things I love to do when I come into a team is build a vision and principles together as a team, and then I just bring that back up, put it at the top of every doc. There’s the principles of our team. Does it track to this? One of the things we have right now is that the teams that we build tools for are empowered to make the decisions that they want to make.

So I’m not going to come in and say, you have to do this this way. It’s that I’m going to make sure that I have the tools so that you can do what you want. You might do something I don’t want you to do. You might be taking more risk or less risk or whatever than I think is best for that company or that piece. But that’s not up to me. It’s just to give you the tools so that you can make an informed decision to do that thing.

And then I can push back on anything that we’re doing and say, hey, does this actually meet that? If we send this person 900 alerts that this thing is out of scope and it’s not a big deal, does that meet that? And you can have that conversation in an easy way in these more iterative ways. Now, if you are more senior, stop caring less, start caring a little bit more, and see what you can own for the people around you. And I do think a piece of that is trust.

A piece of that is just saying this stuff. I am blown away by the times in my career that I have gone straight to directors and VPs, who I am so afraid to be in the room with, and just said, hey, I need these three things from these people, and here’s why. And while I’ve sat in the room, they’ve sent that email, and I’ve had that thing within a week, because they don’t know… You’re not convincing them of something else. They truly haven’t thought about you at all, especially in these larger companies.

So not a bad thing, but they have a million things going on, and the people they do know how to think about and have thought about are sitting in their meetings. So go sit in those meetings and ask for the things and see what happens. And if they’re truly saying no, pick a different goal, right? Don’t climb Everest without oxygen. Think about what it makes sense for you to do in that moment, and kind of chase that thing to whatever end that is, and then bring the people up behind you.

It’s something we’re really struggling with in research right now. We see a lot of people get to the top, and I’m seeing a lot more stomping down than lifting up. Now, remember to lift up.

Henrik Mattsson · 22:23
Interesting. Can you explore that a little bit more? Because I think that’s definitely… What does that look like? You don’t have to name any names, obviously, but people would like to know a little bit: what do you mean, stomping down and lifting up? What are some good and bad examples there?

Zoë Glas · 22:43
Yeah. I mean, I think one of the things that we are seeing is this, oh, well, if you just have the relationship with them, then people will listen to you. I think if it’s purely relationship based, it’s not a good thing, right? We should never be… That reinforces all sorts of old racist, sexist, etc. relationships that we used to have, where, oh, you’re not in the old boys club, so you’re not being listened to. Bring people along. Hey, here’s how I did it, and here’s how I’m helping you.

One of the things I challenge myself to do in any meeting with someone more senior than me is name-drop the specific contributions of other people at least three times. Danielle did an amazing job leading this effort on critical user journeys, and here’s how they did it. Vanessa established this huge effort to start to measure those journeys. Be really, really specific in how other people are doing things, so people can see their value and bring it up and start to understand the full value in everything that goes into these practices.

But also think hard about what you did to get there and how you can actually share that with someone else. And what, by the way, is not sustainable? If the way that you had to do it to grow your org, and to get that feedback and get that influence, was like befriend someone and go to the movies and babysit their kid, that’s probably not healthy to have a conversation with them now, like, I’m so glad we’re besties now, and let’s bring these other people up.

But again, I think this comes back to what we were talking about before, which is we don’t know what we’re doing as a research field. And I know that’s a bit of a hot statement, but we’re still really figuring that out, and we’ve got to be more specific with that. What are we working towards, and how are we making sure that we’re working towards that as a discipline? No engineer is sitting around going, how do I get research to care about engineering more, right? So why are we still asking that question all the time?

Henrik Mattsson · 24:35
Yeah. It’s so interesting. I was having a conversation the other day with someone who is a very senior person in our field, and I’m sure you might even know this person, but let’s not name names. But they were saying basically that this is the year where we need to get more honest with ourselves about how we up our game, and we need to really address some of these issues that are holding us back, some of which are outside of our control, but some of which are very much in our control, you know.

And it sounds like this is one of those, and I’m glad to see that we have a whole week coming up where hopefully we can really have these conversations. I think that would be leadership, really, to kind of help everyone who’s coming up. But I really like the part you said about relationships. You can’t have a seat at the table because of the relationship. You should have that seat at the table because the people around that table will benefit from having a researcher at the table.

It’s so interesting how some of these things are just politics for politicking’s sake, which makes me think… It might be hard to answer, but how much of this resistance or this politicking or whatever do you think is coming from other functions, and how much is just internal stuff, right? Because you could imagine that research is a threat to… So, you know, a PM that keeps hitting their fake targets. I’m sure that PM is incentivized to do that and might not care so much about actually the long-term success of the product or something like that.

How cynical should we be about these things? How much of that stuff have you seen in your career?

The politics of targeting metrics

Zoë Glas · 26:27
You know, it kind of gets into a couple of things. One, control what you can control. And the reason I say this is, yes, I was in a meeting recently, and we’re using a metric that really doesn’t make sense to me. And I was like, this is silly. This person having three additional minutes once a month is not in any way… That doesn’t mean they have this extra hour. When I get three additional minutes, I make an extra cup of tea, right? The productivity of the company doesn’t change.

Let’s look at other things. And the problem with that is, one, that one’s really easy to measure. It’s really easy to put a dollar value to. And ultimately, I think one of the core issues we have in all of tech, and I’ve seen this at every company I’ve worked with, in every company that I’ve had friends working with, is we incentivize individual behavior. We don’t incentivize product outcomes. We like to think we incentivize product control. We don’t. We incentivize people being able to say, I did that thing.

Now, I will probably blame Google’s performance review for this, which has changed a lot over the years. But we really wanted people to come back and say, I made a difference, look what I did. And there’s pieces of that that make a lot of sense. If people do really good work, we should value and recognize that really good work. But if our entire system is based on, how do you judge if you did something? Well, show me the metrics you moved, show me the specific piece.

We are incentivizing bad behavior, right? A lot of the time, or at least short-term behavior. Now, ideally, if you’re running these companies, let’s start to think about short term and long term. And as a researcher, though, one of the things you really need to be doing, and we’ll talk about this a lot in the growth day, is set your targets. What are you trying to move? Often we say, okay, well, I’m going to suggest that you do this, and hopefully engineering put some sort of log in there, and we can see if a behavior change…

Call those metrics, right? I move this button. Did more people rent a car? Now, sometimes that’s great. My own resume has a bunch of these metrics on it. It’s easy to look at. It’s sexy. It’s easy to talk about. Look at all this money I made. Now, the real question is, if you hadn’t moved that button, would that money have been made that differently? We really don’t know. It’s a very biased short-term metric. But as a researcher, as the person helping establish OKRs and establish that test plan, think about what those long-term metrics are, right?

Is the product becoming more usable with time? Is the product becoming more beneficial with time? Now, a piece of that is not getting too sucked into what your partners are doing and saying, okay, let’s take the hit list of all the research you’re requesting from me, and we’ll prioritize it relative to your specific things. They’re not prioritizing against you, so make sure you have at least a couple of projects in there that are about you. One of the things I love for this is critical user journey programs.

They’re a pain in the butt to get ready and to do right. But what they look at is, are the fundamental goals of the user being met through the product, right? One of the big ones for YouTube, they’re very public about, is I want to be entertained. Do I still feel entertained when I go to that product, or has it started to become too busy? Has it started to become too frustrating? That should still be there, even if gaming is trying to improve X, Y, Z metric.

I haven’t worked on that team for years and years and years. I promise I’m not giving anything away here. But we want to keep that bigger picture in mind. And it’s research’s job and responsibility, especially at the senior end, to go, how are we keeping that in mind, and how are we committing to keeping that in mind? So yes, metrics are part of that. But you should also be measuring your own stuff and measuring the impact of those things on that. So if you’re doing an annual survey that’s looking at satisfaction and usability and other changes, make sure you’re really seeing what those changes are and what they might be tracking to.

One of the things I really push people on at any level is counter metrics. We measure metrics all the time. Did that button move it? Great. But what about what didn’t happen? What things can you look at? Let’s say you told someone, you know what, the way that we’re displaying price on the screen is really confusing. And they go, well, we’re just going to see if more people book this or buy that book. Yeah, well, what else could they be measuring? Is it the amount of time they’re spending on that page, the amount of time that that thing is spending in cart before being checked out?

If you have metrics on how they’re visiting other web pages, are they visiting more other web pages? And measure those too. Because here’s the thing: humans suck at thinking into the future. It’s why climate change is so hard to talk about. We’re not good at it. But what can you give them in the short time that meets them where they are, that makes that conversation clearer? And as researchers, we need to take a lot more responsibility in adding to that conversation and setting expectations of what’s brought into that conversation.

Where does the resistance to UX research come from?

Henrik Mattsson · 31:11
I mean, if you’re coming into UX research, you should definitely listen to this advice. As someone who is… I’m a stakeholder of research, right? And I have the benefit of being in a much smaller team, so I don’t have all of these layers, right? But there’s nothing more frustrating for me than when I discover that we have lost precious time because we’ve been chasing these fake metrics, and someone knew, hey, we’re chasing fake metrics, and no one told me because everyone was benefiting from the fake metrics.

That’s when I throw tables and stuff like that, not in front of people, but quietly in my office. And that’s where I think, you know, I love research for this, because it’s kind of my intelligence box that can come and tell me, help me see these things ahead of time, right, where the other parts of an organization might be incentivized not to really tell me, right? And so I think there’s a lot of power in what you’re saying, and I definitely think everyone should do this.

But I can also see how it’s kind of risky, frustrating, and you make a lot of enemies if you do this, because you’re complicating people’s lives, for sure, with this.

Zoë Glas · 32:27
Yeah. And I think what you’re tapping into here a little bit is one of the things that I think fundamentally is disconnected with research right now. So one of the things we talked about already: am I in the room? Am I at the table? Am I respected at the table? That’s actually a really difficult question, and it puts us in kind of a victimhood stance, or at least having to… Like, I have to earn your trust, I have to deserve it, I have to prove that…

You shouldn’t… The company hired you. Somebody thought you had value there. You are inherently in the room. We all know you should be in the middle, however, and you still need to be in that room and continue to prove that value. But he said it is. People don’t know. We go into this assuming that everybody understands what we could be doing and how great we could be if only we were listened to. And so often they just don’t. They haven’t worked with a researcher. We haven’t done a really good job of establishing at a higher level what we’re doing.

It’s even harder now because we are in a state of transition that we’re trying to navigate through, and bring in all these other factors, like working in really limited scopes or working with really limited resources, with the democratization of AI, with fill in the blank here. But we can come in and say, hey, this is what this looks like, and this is what I need. And anybody who has been in a fight with their partner, has managed to pick up and drop off at school or with their dog, you’ve been at this point where you’ve had to step back and say, you know what?

Hey, honey, I’ve got a meeting at nine. I can’t handle this. I can pick up that. And you have to walk through and plan that day, and all of a sudden you’re on each other’s page, and you’re no longer feeling unsupported. The same thing at work: if they don’t know how to support you and what that might look like, you’re fighting a battle that they don’t know that they’re in, or that you’re getting brought into some minutia that’s going to make you sad. But truly, I have had conversations recently where I’ve walked in with directors and just said, hey, I need these four things.

I need you to tell this person this, I need you to commit to this OKR, and this is what it’s going to look like, and here’s how I’m going to do it, and these are the number of engineers I need, and this is this. And they’ve said, oh, cool, okay, I can do that. And it’s not that they were fighting them before. They just didn’t know. And you kind of got to give people the benefit of the doubt there, especially if you’re reforming or setting a real example of what research could look like at that top level.

And to bring it down now, again, if you’re an L3 or an entry-level person, please don’t feel like you need to go show up at the VP’s desk. Go to your partners that are more senior and say, hey, I think there’s something missing here. What can you do to help me? How can I figure this out better?

Henrik Mattsson · 35:13
Yeah, and I think this ties back to what you were saying about the A/B testing, right? You don’t want to be stuck… You don’t want to fight the wrong fight. You want to kind of take a step back and make sure that perhaps it isn’t even a fight, as you say. And I think it can be very scary for people who haven’t done it before to walk into a director’s office and say those things: hey, I need this and this and this. But I think the reaction you’re describing, where they’re going, oh, great, I think that’s probably 95% of the time what you’re going to get.

Because those people, their job is to try to help everyone do their job, and they just want better information on how to do that instead of having to guess it.

Communicating the benefits of UXR directly to senior management

Zoë Glas · 35:55
Truly, I have never had a reaction that wasn’t that, from the VP of engineering, VP of product, directors, CEOs. The answer is always, oh, really interesting. Okay, yeah. What do you need? Who do I need to email? The more you can break it down, the better. And again, lead with that why. Don’t be like, hey, I need you to email that person and tell them this. But like, hey, we have this goal, this OKR of this. Here’s what we know we’re not doing right for the user and what we think is going on there.

But I need more data, or I need more of this, and we need to commit over here, because right now, the way we built this, they’re not getting it right. And have a couple of nice quotes or whatever to go with it. But they’re not getting it. It doesn’t mean that anybody else is doing something wrong. We just need to, together, do something more right. And almost universally, you are either going to get some really interesting information about some turf war that you can’t affect, and therefore you should give up on, we talked about the politics earlier, or you’re going to leave with what you want or more, and somebody who’s like, I really like that researcher.

I really like the way they think about things.

In Lookback Bring the clip, not only the quote: put moments from several sessions into a reel and share it with a link. Learn more

Henrik Mattsson · 37:02
Yeah, yeah. And you immediately become a partner also, because you’re someone who is interested in actually finding solutions. It’s so interesting. I work with two very brilliant people who are not researchers, but they’re a designer and an engineer, and they often work together, and they’re very passionate in everything, and they very often look like they’re in conflict with each other. But actually… And I used to be scared of this. I was like, oh, they need to like each other. I don’t want to mess things up here.

And I was like, oh, I’m not going to get involved. But it turns out actually they’re just very passionate, and they come from different places, and they just want it to work. And sometimes they can’t really unlock this thing, and they go down some track that isn’t really right. And doing what you’re suggesting, which is just like, hey, from my understanding, this is what we’re trying to do, and from my area of expertise and my experience, we should probably not be doing what we’re doing, if that’s what we want to achieve.

Here’s a couple of suggestions. Now, I might be missing information here. Clearly you’re talking to all of these people. You probably know something, but I’m just bringing this information to you. Here’s what I think we should do. That’s actually scary, but super powerful. And no one’s going to be angry at you for doing that.

What is Zoë’s research superpower?

Zoë Glas · 38:22
For sure. And I will say, I think when people ask what my research superpower is, it’s that I can get to that and say, hey, this is what I’m seeing. What are you saying? And invite people to that conversation. I also jokingly, but not so jokingly, often call researchers the therapists of the UX world. A lot of what we’re doing is just, what is it you’re saying? What is going on here? I had an old manager. Her name was Rochelle Edwards. She’s brilliant. If you ever get a chance to work with her, do.

And she gave me some really good advice, which was: if you’re getting really frustrated, if you’re feeling nobody’s listening to you, it’s not moving forward, start researching your team instead. Take off your stakeholder hat, take off your responsible hat, and just start asking questions. Hey, what’s going on there? Gosh, this doesn’t feel right to me. Can you tell me more about this? And usually you’re going to get to something really quick: either you didn’t understand, there was some piece that they didn’t understand, they’ve made an assumption about you or the work you’ve done, or you’ve made an assumption about that, and you’re going to get to something better.

Now, I will also say that can super suck. And there’s so many times in my career that I’ve been like, I hate that I’m a therapist. I hate that I’m this person. I want to just show up to the work and be mean to people and run around with it. But it really does get you further. And then go get yourself a little treat for having done it afterwards. I love a little ice cream we got in the Ferry Building on my way home. Whatever it is for you, go do that thing.

But research your team, just switch it up. And if you’re truly feeling like, God, I’m just not getting here… Also, don’t be shy. If you have trust with one or two people, go to them and say, you know, I’m just not getting here. Here’s what I see. What’s up? And again, they might give you some insight about you, or they might give you insight about the product. And no matter what, it’s going to get better. Every time I’ve opened myself up to negative or direct feedback on how I’ve done something, I’ve become a better researcher.

My partnerships have become better. In one case, I got my best friend out of it. And she’s amazing. So for my fifth shout-out of the day, shout out to Anna for calling me on some bullshit I did back at YouTube. And I’m a better researcher and a better friend because of it.

Henrik Mattsson · 40:35
Well, awesome. You see, everyone who’s listening and watching this: lots of rewards on all kinds of levels for moving in this direction. I love this.

Zoë Glas · 40:44
And it will be uncomfortable at first, but it gets better. If you know yourself, by the way… The two people you’re describing, I love working with them. I love debate, and yeah, it’s great. My uncle still is like, why aren’t you a lawyer? But I love it. And if you hate it… I’ve had a conversation with people, and it’s actually how I start working with people, and I go, I love debating, I love pulling things apart, I love pushing out things. How do you feel about that?

And some people say, great, let’s go. And some people go, I really don’t like that. I like to think about things that are bad. Awesome. I’m going to send you a bullet-pointed list of what’s on my mind two days before our meeting, so that you can think about it in advance, and then we’re going to have a better conversation. Nobody’s winning. Nobody’s losing. It’s just progress in some direction.

Henrik Mattsson · 41:27
Yeah, and the thing you said about being the therapist here, it’s like you can actually improve other people’s collaborative relationships by doing this, because you’re creating space for trust to develop and things like that. You know, people are very kind of… And especially the smartest, best people I’ve ever worked with, they look like they have their shit together, you know, they look like they know what they’re doing. But actually, on the inside, they’re like, oh, yeah, you know, why am I going here? Am I going here?

And for them it’s not obvious where they’re going. And to come in and ask some questions and create that space for, hey, what are we thinking here? This doesn’t make sense to me. That’s a scary thing to say, especially when you’re a junior and you’re trying to advance and you’re trying to build your career. It’s hard to come into a room and say, hey, I don’t get what we’re doing here. Is it me? Am I stupid? But it’s a really powerful way, not only for yourself and your function, but for the whole team, to just build that culture.

The importance of asking questions

Zoë Glas · 42:32
Yeah. And you might find some things that the team’s just built around that are really confusing, especially as a researcher. One of the ones that we’re up against right now is annotations versus attributes. They kind of describe the same thing. We’ve decided one is automated and one is not automated. Our users don’t really get that. So there was a ton of confusion, and some people thought it was straightforward, and some people didn’t. And just being able to say, hey, that seems really weird. Like, oh, there’s actually a bigger issue here that we didn’t notice, and we can come into some sort of agreement or alignment on, or focus on and fix in the future, or decide we don’t need to fix it because it has some other answer.

But open your mouth. Share that. If you’re super uncomfortable with that, or if it’s just too new, just find one person, right? Find your PM, find your PDM, find the researcher that’s been in that space a little bit longer and be like, hey, I’ve got some dumb questions. Can I ask you my dumb questions? And the other thing to remember: one of the best things you can do to build community and to build trust is to ask something of somebody else, that you need help.

We often go the other way. We want to be generous. We want to be the person that drops the cookies off next door. If you are always dropping off cookies next door and never receiving those cookies, never receiving the offer to pick up an item at Trader Joe’s, never receiving the offer to watch your kids that night, you’re not building community. You’re building debt. So build community. Ask for things. Ask for help. Receive that help. You’re going to have a better relationship with everyone at work, at home, everywhere else.

Henrik Mattsson · 44:12
Awesome. Well, I feel like that was a very good note to kind of start ending the call. I mean, that was a lot of good stuff in there. And so this is what growth is about, really: just building community, building trust, asking questions. Simple as that.

Achieving growth through UXR

Zoë Glas · 44:29
And growth. Simple as that. So easy. You know, it’s the Tower of Babel, right? For those of us that studied any of that as kids or adults, we need to align on what we’re doing. We need to be able to contribute to it if that alignment makes sense. And maybe you can’t decide if the tower is getting built, but you can probably decide if it has a staircase or an elevator, and just see, what are the constraints? Where can I work within that? And what can I own in getting the yes I wanted or avoiding the no I didn’t, in measuring that and supporting that and being a vital member of that conversation, that you can still build towards whatever it is, right?

You’re not going to turn YouTube back into a dating platform, even if you really think video-based dating should come back. But maybe you can notice that there needs to be more social connection or opportunity to connect in real life. I don’t know if that’s a good idea. It’s probably a terrible one. My safety had habits going off as I say this. But you can contribute to what you’re noticing there and feel good about it, right? At the end of the day, you should leave and say, I did good work.

I feel good about the contributions that I’m making, and I’m not fighting to be at the table. Right?

Henrik Mattsson · 45:43
Yeah, yeah. Lovely. And remember, everyone, it’s hard to build a great product, otherwise everyone would do it. So, you know, don’t be too hard on yourself if this is not working on the first try or something like that.

Zoë Glas · 45:56
Definitely. And to your point earlier, it’s harder to do it well. It is so much harder to do it well. It is so much easier to just A/B test everything that we want and push it out and see which one goes better. But you might never reach the Golden Gate Bridge.

Henrik Mattsson · 46:12
For sure. Exactly. That’s awesome. And yeah, it is hard. But one final point there, and I’d love to hear if you agree: I note that even though this is hard, a lot of the skills required for the kind of stuff that you’ve talked about here are actually pretty core researcher skills. Ask questions, try to figure out who is my user, who is using this research? How can research be helpful? How can the product be helpful? You have all of these skills in you. You just go do it, right?

“You’re a researcher all the time.”

Zoë Glas · 46:47
Yeah. Use them everywhere, though. You’re a researcher all the time. You are not only a researcher when you’re talking to a user. You’re a researcher when you’re writing that study plan. One of the things I see with junior folks all the time is they send their plans off, somebody says, oh, add these three questions, and they just add them. Don’t just add them. There is a reason you missed that question. Go ask why. What do they want to know about them? What is that information going to do for them?

How is it going to contribute to the decisions made? Because you might choose not to ask that question, or to ask for more, right? It’s a conversation. There is no right or wrong. It’s a conversation. One of the things people ask all the time: how do you know that you’ve done enough research? You don’t. You will never have done enough, right? Because we don’t know where we’re going. We don’t even know if we’re heading towards the Golden Gate Bridge. We just know we wanted to go have a good time and have a nice walk one day.

But at some point you got to take that trip, and that’s it. It’s like, how do you feel confident enough that that’s a good direction to go, and then be open to maybe going another way if you need? And what are the things you need to see to have that?

In Lookback When stakeholders add questions, record why in a stakeholder interview, and Lookback turns it into a research goal. Learn more

Henrik Mattsson · 47:54
Yeah. And it’s so interesting, the example you’re giving. As a researcher, if a user you’re interviewing would say, hey, add these three features, you would never just be like, okay. It would be more, you know, why? Tell me exactly why are we doing this. So, yeah. Very good, very good. Awesome. Well, this was a great first episode, I think, anyway. Self-praise. Exactly, ice cream. It’s a bit too cold for ice cream in Montreal, but it is great. Thank you so much. Awesome. Look, this was amazing.

Thank you so much. I will see you all in… And now I’m talking to the audience here. We will see you all in San Francisco for Research Week, I hope. And as you have heard already, clearly, Growth and UXR is definitely a half day or a day. Is it a half day or a day?

Zoë Glas · 48:55
It’s a day. Come hang out for the day.

Henrik Mattsson · 48:55
There we go. It’s a day. It’s going to be…

Zoë Glas · 49:00
Awesome day. Maybe it’s a half day. Now you have me doubting myself. It’s going to be great no matter what. I’m going to be there. You’re going to be there. All the cool people that are listening are going to be there. Bring your questions, bring your challenges. We’ll get through this together.

Henrik Mattsson · 49:15
Awesome. Happy researching, everyone, and stay safe out there. Bye-bye.