newsletter icon
Stay up to date

ON AIR

On Racism and Bias in Research

An illustration of two hands—one black, one white—with interlaced pinky fingers rest beneath a collection of charts and graphs.

    Our VP of strategy, Jessica Hartley, is joined by Accenture Senior Research Principal, Tchicaya Robertson, Beacon Raconteur Founder and Strategic Researcher, Leslie Marable, Instrument Research and Testing Strategist, Morgan Rose, and Instrument Strategy and Research Director, John Whaley to discuss how industries can mitigate racism and bias in research, how we can better approach research and what that research ultimately impacts.

    Jessica Hartley (JH): Hi, everyone. Welcome again to On Air, a podcast hosted by Instrument. A digital and innovation agency headquartered in Portland, Oregon with offices also in New York. And now we are hiring remote. So we have folks that are working all across United States, which is very exciting. I am so happy and excited to bring you another episode of On Air. We started this podcast very focused on understanding their root issues and causes of systemic racism in the advertising technology and consulting industries, and also understanding how we can mitigate bias and ultimately do better for ourselves and for our clients. Today's episode is focused on mitigating racism and bias in research. And I'm very excited to bring this amazing panel to you today. And so I am going to let them introduce themselves.

    Tchicaya Robertson (TR): Hi, Jessica, and thank you so much for having me. I am uber excited about it. So my name is Tchicaya Robertson, I work with Accenture, I've been here for going on 15 years. But I've been doing research for close to 25 all totaled.

    Morgan Rose (MR): I'm Morgan, my pronouns are she and her. I've worked in varied strategy contexts and research has really always been a catalyst for that. So I'm part of the research and testing team here at Instrument. So that's really from discovery research to some iterative design testing. We're really just shaping our version of user centered design. So thank you, Jessica, for including me in this conversation. I'm really excited.

    Leslie Marable (LM): Hi. Hello. I'm so happy to be here. My name is Leslie Marable. I am a seasoned digital marketing strategist, data storytelling researcher, and I've been in the game for more than 15 years. And my aim really is to support critical thinking and to power others through the consumption of database information and better decision making, particularly now. I have my own consulting company, Beacon Rock on Tour, and I currently work with an agency called Randstad, and I'm so happy to be here.

    John Whaley (JW): I'm John, I run the research and testing team at Instrument. I've been here about two and a half years, and prior to that, I spent about 12 years in market research. A lot of work in technology in general. Silicon Valley tech sorts of things. Human-centered psychology is where it all started. So I think that's where my interest started and where it continues. And I'm excited to have brought this work to Instrument and really flushed it out here. And also, really excited to talk to all of you. It's going to be great.

    JH: Awesome. Well, thank you. And again, thank you for taking the time to be a part of the podcast today. I'm going to start us off with a softball question. I promise audience, we're going to get into some tough stuff later. But I wanted to just start with, how did you come to research? And I know I'm using research and I'm doing air quotes for those that can't see me because, research, and strategy, and data, and insights are really, truly core to all of our roles in any number of ways. But how we get to it, and how we got into a role that combines all these things together is always pretty fascinating.

    For myself, I went to school in journalism, so there was always this foundation of data, and insights, and facts. And my first job out of college was at a tech startup, but it was focused on content. And we had a market research angle to us and we would do marketing research and I controlled a panel of 100,000 black and brown women, which was pretty amazing. So that's how I got into research, or having research be a part of my day job. I'd love to quickly hear about you all and how you all came to research. So John, I'll start with you because you teed up the question a little bit. Can you just share a little bit more about how you came to research?

    JW: Sure. Yeah, I think I started studying English and psychology and I'm also really into music. And I think all of those worlds congealed, like I said in this world of just interest in humans. And I stumbled however, accidentally into actually the psychology of music, which is something I researched in grad school, and then discovered marketing research. That was the beginning, I guess, of the commercial phase, if I can call it that. And I've tried to keep that human bent, even as I've continued into web design and testing and those sorts of things where it's always, John the human, the musician, the sociologist, the anthropologist, but also the researcher. So I try to keep all those people alive in me.

    JH: I love it. I love it. We didn't ask you to bring your trumpet today, but maybe on a later, different podcast. Tchicaya, can you share a little bit about how you got into the amazing role where you're doing global studies? I mean, just really tremendous work and impact.

    TR: Yeah. Thanks. I do feel like a short changed myself at the beginning. So I'll add just a little bit more here. It's so funny because the more people that I have met, I'm going to say, since the pandemic, I just discovered so many people in the area and I hear it just like John said. Well, I happened into it. Well, I purposely got into research. When I was really a psych major in undergrad, I had just some phenomenal folks around me, some phenomenal professors. And this is going to sound really shallow and bad, but I promise you I'm a good person. So my professor was the world renowned sexual harassment researcher, but she was an IO psych, so industrial and organizational psychology. When I started my master's degree program, I didn't know that there were over 100 disciplines in psychology. I'm like, "What now? Well, which one makes the most money?" Okay. So I'm like, I can't be a struggling psychologist, because I'll hit it, I'll make a whole lot of money.

    So the two things that I needed to do was, get my PhD, right? And then I needed to get into a field that was more closely aligned to business. So the truth of the story is, I specialized in industrial and organizational psychology, but along the way, I focused and specialized in measurement and psychometrics. And we had this really cool opportunity to do a really deep dive into loyalty. So I have a really big focus toward the end that formed the basis of my dissertation, which was in consumer psychology. So I have had just the tremendous blessing to do both organizational psych and consumer psych in every role that I've had since grad school. So I'm a glutton, I guess, for punishing because I chose this career and I absolutely love what I do every single day.

    JH: Well, that's all right. I mean, to love what you do, but also get paid relatively well to do it, you did both in one fell swoop. And I'm sensing a theme here with psychology, and sociology, and some threads here. Morgan, how about you? How about you share a little bit for us?

    MR: Well, I think part of the reason I found myself in this particular role is because you Jessica and John took a chance on me and have given me a seat at the table. But really from a macro perspective, I think I found myself in this place because I had this realization that as strategists and designers, we really can't make decisions that impact millions of people based on these shortcuts that just exist in our hand. So my goal, if you will, is that we can give some of that power back to the people who are using these experiences that we're creating. And I think looking to protect their role or position in that process is something that's really important to me. So that's why I'm doing this for now, at least, I'm still pretty early on and interested to see where it takes me.

    JH: That's great. Thanks, Morgan. And finally, Leslie, you have been in research, and data, and intelligence for a while, but you also had an amazing career even before that. So tell us a little bit about that and how you came to be where you are today.

    LM: Yeah, thank you. I definitely resonate with everyone else. I also studied in college in a sociology background. I really enjoyed that. Ultimately decided, I don't want to do the PhD. Worked outside of that for a while, and then after a while I said, "You know what? You've always enjoyed writing." I decided that I wanted to become a journalist, went to grad school, really enjoyed that. And then eventually landed as a professional, digital and print journalist for National Publications, including a money magazine at Entertainment Weekly, really enjoyed that. And after a while, I realized that I gravitated to a lot of the data or the research's driven stories. Some of the editors would throw it to me and say, "Well, put Marable on that." And I really realized that I actually preferred that. The tying the story together, using the data, using the illustrations.

    And so from there, I made a concerted effort to choose roles in media or assertive companies that looked specifically at insights and data-driven information, for example, at Nielsen, when they had their early net ratings company. And from there, I transitioned to more specific heart analytics, ethnographies, a survey work. So now I look at myself as a data journalist. I really enjoy that journey of helping people to consume information, helping them to figure out what is most important for them to understand, and really it's about them and really advocating for themselves. So I'm really happy to be here.

    JH: That's perfect. I think what you all have highlighted even with the thread of the psychology, sociology. For me, is that, there is the hunger and desire for continuous learning, right? In consuming of information. And how do you distill that down and then push that back out. Knowing, and having worked with you all personally, and in many ways knowing also too, that we're very thoughtful about that human-centered perspective about giving voice to the consumer or the customer to underrepresented communities. And so I think that, we also bring that perspective and lens to the work, but we're also highlighting, people that come into research or are connected to research are very much coming from different backgrounds and perspectives. And we know with that, also comes bias in terms of how we approach our work and what that work ultimately impacts.

    And ultimately what's going to show up in the marketplace and in the world. So let's take the audience on a little bit of a journey with us as we really dive into some of the key areas that happen through research. And we know there's quant and qual. I know you all have a varied background with quant and qual, and all of the things in between. So, love for you to bring that to the conversation. But we're going to talk about what it means to brief for research, talk about designing the research plan, data collection, synthesis, publications, framing of the work, we're about to get into all of that. First, let's start with the brief, right? A client, a customer is coming to you, we are trying to launch X product. We need to understand more about X audience. We are trying to make a pivot in the marketplace. We need to do some research. Sometimes those clients know what research actually is, and sometimes they don't. And so they come to you with a brief that might be just in their head or a brief that's actually written down and well thought out.

    Let's talk about that first part of the process, right? Because whomever's crafted that brief is coming with the set of preconceived notions, preconceived biases. We know that systemic racism has weaved in threads through all of that. Talk to me a little bit about when you get a brief, what are some of the immediate questions or what's that level of discernment that you go through as you are evaluating, what is being asked of you and what you think you need to do? Let me start with Tchicaya.

    TR: Thank you, Jessica. So I think that this is a big question. Because a lot of times, people come with their own biases. It's just natural, right? As researchers, I think we've got to figure out how do we contain those biases so that when we deliver the end result, it stands on what the research says and not how the researcher is trying to bias it, or determine what that looks like from a researcher standpoint. So for me, the first thing I like to do is, understand what is the headline? Right? You got this brief and you think you want to do X, but you may really want to do Y. And I'm a self-proclaimed quant geek, right? I'm just going to say that right up front, because I do believe in data, I believe in numbers, I believe in, you know what I mean? Regression analysis. And I think that if we understand really what is the ultimate hypothesis?

    And again, we're all social scientists, right? So I think we have to start there, because it's really difficult to do a good job if we don't know what we're testing for. I mean, if you'd just dig into some of the details of research, all of your stat testing is based on the fact that you have a hypothesis. Whether or not you're trying to test, disapprove or approve that hypothesis, or prove that hypothesis is important. So I like to get into, what is your hypothesis and what is the headline? Because for me, if I understand what the headline is, I can better serve your interests and your research needs more than you even thought to think. Because a lot of times the people come with the brief and say, "I'm going to do three focus groups, and I want to just know X, Y, Z."

    Well, I think it's our job to say, "Well, have you considered?" Maybe how race, if we're going to talk specifically about race and bias. Have you considered how race might moderate that relationship or have you considered how one audience may take what we're giving. If say it's a product, take what we're trying to develop differently. Maybe their context is differently.

    So I think the opportunity to influence and de-bias the process, happens really early up. We can't think about it on the back end when we start analyzing the data by race, we have to think on the front end, because there may be questions that we have to consider based on the audiences we're serving. So, I think that conversation upfront and really honing in on what that headline is going to look like. Because if I say, "80% of everybody is going to love this product", they love it. But maybe there are some differences within that, that mitigates the love that they have. And a lot of times that is as a result of even ethnicity or racial differences that people just don't come to the table thinking about initially.

    JH: Yeah. Okay. That's great. Morgan, you've had an opportunity this past year to work on some global projects. When you're getting a global brief, what are some of the things that have a global testing or audience or request and brief, and it's going to have an impact, and you talked in your setup about being thoughtful around work, that's going to ultimately impact millions of people. What are some of the questions that you were asking or even thinking about as that brief comes in from one of these clients?

    MR: I think some of the first questions are actually around our self-awareness at Instrument. So how much of this work are we going to take on totally, independently? And where do we need to engage in-market experts that can really authenticate the work or just help us know what we don't know. And so I think that would be the big first piece and making sure clients are receptive to those types of partnerships. And that we have the tools and the time to really onboard folks, not just to what we want to learn, but maybe the designs that we're creating. Because when we're putting together the questions we're going to ask, it's really no good for me to make a mod guide and expect that it's globally applicable. We actually also need those folks in the room helping shape the questions that we're asking, seeing as they know what's most relevant in their market.

    I think part of working with clients on that too, is the question, "What do you want to learn?" Is never easily answered. You got to shake it out of people sometimes, and I've found that almost a workshop conversation, if you will, at the kickoff, can really get people thinking clearly not only about what they want to know, but why do you actually want to know that? Why is it valuable to the bigger picture? So I think that's one part of it.

    The other piece is the context, like Tchicaya was saying. We can't come in and speak with folks about, just what our brand is creating or it could even be the topic of there's a lot of brands, we're getting briefs, at least at Instrument, where folks want to show up and speak on oppression and systematic racism. And they want to really do it the right way, so they say.And often, the second part of that brief is really not just what do they think of what we're creating relative to these topics? It's, where's the consumer context on this as a whole. It's a bit tone deaf to show up and just start creating conversations around this really, really small piece that your brand is responsible for. So I think zooming out and setting that context could be a really important place to start. Both with clients and participants.

    JH: That's great. As we're continuing down this journey, Leslie, when you get that brief and it comes in, sometimes the clients are coming in and they have a pre-defined idea. And John, I want you to get winging on this too, after Leslie. They come in and we're like, "We want to do a survey and we want to talk to a million people." Okay, sorry, not happening. Right? Or we want to do a bunch of focus groups because those are fun. Well, before it used to be, you sat in the room behind the wall and you had snacks and you could watch stuff right in the room, but we're all virtual now. So that's a little bit harder. But how do you translate that brief? Asking those questions obviously into a research plan that feels complete, that also feels equitable, right?

    As you're thinking about the audiences that you've been asked to go after, as well as the audiences you're probably advocating for as well, how do you go about, really getting to what is a good research plan?

    LM: It very much so matters, and I'm sure all of us have been in that situation where data is power, data is politics and whomever owns the message owns the politics and the power. Right? So, it's helpful for me to probe a little bit to figure out, "Is this person the ultimate decision maker? Are they a participant? What is the hierarchy within their organization? Because ultimately, whoever is commissioning the project, if you will, they're the power center. So that helps set the stage. Now to the other piece, what I like to know is, once I get a sense of what the brief is, I like to get a sense of, what are your business goals? And that could be different based on the role within the organization. If you're the marketing person, your business goal is going to be different than if you're sales, or if you're doing research and development, for example.

    So I like to get a sense of what they think that is. And then based on that, come back and say, "Okay it sounds like you're trying to reach these types of people, I might suggest that you use these different methodologies." And also I'd also get a chance to get a sense of, what internal research have they already done? So I can assess that, incorporate that, within a research and development plan.

    JH: That's great. John, love to get your thoughts on your approach to a research plan and how you're also balancing all the things. What can actually and physically be done with the clients needs, desires and business needs. And also again, finding ways in which you feel like you're bringing equity to that process.

    I'm sure all of us have been in that situation where data is power, data is politics and whomever owns the message owns the politics and the power.”

    Leslie Marable

    LM: Yeah, that's really a great question. I think that it's really an evolution. I think there's two pieces that I look at and they're both investigative and they probe. I think the initial thing that I like to do is figure out who's writing the brief, what is their role and what is their goal, their agenda? Oftentimes I find that it's helpful because it will tell you not only what their expertise is, but will also give you a subtle clue as to, what are they maybe not so good at, for example, or maybe there's a gap in their personal expertise that may be helpful for you to know, for example about their gaps. Because you can play to that, you could say, "Okay, maybe they're not an analytics person. So I have to over evangelize and articulate why this project is important or why we might need to explore several different approaches."

    So I think number one, understanding who wrote the brief and why, and what is their agenda is really important to know. And this gets into the politics a lot and we don't have to go there, but essentially I think it's important to know.

    JH: And it's okay to go there because it matters.

    JW: Yeah, I think first, I just rubber stamp Leslie's points about knowing your audience before they think they know their audience. So there's this whole step where you have to know that they're coming in with some form of bias, however, innocuous it is. And you have some too. I mean, we all live in our clothes, so we have to start where we start. So I think there is a moment where especially in a time crunch this can be difficult, but there is a look at the macro that says, "Okay, you've told me that this is your audience, and they are this gender, and they're this age, and they're this ethnicity. But let me look a little higher with you about, sure that's your target, but what is your audience two, and five, and 10 years from now? And is this product for them or is it for everyone?"

    And there's a 10,000 foot picture that I'll admit I think in the business world, we push past a lot, because we get the brief, we say, "Sure, I can run three focus groups." And then you go run three focus groups. That's the path of least resistance, right?

    JH: Right.

    JW: They asked me to, and I can. But there is a moment where you have to know them, you have to know their audience maybe better than they do, and take that moment. And I don't want to set this up to be a fight between you and your clients, that doesn't help anyone. But in order to push this whole idea of research past using the same folks to ask the same questions, which we've done for the last 100 years, we have to take that moment to say, "Could we do this better? Could we do this broader? Could we do this blacker?" There are a lot of questions we can ask in the brief to prevent the garbage in, garbage out. Sure. I can prove your point. I could prove any entrepreneur's point if he wants me to, but why? So I don't want to sound like a nihilist, but there are ways to do research the right way, and there's ways to do research that you're asked to do. And I think I'd like to do the former more than the latter.

    JH: Tchicaya, I see you nodding your head a lot over there. Anything to add on that?

    JW: I just have to say, John, you hit the nail on the head. As a researcher in this day and age, in this environment, you have to put your boxing gloves on and be ready to fight. For real, for real. Because we can keep doing the status quo, but then we'll keep getting what we're getting, and none of us will grow. And I think all of us want to do the right thing, but it is very difficult. You have to decide that, today is the day I'm going to fight, right? Because it is a fight. And sometimes you just get tired of fighting and you want to take your gloves off. Now that I think there is a stage set, right? There is an appearance at least, and maybe there's more behind some companies. But at least there's an appearance of wanting to do the right thing. Like Morgan said earlier. People want to do the right thing. They really do. But do you want to do the right thing because it looks like the right thing, or do you want to do the right thing because you really want to know the truth?

    And so I think the fight is so important. And we've got to start to come together really as an insights industry, so that we're not at least fighting each other. And the fight is about, how can we advance the truth that comes from a narrative that has not been told before now? And I think to discern the difference between, "We want to do it because it makes us look good versus do it because it's the right thing." And oh my gosh, there's a whole world of knowledge that's opening up, that we were never open to hearing before now. So I really appreciate the point about that, and I really appreciate the point about what Leslie said, I'm sorry. I just got to mention it because I wrote it down. But, whoever owns the message has the power. You can't even imagine how you struck my soul, Leslie, with that.

    Because there's a narrative that we all want to tell, but who's writing your check? That's the narrative. We have to balance between doing the moral thing and doing the thing that's going to keep you employed. And that's not a good place to be in. So I think to tie the two things together, if we're fighting in the right place, and people's hearts and minds are in the right place, the power becomes less of a barrier, right? And becomes more of a welcome agent of change. And so when we can get the power to be aligned with the truth, then I think we have an immensely powerful opportunity here with research.

    JH: Yeah, no, I agree. I agree. And one of the things that was kicking around in my head listening to all of you talk, is the truth can also equal money, right? In this capitalist society. Right? You get a lot of clients that come to you and like, "I want to just look at white suburban moms." Okay. Well, black suburban moms make a lot of money too. Right? "I only want to sell this to rich white guys, so can you go research rich white guys?" It's a lot of Latin X and Hispanic folks, they have hell of a lot of money that would love your product and be willing to pay for it. Right? So part of my tactics in the past have also been like, "Let me expand your pool or your data set. Let's look at more audiences because I guarantee you will be surprised, and you might even potentially open up a new consumer demographic for you that will help you leapfrog your competition, who are also only looking at white suburban moms or rich white guys that own yachts. Right? And give you longevity, viability, stability for years to come."

    For me, I think we also, as researchers that are doing this work, also have to say, "It's a little bit different from just the traditional diversity, inclusion, equity and belonging, because it's morally, "You should do it because you should do it." But I'm like, there is a financial underlying benefit here, right? That I think is important for us. And I think that's why going all the way back to the brief, and the person, the owner, understanding those business needs is really important for us as well to tie that all together. So this is great. This is good. This is all the juicy bits. So let's dive in just quickly on quant. Right? And then we'll get to qual. But I want to dive in on quant. Tchicaya, I know you're a quant nerd, and John, and Leslie.

    Because quant can be everything from a really small survey, small panel kind of thing, to going very large. How are you making decisions? And a part of it goes back to that research plan, but even as you are putting the panels together, and making decisions about who's in and who's out, how are you approaching that process?

    You have to start from a place that puts people in boxes, right? That’s how zeros and ones and quantitative surveys work. If I take a survey, I am this color, I am this age, I am this gender, it’s just how it categorizes me. And so, we as survey designers and quantitative researchers have to understand that, we've started by making boxes.”

    John Whaley

    JW: Okay, I'll try. I think the challenge with quant is that, we are in a science of segmentation and in a sense discrimination. You have to start from a place that puts people in boxes, right? That's how zeros and ones and quantitative surveys work. If I take a survey, I am this color, I am this age, I am this gender, it's just how it categorizes me. And so, we as survey designers and quantitative researchers have to understand that, we've started by making boxes. There's a little inevitability there, but we can do that right. And we can also do that with open boxes. This is really small, but we've started opening up our quotas to 40% male identifying, 40% female identifying and then 20% fall into any other bucket. So if our client wonders how do males like this, they have enough data there, how do females like this, they have enough data there. But then the other 20% falls out, however it falls out. Right?

    So, we have opened a box in that way and let that category be what it will be. And it might end up being 49, 51, we don't know. But that is a quantitative way to allow gender, to be the thing that gender is in the world. It is for real, right. We don't have to make those boxes with lids in a way. So there's a way we have to design surveys knowing that we start with those boxes, and then we ask the questions. So there's both those steps have to be done with a ton of care, knowing that we've introduced bias almost from the get-go. And then at the end, we come back with those zeros and ones, the actual data, we have to recognize where we started. So, Jessica, where we started this was, how important is the brief? And the answer is critically important. Because on the end, all we have is the dataset.

    And that's not telling us anything about bias. It's just zeros and ones and that feels pure, but it's only pure to the extent that the survey itself is designed well. So lots of cautions in there, but it can be done. I'll just say.

    TR: Let me just tell you, I'm going to follow John every time. Okay. So I think the boxes are critical. We start out, right, with biases. Because we have to. Because the biases, I think have to be built into the research design. And a good researcher knows that they exist. The question is, how do you limit them? Right? And so I really love the box analogy. And it's so funny because I would bet that, most of the researchers out there would set this up, and there's a little bit of, "Well, I prefer not to say what my gender is." And then what do we do? We take them out of the sample, right? Like they don't exist. So I really love the fact that you say we let them in, and then we actually going to research them. We're going to analyze them as well, not just identify them so we can remove them from the sample, because guess what? There are real people that sit in those boxes.

    And I think what we have to be more comfortable, even me, what we have to be more comfortable doing is understanding the context of that other box, and understand what challenges, what is their context? How does the product development differ, because they sit in a different box that we're just not as comfortable or used to talking about? And in today's day and age, there are so many freaking boxes, right? There's too many boxes. But I love how you did that, because I'm guilty of identifying folks in that box and then saying, "It's not the male column or the female column, it's a different column, where do I put them? Are we going to rebase the sample and take them out? I mean, these are challenges that researchers are dealing with every single day. And I think we're starting to get more comfortable identifying or learning what the common language is to identify the boxes, but are we really now taking the boxes and analyzing them, and trying to understand, what does it mean to sit in this box?

    So I think quant is so critical because at best, we are trying to, as psychologists, right? As behavioral scientists, as researchers, we're trying to predict what people are going to do. We're trying to predict what they're going to like, and how they're going to think and feel. So quant to me is the tool and the instrument that allows us to do that. So if we don't get the boxes right, then there's no way we can do it effectively. Right. And so for me, it's just such a powerful tool in our research toolkit, the quant piece, because it tells the story of everyone as long as you've done right. Versus some of the qual techniques, which I'm sure, Leslie, we're going to go talk a little bit more about.

    JH: Yeah. And that's so great. Part of this is dropping knowledge and giving advice to each other, into those out there. I love the idea of, let's start to open up some of these boxes and allow people who identify as trans or who identify as non binary to be able to be counted and to have an impact. Right? And even as I think about race, we are still very much in just the boxes of the census data. Right? I'm like, "What box is Kamala Harris checking?" We know she considers herself to be African-American, a black. Her father was Jamaican, right? We forced her to check boxes. But she also is of Asian descent as well. Right? And so how do we make sure we don't exclude a Kamala Harris or a Barack Obama because they check more than one box. And many of us check more than one box, and we will see when the census data comes out, in the next six to 12 months, that there are a lot more people checking a lot more boxes.

    So I love that idea of pushing our clients and pushing ourselves to create more open boxes. So let's hop over to the quant side of the house, as we think about those more smaller and intimate settings, as we think about more targeted and focused. It sometimes can be everything from what type of cereal did you eat this morning for breakfast, to which leg do you put your pants on every morning? Right? Could be very personal. Let's talk about, as you're preparing and thinking about both selecting audiences. And Morgan, you touched on this globally a little bit around, if we're interviewing people in Japan or in another country, is having someone from that culture or perspective going to have more? And will that influence the outputs that you get?

    So I'd love to hear. I'll start with you, Morgan, and then Leslie, you can then add on. Can you just talk a little bit about how you are approaching the questions, the focus groups, or even some of those one-on-one conversations?

    MR: Definitely. The answer is yes. It does influence. It definitely influences the feedback you're getting back. I really think the planning phase, when you're preparing a mod guide, if you will, is the most important part. If you're not getting it right in the planning phase, with screening and recruitment, like John and Tchicaya just touched on, there's no point in even moving forward. And so, really when I'm developing a mod guide, I'm trying to create space to learn from both participant’s intentions and their actions. So it's not just how they're completing certain tasks, it's really about getting a sense of their motivations or that underlying relationship with the subject matter.

    And I think part of doing that is providing a safe space for people to share their perspectives and really deciding which questions am I going to ask? What order are they going to be in? Am I going to give them that floor time to set their context and speak on this subject independently of whatever design Instrument might've cooked up that we want to share with them right after?

    And I think really maintaining self-awareness throughout that process. How is this impacting people? How are they responding? Can they even speak credibly on this topic? Do they want to speak on this topic? And knowing when to adjust accordingly. I think when you put it in a global context, I think where, a learning for me quite recently was, you come up with the mod guide, you brief in the local moderators, and really, we shouldn't be moderating conversations that are about a localization strategy at scale globally. We just shouldn't. You need people that live in those countries and that understand the nuance of the culture to really be the face of it. Pulling that out of consumers.

    And so, I think the learning for me was not just briefing in on the mod guide, but really taking the time to onboard to the design work, and really let them know that there's space here to add questions, there's space here to totally remove questions If this isn't making sense. And also the awareness of the time that we're in. And how the types of projects we take on and the types of questions that we ask or even just the folks we're doing research with might have to adjust accordingly.

    Maybe we were only including folks that had a full-time or a part-time job. Well, we're in the middle of a pandemic, so we need to open that up...maybe letting some folks in that are recently unemployed. Now the clients want to have a conversation about how some fundraising portal they're making does X, Y, and Z. Well first, let's talk to folks about, do they care about this topic? Do they want to invest in this topic? And so really for me, it's just the ever-changing context of line-by-line being in the conversation and what's going on in the world, and just staying in tuned with both of those things.

    JH: Oh my goodness. I have done several talks around, that's macro context, right? We as researchers, we would operate in a vacuum and you sit and answer, right. But there's so much other things going on in the world that are impacting even us as researchers every day. Well, we're showing up at our jobs in our homes, we're all at our respective homes. Right? And I'm letting some other stranger in my home as I'm interviewing, right? There are so many things that are impacting and affect us. And I think we like to think of research in a vacuum and it's not. And I love that perspective of being malleable and needing to adjust or shift if you have to. Because that makes sense. Leslie, let me let you add onto this.

    You have to put your boxing gloves on and be ready to fight. For real, for real. Because we can keep doing the status quo, but then we'll keep getting what we're getting, and none of us will grow.”

    Tchicaya Robertson

    LM: I am so loving this conversation. This is fantastic. I really wanted to add to what Morgan had to say primarily from the qualitative experience. I think as researchers and analytics people, we're navigators we're interpreters, if you will. And we can help to push clients whether they be inside or external to try new things. And so, for example, I've had several experiences where there was a traditional client and they're used to doing the traditional survey and we've always done it this way. And then you're thinking, "Well, you say that your challenge is this, this is the problem that you want to solve. And these are the audiences that you want to read. Have you considered A, B and C?" And I've actually had the pleasure of being able to sway a client, whether it be internal, external, from what they wanted to do and what they said they're going to do into something new.

    And I think that's really important that we do. That we actually see, or we guide clients as well. The other thing that I think is important is, really the role. When you look at more of a qualitative experience, you really have the opportunity to influence diversity inclusion. So when you look specifically at ethnographies, and I love ethnographies, or focus groups. This is really when you can help the client understand, why diversity inclusion is so important because you're actually talking to people who are going to look like the people that you're trying to reach. Right?

    And so a lot of those inherent biases of like, "This is what our client looks like." It melts away when they start to see that people are not that little box or persona that they've created perhaps out of thin air, that they actually look like different people. Right? And to your point earlier, Jessica, that they do have the income, right? And so, you can suddenly get them out of their comfort zone and their biases by having that actual human being in that physicality. The other piece that I wanted to talk a little bit about is, as it relates to the role of the analytics or researcher. I've been in a number of pitches or discovery meetings where, let's say the salesperson, that a biz-dev does all the talking. And I felt that it really wasn't instructive. But really when the analytics or the researcher was in the room, they were able to prove and get far more impactful information, which helped the agency derive great insights or really great research plan.

    So just wanted to shadow to those analytics people that, push to make sure that you're in that pitch. Push to make sure that you're in that discovery session with the stakeholders, because you really are going to bring that value, that extra dollars and cents that was not intended before.

    TR: Yeah. I just want to add this. Somebody told me once that, common sense isn't so common. Let me just say this thing. And this is the thing that I keep seeing over, and over, and over again. Because Leslie, when you say we're translators, man, oh man. Are we ever? Let me give you an example and I'm sure everybody's heard this, and with this environment again, right? The backdrop of these pipeline discussions, we can't find black. So you're trying to recruit at the HBCU, right? Okay. Now, I didn't go to an HBCU, I didn't. I went to what we call a PWI, a predominantly white institution. But don't send the white lady, right? To go to recruit at the HBCU. When you explain that, it's so critical for you to understand, you might not be able to connect. You can't translate. So when I think about that, and it doesn't say all of your recruiters need to be black, but there is a connection, a language, there are some questions that I'm going to ask a black recruiter that I probably don't feel so comfortable asking a white recruiter.

    Now think about a research group, right? A moderator going into, let's just say, I know that there is a challenge with, this is a real problem. With black teachers, right? There's not enough black teachers. And the research says that, if you want to encourage, and motivate, and reduce children that are black, right? From dropping out of school, you need to have a same race teacher. So if it's a Latin X child, you need a Latin X teacher. It's going to reduce that. So when you think about, who do you need to send into the room to talk about these issues with black teachers, about why they're not staying in the field, or what can get me motivated to stay in the field? And then, the surprise when it says, "We might want to get a black moderator." Because there's probably issues at play that a white moderator isn't going to understand to ask, right?

    There's different things that I'm going to even share. Because I know that perhaps, and I'm not picking on you, Morgan, but if you send Morgan in a room to talk about my blackness, and she's supposed to go and interpret that to the folks that are making the decisions, you're going to lose some contexts. You're going to lose some things in the process. And I think we have to be more careful about how important it is. Just like a black medical doctor. There's things that my black medical doctor is going to connect me differently on than maybe my white medical doctor. And I hate to even go here and say this, but it's so culturally important to have that alignment in life. And then when you think about having that alignment in research, it's the same exact thing. So how can you truly extract the knowledge and tidbits and understanding when you can't necessarily connect with your audience, right?

    You can't connect with the person that you're trying to glean all these motivations and things that we have to glean out of it. So I say all that to say, even though I'm not blah, I don't like qual. Don't make me do qual research please. But if you do, I think those of us who are making the decisions about the research team that does it, or the moderator that does it, we have to understand that it's so important to send into the ground the people that can understand who's on the ground and then use that to your benefit. I mean, you deserve that. You pay for the research to do it right. And I think we're still not quite there yet, when it comes to making sure all the right people are in the room.

    And it floors me how that isn't just a thing that we should know. It's just something that we should know, but we don't know, but it's so important. I don't know. I just wanted to add that. You're not going to get that flavor out of a survey, so to speak, but it's so critically important to have that match up when you in a qual situation. I just had to add that.

    How do you create safe spaces, right? Spaces for people to be able to share more.”

    Jessica Hartley

    JH: That's great. Because if you're not talking about cereal, but you're really trying to get into some of those nitty gritty topics. We even talk about in work and in our environment, how do you create safe spaces, right? Spaces for people to be able to share more. So if you're selling cereal, you're probably flying. But if you're trying to get to something a little bit deeper or closer to home, yeah. Having that cultural relevance I think is absolutely important. So let's take us then to the end of our journey. And I know we are getting close on time. And let's talk about that narrative and storytelling, right? So we've gone through this journey of talking about the brief and the audience and the plan, and now we've conducted qual and quant and all this yummy goodness of data that we're now getting in, and now we're obviously responsible for gleaning insights, and observations that ultimately be the catalyst for what our clients and our people do next.

    And there's always a narrative, a story, right? And we've talked about a lot and you guys have already. We've been talking about the narrative and the story. And I feel like, sometimes it's reality versus fake news, right? In terms of what we're trying to push forward. So let's do a quick round robin of, how do you do your best to maintain the integrity of the brief and what you've been trying to achieve with what that final finding, or output in that narrative and story is. John, I'll start with you first.

    JW: I just think she's better at this than I am. But I think the key at this point, I'll start where we ended is context. So if you've been on the journey, if you are a part of the brief, if you know the client, if you know the audience, if you've been through the creative, if you've seen the things that have gone into research, at this point, you should know and the question is, can you, even at this point, get over the biases? But you should then know what the story becomes. So it's a bit of a natural evolution though again, you need to be sure that your blinders are off at this point, and you're not so deep in the work that you can't see some angle that you didn't anticipate. I considered those moments, the reward for having done good work. And I'll admit that this isn't for me, the most exciting part, and that's why I want to hear Leslie's input on this because I love getting good data, I love the work that it takes to go get good qual and quant data.

    And there are times when I look at even strategists at Instrument who take that story and then make it something. And I think that's a little magic to me. And my ideal is, I can get them great findings and great data and my strong suit, and I'll throw it soon to Leslie, but isn't then taking that to, what does it mean for your website? What does it mean for your product? What does it mean for your people? But if you've done well, it's not so hard, but Leslie tell me I'm wrong.

    I'm going to tell you what you really need to know, not necessarily what you want to know.”

    Leslie Marable

    LM: No, I completely agree. What I was thinking of is really complimentary. I'm going to tell you what you really need to know, not necessarily what you want to know. Sometimes that could be political, right? It could be, how deep do I go? And that's why I think initially, it's important to the degree that we as data, and analytics, and researchers can have that relationship with the brief writer or the stakeholders or the clients, so that when you have to deliver sometimes unpleasant news, or news that they didn't anticipate, they can absorb that and they can marinate on that. But I still think in order to service the client well, you really have to tell them what they need to know, because then they can learn from that experience and move on with that.

    I also think that it's really good to base any of your decisions or whatever your narrative based on data driven. In other words, it could be according to what a quant research said, blank. According to the anecdotes that we derive from our 50 panelists or our 50 participants. It's really important to constantly tie it back to here are the numbers, and here's what the people said. Because if you say Bob the persona, then it becomes this vague generality of what they think or what they thought versus, Jessica said this or John said that. I also find that when you're delivering this information, let's say the brief writer is Susan. And if you say Susan, as you said at the previous meeting, and you really tie it back to what they said, it's a reminder that, yes, I was heard. That you listened to me. That number one. Number two, that you answered my question, and you're validating. You're validating that you heard me.

    And even if your results are different than what I thought, they feel like they've been listened to. And I think there's a piece there, I can't really describe it, but they have to feel listened to as well and validated, no matter what you're delivering. And I think that's important.

    JH: That's great. Morgan?

    MR: I think when it comes to, running that synthesis, I try keep myself in check by erring on the side of more participant quotes, rather than my own copy if I'm doing qual work. So I really challenge myself to tell that story, using the words that real participants used, and that made it come together in my head. Because really, I didn't just come up with it. So as little as I can do to really shape that, is important. I'm just the organizer of the information, I'm not necessarily shaping the narrative. And I think by pulling as many quotes and actual verbatims that we heard into that, is really just the most effective way to get after it. So that's just a thing I do with myself to keep us aligned.

    JH: I love it. Tchicaya, you've done a lot. A lot in your career alone at Accenture. How are you that final phase, the last mile? How are you approaching?

    TR: Yeah, well, let me just say that I am the least creative person in the entire planet. Okay. I am like, "Just don't come to me for that piece." And I think one of my challenges, because I am so hard in the zeros and the ones like John explained it, because I'm so black and white when it comes to data. I was challenged for a long time with really understanding how to best tell a story. And I'm still. I've been at it for a long time, but because I'm so much in the weeds, that was probably, I think the last thing to develop as a professional, to be totally honest. But I think that if you don't have the right story, everything we did up until that falls apart. And I think the story is so critical. And I believe when John said, "It's magic what they do back there and creative." I don't even know how they figure all that out. Like it's like, man, you guys just wave some wands and then voila, this research I did just look so much more exciting. So it is a skill. And I think one of the challenges is that we are looking at data, and we're not always looking at the implication of the story and how does it feel?

    And I think when we start to integrate more video into the research process, and we can put a face to a quote, and an emotion to a quote it just brings a whole nother level. So I really liked the fact when we start doing less PowerPoints, right? And more video, and more expressive insights. And I think that just takes the research to a whole nother level. Garbage in, garbage out. I think somebody said that earlier, but if you don't have that story, man, it falls. I'm literally working on a project now where trying to get the human element into the research, right? So that the message gets through, it's difficult. It's really hard to do. And I think even still trying to get that right is probably one of the hardest parts of the process.

    JH: Well, that's great. I mean, I appreciate that. And as one of those strategists who helps to turn some of those data and the insights into magic, it takes a long time to get there. I mean, it takes years of working and practicing and partnering with others. And I think also to our industries and our roles have evolved to be community and collective storytellers, right? It's not just what my POV, and my perspective is, which is a lot of what all of you have talked about, especially Morgan. And so we are collectively saying, "What does this mean for us?" In understanding the impact. So I appreciate all that. Well, we are almost out of time. I wanted to ask you all one final question. And even though we've talked about some of the challenges, and problems, and biases around research, and design, and data, and now ending with the narrative and storytelling and really talking about also how bias and systemic racism shows up in research.

    I'd love to ask you, what's your hope for the future as you think about research, as you think about the evolution of research? Especially as we think about how we support, and amplify, and include underrepresented communities. So I'd love to hear about what your hope is for that. I will start, and give you guys sometime to marinate. My hope is really, truly, as I think about all of these moments that have become a movement, I've said that a lot. Around diversity, inclusion, and equity and belonging, for me, I need more of us in research. I need more of us helping to be a part of the briefs, and the plannings, and the research, and the plans. I want the kids in college that are majoring in psychology and sociology that are kids and young people of color to recognize one that you can make pretty decent money, right Tchicaya? In research.

    MR: I'll jump in. I think for starters, my hope is that we could be vigilant and intentional about when we're engaging with participants, especially during this time. And for clients, I really think it's important that we continue to have conversations with folks that aren't from a marketing background, because we need to bring a reality check to this advertising dystopia that we live in. But more importantly, every time we launch a study, we need to be talking about representation and that's for both researchers and clients. So if the folks that are conducting the study have not been transparent about who they're talking to. And if that sample is diverse, we need to be asking for receipts because it does affect the results. And that is my final two cents.

    JH: Yes. Receipts. I have all of that. I'm doing a finger snaps over here. As for the receipts, yes.

    JW: Right. Love that Morgan.

    JH: Thank you Morgan. Leslie.

    Every time we launch a study, we need to be talking about representation and that's for both researchers and clients.”

    Morgan Rose

    LM: I have different approaches, but I think where I'm going to land is this. I really think we have our own mantra for the analytics researcher point of view. I think number one, it's important to more often ebb and flow with as the market or industry dictates. I think to what John said earlier, sometimes it's a little bit too regimented. And so if we can, in the future be more fluid and ebb and flow as the industry ebbs and flows as the market changes, I think that's one thing to be hopeful for. And I think the other thing is, what I consider my personal mantra. And I think it's really important as a researcher and a data analytics person is, never let someone else define who you are or who you will be. You were the architect of that journey. And if you can do that, you can better serve your clients and your brands.

    JH: Yeah. Oh, that's great. Thank you, Leslie. Tchicaya

    TR: Man, Morgan ask for the receipts. Let me just say, she's going to be my new research BFF and ally. I love that. Man, I love it. My thing is, and you all could probably imagine, I'm a handful. But I think you got to be authentic. When we lose the authenticity of who we are in our work, it is a wrap, right? And not enough of us are authentic. And being authentic, you could probably lose your job, but I think you have to let... So we talk about differences, right?

    But we're afraid to show them. So how can we be upset at leadership, about not embracing differences when we're hiding ours? And so what I had to learn early on is that, the authenticity and being intentional, that's my favorite word of all time. Being intentional in everything that you do. And I think that, as long as our leaders... Because leaders are influencing society, right? If they are open to understanding that there is something else other than the table that I'm used to sitting around, we are just generally going to be better off because there is truly a benefit in diversity. And my hope is that it becomes standard operating procedure and not a request on the backend to say, "Do you have a representative sample of all the different ethnicities? Did you consider including these questions like, that should be standard?" The question shouldn't be, did you include, because it's usually, "Oh, snap. We didn't include."

    So that should be standard. And once we start doing the research to uncover these important things as a standard, now we can elevate Maslow's hacker. Now we can elevate to something that then we have a conversation, but if we don't have the mechanics, we're never going to get to the conversation and all the nuances that we've just talked about today. So that's my hope, that it becomes standard operating procedure to make sure that our research has set up to identify and be inclusive with not only our samples, not only the people doing the research, but the leadership who guides the narrative.

    JH: Love it. That's great. John, I don't know how you gone follow that brother.

    JW: I am intimidated officially. The only thing I can think to do is, put together Jessica and Tchicaya's points, which is basically, recognize where research training is and research experience is. And I can see myself 15 years into my experience on a call with this set of people. And I look back at myself and say, "Okay, am I doing this right? Have I been doing this right? Can I do this better? Did I learn?" I'm a decent researcher, I'll say that, but I can evolve. There are things I need to learn, there are things I need to think about. And I think if someone is 18, or 20, or 25, or 30, or whatever they are in their career, if this becomes standard operating procedure, to your point, we're all better off this. This conversation 10 years from now is about, much smaller minutia and we're just fighting the last little fight because right now we still have a big fight. So those are a little optimism at the end.

    JH: I love it. Well, that's what we like to end on. And I don't ever want anybody to tell me that researchers can't talk, but who boy? I mean, we're clocking in on this podcast today. But I want to authentically and intentionally say, thank you Tchicaya, thank you, Morgan, thank you, Leslie, thank you, John, for your time, for your love, for digging deep and all the things. These are the conversations that we should be having every day, and we don't, right?

    We don't. And these are the conversations that we should be having with our teams, with our leaders, with our clients. Because to that point, Tchicaya, if we're not talking about these challenges, they will persist. I am so thankful for the time and effort and you all just sharing a little bit of your soul, and also a little bit of your nerdom with me today. This has been great. And I look forward to hopefully having all of you back more in the future. And I hope for our audience listening that this was insightful and helpful and maybe a little bit fun as well. So thank you all again for the time. And that is a wrap.

    Other On Air Episodes