Cale Guin: I am with Kathy Steinberg – Harris Poll. Hello, Kathy. How are you today?
Kathy Steinberg – Harris Poll: I’m good, Cale. Thank you so much for having me. How are you doing?
Cale Guin: I’m doing great. It’s more my honor, I’m sure, than it is yours, coming from the prestigious or prestigious, however you want to say the word, The Harris Poll company. What is your role there?
Kathy Steinberg – Harris Poll: I’m vice president on a team that specializes in research for public release.
Cale Guin: I’ve worked with Harris Poll in the past. Fantastic teams. Some of the things that we’re going to talk about today, I’m super excited that people get to hear about some of these services that I think we might have mostly thought that they were out of reach or not for small business and what have you. I think that you have some great options there, so we’ll talk about that.
The big question I think that everybody has, including myself prior to working with Harris Poll, is when we do research or when we think about doing research, it seems expensive. It seems like, “Ugh, how am I going to– It’s probably just going to sit on the shelf. Do I really want to do this? What’s the value?” How do we make it worth it at the end of the day? How do we make sure it doesn’t sit on the shelf collecting dust?
Kathy Steinberg – Harris Poll: That’s a really good question. Of course, for any small business, any money that you spend has to be worth the investment. Before you decide to do research, it’s really important to be very clear about what the goals are and what you hope to get out of it. I think if you go into research with very clear goals and objectives, then that reduces the likelihood that it’s going to sit on the shelf.
If you know what you want to do, you have ideas, you have strategies in mind, that’s one way to make sure the data will get used. In terms of research being out of reach, again, it really depends on what you want to do. A lot of research companies offer a wide range of services. Depending on who you want to talk to and what you want to do, yes, research can be quite expensive, but it can also be really cheap. We have a research platform that starts as little as 1,100 per question. You could get a lot of bang for your bucks if you can be flexible and creative once you figure out your goals and objectives.
Cale Guin: What do you see typically as the reasons companies are doing research? A lot of what you do, especially in your department I’m guessing, is, “I want a headline, whatever the goal is there,” but what other types of research are people looking for, typically, from Harris Poll?
Kathy Steinberg – Harris Poll: We do all kinds of research. As I mentioned, the research that I specialize in is anything that’s intended for public release. Any kind of research that is going to be distributed outside the coffers of the organization that sponsored it. That can mean anything from earned media and trying to get a headline, publication in a peer-reviewed journal, presentation at a conference, for businesses, for B2B. We see a lot of our data being incorporated into white papers that are then distributed to customers or prospects. We also work with businesses to do research that is not intended for public release.
If you want to figure out who to target, what kinds of customers to try to reach out to, traditional brand tracking, communication, testing. We can test messages and ads. We can do a wide range of things. Research can be used in a lot of different ways to help make decisions strategically and internally, as well as to communicate with external stakeholders by using the results publicly in a lot of different ways.
Cale Guin: Content is king. That’s what everybody is trying for right now. They want more and more content. Even just in the small amount of work that I have done with Harris Poll, which was a series of questions in a specific vertical that was just trying to dig into a little bit of the consumers’ opinion on that particular industry. You get a lot of data. You’re not just getting a single answer to a question, you’re getting a single answer to a question broken out by age demographics, location, all kinds of things. Talk to us about that a little bit. When I call Harris, how do you determine what kind of format this is all going to take and what I’m going to get in return?
Kathy Steinberg – Harris Poll: When we work with clients, depending on what you’re interested in doing, we know that we are the research experts. We’re going to consult on how to make the most of your investment, how to ask the best question. We’re going to start backwards from your objectives. For example, if the research is designed for public release, if you know that you want to release the results publicly, maybe develop an infographic or drive some earned media coverage, we’re going to ask you for your dream headline. You may have some of the question ideas, you may have drafted some questions, we’ll take those as well, but really, we want to know what is it that you hope to get out of the data? Imagine you get the data tables back, the results are exactly as you hoped they would be, what would the headline of your press release be?
Then we work backwards from there and we come up with a few question ideas. Survey design is often a very collaborative process, lots of back and forth, several iterations of questions. We may get a list started for you, but then really rely on you to help us build out the list of responses. Make sure that we are asking the questions in the right way. For example, if it’s a technical survey, like if we’re surveying IT decision-makers, we’re going to really lean on you pretty heavily to make sure that we’re using the appropriate level of industry or technical jargon so that we don’t rub survey respondents the wrong way and to ensure that it’s accurate to whatever product it is that we’re talking about.
Cale Guin: Goods thing you’re working with a dope like me in certain circumstances where the bar is not that high. You’re not going to need to lean or you probably don’t even want to lean on me for much. The great thing is the breadth of data that you get. Talk about that a little bit. Is it the same all the time? I’m always getting the demographics and the different tabs? Talk about that because most people aren’t going to understand or even know that that kind of thing exists. Help us understand that.
Kathy Steinberg – Harris Poll: That’s also a part of the value of the research investment. If you are thinking about doing research and you’re talking to multiple vendors, it’s always important to ask what you’re getting for the price that they’re giving you. Are data tables included, how many banners have crosstabs? We have an omnibus platform that surveys a nationally representative sample of 2,000 adults, 1,100 per question. For that investment, you get standard data tables with two pages of demographic crosstabs: age, gender, income, education. Our sample size is 2,000, which may be larger than some of our competitors.
It’s really important to understand what you’re getting for your investment. If you’re talking to multiple research partners, make sure that you’re comparing apple to apple. The platform that I know you’ve used, and we’ve been talking about this, this omnibus vehicle includes two banners of standard demographic crosstabs. If we’re doing custom research for you, a lot of that would be customized. We would work with you to understand what is it that you’re most interested in?
In our omnibus platform, we have a certain set of age breaks that we use: 18 to 34, 35 to 44, and so forth. If we’re doing a custom survey, it may be that we’re only surveying adults ages 50 plus, so we’ll have different demographic breaks there. Then, of course, we can provide a range of services, including full-service reporting. We can deliver analysis, as in an executive summary, do a full-blown PowerPoint if you need. We can assist with infographics or white papers. It really depends on what you need, what the goals are, and we can itemize all those things for you.
Cale Guin: I’ll say this, that when somebody like me gets that data, my eyes light up like it’s Christmas morning and, “Oh my God, I didn’t know I was getting all this and all of that. Then my head spins with, “Every one of these categories is something that I can use.” For the average person, one of the things that we’re going to be dealing with in this series of podcasts is going to be specifically that they don’t know all of these things could be things for them.
At the same token, they just need to know, “I just need to know this. I just need to know that. I get all of this data.” You guys will help them break it down. What you did for me, just a quick personal note, was the team I was working with was great. I told them what my suspicions were, and this is why I wanted to get the information. My suspicions were that in this industry, when they were reaching out to their consumers, that price was not probably as big of a hurdle for the consumer as they were thinking.
I wanted to go back to them and say, “See, it’s not that big of a hurdle. Let’s concentrate on some of these other things.” That’s where my angle was coming from. Your team comes back and they said, “Here’s your headline and here’s a potential headline.” They were fantastic. I felt like I could write my email, I could write whatever my sales piece was going to be and there it was. I love that. That was great. The other thing that I loved, and maybe you could talk about this a little bit because you have a number of different services, that omnibus platform, I think we put the poll out or the survey out on a Tuesday or a Monday or something and we had the information on Thursday or Friday. That is fantastic.
Kathy Steinberg – Harris Poll: Very turnkey and fast turnaround. The platform you use that surveyed 2,000 adults, ages 18 plus, one of our most popular research vehicles. We are in the field for only two business days. If we start on a Tuesday, data tables are available on a Thursday. Our consulting is included in the price per question. We can schedule a debriefing call with you for Friday and actually talk you through the headlines, answer questions about how to read the data tables.
Actually, in response to the pandemic, about a year ago, we launched an even faster turnaround vehicle. If you need results faster than two business days, which can be the case nowadays, we have something called Flash. It’s an overnight poll, one business day, and you get 1,000 complete for a nationally representative sample.
Cale Guin: That is fantastic. One thing that you and I talked about the other day, and I probably shouldn’t give away the fact that you and I have had all of these conversations already and I already know all the answers. I don’t know all of the answers. Let me just say this, that you had mentioned to me the other day that there are tools that you have that are free, and there’s information that you have already out there that people can go and look at the headlines in the data you’ve already collected. Where does that come from? How and why is that information there? How does one find it?
Kathy Steinberg – Harris Poll: Harris has a rich and long history of conducting research for public release. The Harris Poll was founded in 1963 as a barometer of public opinion research. We have been conducting public opinion polls for decades and all of that data has always been publicly available on our website, anywhere that you can find a press release, basically, as well if you just do a Google search. Additionally, in response to the pandemic, we launched what we call the COVID Tracker. I think, actually, this week is week 52.
It’s a survey that is conducted every week. It started pretty focused on awareness and attitudes around the pandemic, then about the vaccine as that started to hit the landscape. We’ve been adding to it as other events have evolved. It’s not just about COVID, we’ve also asked questions about the racial equality protests that happened last May and June. We’ve been asking questions about the election. That data is publicly available.
It’s on our website. There’s a section of our website called the COVID Tracker. If you go there along the right-hand side, you can scroll through and the data tables from each wave can be downloaded. That can be a bit overwhelming, especially to somebody who’s not familiar with looking at data. Especially if you look at the latest wave, we show the results for each wave. You can see a page with 26 columns. I think we can only fit 26 on a page. They roll over. We also have a newsletter with highlights. That really focuses on some key headlines and takeaways from that particular week’s tracker.
Cale Guin: How do you guys determine which topics you’re going to cover in those polls? If you go to the front page of your website, and maybe I’m wrong, maybe I’m missing that there’s a theme there, it seems like a wide variety of information and topics that, “Here’s the headline. Here’s the headline. Here’s the headline.” They seem wide-ranging. How do you guys determine those headlines? Why are you doing that?
Kathy Steinberg – Harris Poll: Partly, as a social good. We see ourselves as social scientists and serving a purpose for the media, for public policy decision-makers. Our research over the decades that we’ve been doing it has been presented to Congress as part of lobbying efforts and actually helped pass the Americans with Disabilities Act, so we know the power of data. We recognize the value that data can bring to helping a lot of different stakeholders understand the opinions of the public.
In terms of the topics that are decided upon for each wave, a lot of it is just organic brainstorming within Harris, among the people who work on the poll, and then laddering up what they’re hearing from clients, from the people that we talk to and work with, as well as various media partnerships that we have. I know that we have a partnership with Crain’s Business Chicago, and we do some very targeted surveys there. We partner with Axios on some research. I personally lead our media partnership with HealthDay, which tracks American public opinion on various health-related issues. We try to cover what we think the public and the media will be interested in.
Cale Guin: You brought up the fact that you guys have been involved in even some political lobbying-type data gathering and what have you. This brings up a very important question. Of course, I’m going to use politics to exacerbate the crap of this particular question. It happens to all of us and including myself. I did a poll with you guys, and I was looking for an answer. Then I get these answers. Anybody can look at any data and twist it any way they want, and they can make it say whatever they want. How do you, given those political polls, and how do we avoid that bias that we have? We’re looking for this data, therefore, somehow we’re going to find it even if it’s not in there.
Kathy Steinberg – Harris Poll: That’s a really good question. When I said earlier that we ask you for dream headlines and we’re going to help you design questions, working backwards from those, it’s not just that we work backwards to get you the answer that you want. We’re going to try and we’re going to work with you to ask that question in a balanced way, in an objective way so that it doesn’t get accused of being biased, or leading, or cherry-picking. We take a really strong stance on that throughout a project, during the design phase, in terms of coming up with the right way to ask the question using balanced scales, making sure that our list of responses is comprehensive, and that we’re not prioritizing just the one response that we know you’re looking for.
Then on the back end, when you get the data, we also make sure that you’re positioning the results in a way that is valid. When you do this type of research for public release, there are industry associations that have very specific guidelines and best practices for releasing data. That means, when you release survey results, you need to include a method statement that says, “How many people were surveyed? When was the data collection conducted? Who did the research? Who sponsored it?”
You also need to make the survey results available upon request. That means the complete questionnaire and the complete data tables. If you do something and put a data point out there, that means that once your competitor sees this, they have the right to see the complete survey results and make sure that you’re not misleading in any way.
Cale Guin: A government agency can’t just pay you enough money to, “Give us this answer? We’re looking to say this, help us prove it”?
Kathy Steinberg – Harris Poll: No. The short answer is no. We’re here to be a third-party, independent, objective research partner, and to make sure that research done is in a credible way, using the best methods available and the highest level of rigor that is needed. We’re not just here to give you the data you’re looking for.
Cale Guin: Very good. Let me just say that I get to see you when nobody else gets to see you, and you were damn serious in that answer. You were like, “No, our reputation is not for sale.” It was great. It’s appreciated. As consumers, and even as business people, we don’t know. I’m convinced in my own life that no matter how much we think we know, we don’t really know. We only know what we’re being told. There’s this whole realm of things that we don’t know about that given topic that tomorrow we could be like, “Oh, well, if I would have known that, I wouldn’t have formed my opinion yesterday.” I always feel like that. It does conjure up this, “Who paid for it?” There’s always this skepticism about the results. It’s good to hear that somebody is going to definitively say, “We’re not for sale that way. No, you just get data.” Now, the way they twist that and the way they put that out there is probably up to them and their own moral compass, I suppose. The other thing that we talked about the other day, which I loved, was some of the interesting polls that you’ve been a part of over the years. I’ll let you tell your own story. Then, if you don’t tell the one I think you’re going to tell, I’ll ask you about the one that I want you to tell.
Kathy Steinberg – Harris Poll: Oh, I’m so curious because we talked about it with you and I’m so curious which one it is that you’re thinking of.
Cale Guin: No, you go first.
Kathy Steinberg – Harris Poll: Let’s see. Some fun polls I’ve worked on over the years? I personally specialize and have done a lot of research in the healthcare space, which doesn’t necessarily lend itself to a lot of fun surveys. A lot of it is highly technical. We deal with patients with very low-incidence conditions. It’s important to be very sensitive when you’re talking about chronic health conditions. Every once in a while, I get to take a break from those types of studies and do some more fun things. I did a study with Chevy a few years ago about pickup trucks. We surveyed pickup truck owners across the country about how much they love their trucks, why they like to drive a pickup? We had a nationally representative sample, but we also had larger samples in certain states where we know there’s a higher incidence of truck drivers, like California, Texas, and Florida.
This survey was a quantitative survey. Most of the questions were closed-ended. We also had some open-ended fields. We don’t often ask these types of questions in research intended for public release because, when we asked for dream headlines, typically what we get is, you’re looking for a big percentage, you’re looking for a number or some big number. With open-ends, you’re not going to get a number or a big number if you code the data. We asked pickup truck drivers if they’ve ever named their trucks and what their trucks’ name was.
Then they put together a word cloud depicting the results. The word cloud was in the shape of a pickup truck. A lot of the names actually had some color in it, like big blue, or black Betty, or big red. Big red was actually a pretty popular one. That was what the word cloud looked like. The word cloud was a red-tinged pickup truck. The words in it were representative of the number of people who said it. You saw a lot of Bettys and a few Joes. That was one of the more fun surveys that I got to work on recently.
Cale Guin: You won the set of steak knives because that is the one that I was thinking of. I loved it so much because one of the things that I thought of was that, I don’t know to ask that question. I would never have thought to ask that question. When you told me about it, I’m like, as a marketing guy, I’d like to design a whole campaign out of just that information. TV would have somebody padding blue Betty or whatever, and just talking about their love for their vehicle and how that vehicle got them through the hard times and whatever the thing was. It’s just so awesome. I don’t know it. I might be overreaching when I ask this, but can you help us find those kinds of questions like that because, to me, it’s never anything I would have thought of. I very seriously need this specific type of information. Hey, while we’re in here, why don’t we ask this irreverent, fun question and see what happens. That to me is fantastic.
Kathy Steinberg – Harris Poll: That’s exactly the stuff that tends to come out of the conversations that we have with clients during the design phase. I’m not sure if that was a question that the client came to us with. I don’t think that that was included among the dream headlines or the draft questions that we started with. I think it’s something that just organically came out of the conversations that we had during the design phase. We were reviewing the second draft questionnaire. We decided to cut one question. We decided we really like this other question and, “Can we go down this road a little bit deeper?” As you have three or four or five people on a conference call brainstorming, that’s where the magic happens in survey design, not to sound overly hokey about it
Cale Guin: [laughs] I’m all about hokey. Let me just say that that’s the kind of thing where you’re getting information that you probably wouldn’t have seen coming, but then what can happen with that? You mentioned infographics and things of that nature, white papers. You’re only providing the data. Did you mention that you also maybe help with infographics and things of that nature? Did you say that before?
Kathy Steinberg – Harris Poll: We could, yes. That could be a deliverable that’s included within the scope. It’s something that we can provide for a fee. Whenever we do research for public release, we consider it part of our due diligence to review any publicly released materials that contain that survey data. If you develop an infographic, or you write a press release, or there’s a white paper that you put together, we would also review that for you. That’s both a QC process. We’re going to check the numbers, make sure that the data is accurate. That’s another place where we can also add value and give you some advice on. “Okay, you’ve chosen to highlight this stat in your headline. Consider including this additional stat as a subhead to strengthen it.” We’ll give you some advice there as well.
Cale Guin: Oh, that’s great. Talk to me a little bit about the value that when I say, “Here’s my headline world,” and you’re okay with putting the Harris Poll name on that. What is that QC process like and what does it all involve?
Kathy Steinberg – Harris Poll: We recognize that, over the course of our many decades conducting this research, we’ve earned tremendous trust with the media and our name does carry a lot of weight. We don’t just put it on anything. When we’ve partnered to conduct research, when we work with clients who commissioned research, we have to be involved throughout. We’re not just going to take a set of questions that you provide to us and program them, and fill them, and deliver data to you and say, “Here you go. Have at it. Do whatever you want.”
We will always review your questions to make sure that we feel comfortable that you’re not asking anything biased or misleading. If you put out anything that talks about the data, we want to review it. That’s part of the process. The guidelines I mentioned earlier for releasing data publicly, include having the research company review and to see those materials. When you see a press release that says, “A survey conducted by Harris Poll on behalf of so and so,” you know that that was designed together, that there was a lot of work that went into it, that the research was done with care and rigor, that the results are representative.
Obviously, it depends on exactly what was done. If we’re doing a gen pop survey, the omnibus platform that we mentioned, that’s a nationally representative sample where raw data are weighted. All of that information is typically included in public release methods. One of the other things that’s included in our services is, if your research is intended for public release, and you go out there with your data and you get questions from the media about the research, we are available to answer those questions on your behalf.
The researchers who conduct this research have all been media-trained. We can work with you however you like. If you get a question from a recorder about the research method that you don’t know how to answer, we can either answer that question for you, you can send the reporter our way, or we can draft a response, share it with you. You can pass it along to the journalist however you prefer.
Cale Guin: I like washing my hands of those things.
Kathy Steinberg – Harris Poll: I do want to clarify, though, in the release that goes out, the study is owned by the sponsor of the research. If you commission the study, it’s yours. The media contact on a press release should be usually whoever the PR contact is for that company. That should be the first line of defense for the media. We don’t typically like to have our names as the contact people on clients’ press releases, but you can always send direct questions our way behind the scenes.
Cale Guin: Fair enough. One of the questions that I really wanted to get into was this idea of the flexibility of Harris Poll. I’m not sure if all polling companies, or all survey companies, or research companies are like this. If I have my own list of people that I want to reach out to, how does that impact the working relationship with you? Do I send that to my people? Does Harris Poll send that to those people? Is there savings to me because it’s my list and you don’t have to field them to whatever people you can find? How does all that work?
Kathy Steinberg – Harris Poll: The short answer is, it depends. It really depends on your willingness to share your list. If you want to share your list with us, turn it over to us and we can then treat it the same way we would our own sample. I think that would be preferred, because then, we have a lot of control over the data quality, and how to recruit respondents, and when to send emails, and when to send reminders, assuming this is all done online. We can also be flexible. If you don’t want to turn your list over to us, we can provide you with an Excel spreadsheet of however many links you need.
That also depends on the capabilities. If you have a member database of 2,000 people and you have a platform, and you’re able to do a mail merge that sends customized emails to all those 2,000 contacts, we’ll give you an Excel spreadsheet with 2,000 unique email addresses and then you would have to do that mail merge. If you have that capability and you have the time and effort to do that, that certainly means you’re spending less with us, but there’s a trade-off there.
There’s another option. If you’re not able to send customized emails, we can provide public links. You can recruit people just by sending an email blast to your whole database. Or maybe there’s a private Facebook group, and you can post the link to that group. Another example of our flexibility. You tell us what you want and what you’re willing to do, and we can give you a few options. We can absolutely work with client-supplied list.
We would want to discuss the implications of doing so. There’s not only the logistical implications that I’ve already mentioned, but there’s also data implications. We know that, depending on what we’re talking about, a member organization or a list may have bias associated with it. If you’re looking for a representative sample of something, then a list that you own may not be representative of what you’re looking for.
If you’re looking to survey your own customers, then that makes sense. That’s part of the consulting that we can offer. We can give you feedback on the pros and cons of using that list versus a different sample source.
Cale Guin: I’m going to flip that on its head and ask you, is there any world in which– Is omnibus all online? Can I just ask that? Is that all 100% online respondents in the omnibus platform?
Kathy Steinberg – Harris Poll: Yes, our entire suite of omnibus surveys is all online.
Cale Guin: The gen pop survey through omnibus is online? Is there a world in which there could be a question and opt-in? Can I get email addresses and names from that if they opt in? Would you ever entertain something like that?
Kathy Steinberg – Harris Poll: That would really depend on the type of respondent and the goals for that data. There’s a lot of privacy issues involved there. We would need to understand, how do you plan to use that information? I think it would depend on what type of sample it is. Are these physician respondents? Are these B2B, C-suite executives? Are these just consumers that we’ve recruited from a gen pop survey and they happen to answer questions a certain way and you want to follow up with them on something? There’s different rules and different questions, depending on how you want to use that information. Generally speaking, we tend to stay away from sharing respondent data because that is considered personal and private information.
Cale Guin: I really want to know, as small business people, we’re always looking for an opportunity to be in front of people. If I’m asking 2,000 people a question, is there any room in there that can say that, “If you want more information on this, go here”? Can that happen at all in there? Advertising. I know that sounds so cheap, but it’s my survey. We’re asking questions about what I want. Is there any way to do something like that? I will absolutely edit this question out if you think this is the most ridiculous thing anybody’s ever asked.
Kathy Steinberg – Harris Poll: No, it’s not ridiculous. It’s certainly a valid question. I personally have not worked on any study that did anything like that, certainly not for promotional purposes. I’m fairly certain that the panels that we use to actually reach these respondents, they promise these panelists that they will not use their names for promotional purposes. That’s why they agree to participate in survey research.
The few instances where we have included like, “For more information, contact this,” as part of a survey question, the few times that I’ve done that, it’s only been in the healthcare space and for patients’ health reasons. For example, I’ve done a couple of surveys on suicide prevention. At the beginning and end of that survey, we include a link to the suicide prevention hotline and that phone number. We have included links to resources, but I don’t think we would do something like what you described, where it’s, “For more information on this product–” I don’t think we would feel comfortable with that.
Cale Guin: I’m all about the cheap. I really don’t want to make it sound like I’m cheaping out. We’re always looking for those types of opportunities. If you’re reaching out to 2,000 people like that, is there an opportunity in there for us to– Maybe you can do, I suppose, name recognition-type things and what have you.
Kathy Steinberg – Harris Poll: You absolutely can do surveys about the brand or the product. If you want to understand brand awareness, or brand usage, or likelihood to purchase, or purchase intent of your product versus a competitor’s product, we can absolutely do that. That is a different type of study. That’s research that would not be publicly released. You wouldn’t share those results publicly.
Then you can absolutely ask for deep dives about the brand and questions like, “What would make you more likely to buy this product? Which of these features is most appealing to you?” There are ways that we can ask the questions to get strategic and helpful insights in a less salesy-sounding way.
Cale Guin: So diplomatically put. Do you have any information that you think would be super helpful to a small business looking to potentially do some research in ways that maybe they haven’t even thought of? Do you see any common themes where people are always looking for this kind of information? What would you have to bring to the table that maybe a dope like me hasn’t thought of?
Kathy Steinberg – Harris Poll: You’ve probably already thought of it. I want to give you more credit than you give yourself. One thing we hear from small businesses and even not so small businesses, especially in the B2B space, if you want to survey a highly specific audience, a targeted audience like C-suite decision-makers, or IT decision-makers or something like that, we can’t do that on an omnibus platform. Certain audiences require a custom approach, custom recruitment. Some audiences require financial incentive. When we talk to consumers, they’ve opted into panels, where they’re happy to take surveys, and they usually don’t require cash incentives. If you’re trying to get an IT decision-maker to take a survey for 10 minutes, that usually warrants a $10 or $15 incentive. That starts to add up. That goes up even higher if it’s a C-suite executive or if it’s a longer survey. That type of research can be quite expensive if you’re trying to reach a highly niche audience.
One thing that businesses can keep in mind is that, even if your target messaging, if your communications are targeted for this audience, that doesn’t necessarily mean that’s the sample you should survey. In one of my presentation decks, I have on a slide something that says, “Target audience does not equal sample definition.” I think that’s an opportunity to get creative.
If you can’t afford to do a custom survey of decision-makers, then maybe you could leverage an omnibus platform to survey the end user of the platform. An example would be, in the IT space, let’s say you have some enterprise software product that support HR benefit management or something and you really want to reach HR decision-makers, that’s a specialized sample. That’s expensive to survey. HR decision-makers serve the needs of employees, employed adults. That’s a much bigger pool.
We could do a survey using our omnibus platform. Out of 2,000 adults, depending on how you define employed, you’re going to get at least 1,000, maybe smaller if you’re looking for, let’s say, employed adults who work in an office setting. Even that is a big enough group of the American population, that we can use an omnibus platform and get 500, 800 interviews. You can do a slightly more customized survey leveraging an omnibus as the data collection vehicle.
Cale Guin: Two things. First of all, that is a great idea. I think that we should always be thinking of that. Sometimes we think that we want the C-suite opinion on something, but it’s, we want that for a reason. The reason might rest below that level and you could then be saying, “What do you think your bosses think of this?” That can be then the survey. That’s brilliant. You’re right, I thought of all of those things. [chuckles] I didn’t [crosstalk].
Kathy Steinberg – Harris Poll: I knew it.
Cale Guin: Anyway, are there guidelines for how big a sampling needs to be in order to get reliable data? Do I always have to do at least 2,000? Can I do 50? How does that break down? Tell us how we should consider that?
Kathy Steinberg – Harris Poll: I think anyone who’s watched TV at some point has seen on the news a recent survey of 1,000 adults, fine. From a consumer survey perspective, 1,000 tends to be considered the gold standard. If you’re talking about US adults ages 18 plus, 1,000 is sufficient. We offer 2,000 with our omnibus. It’s more robust and you just get a lot more room to do subgroup analysis. If you want to a screen for employed adults, if you want to a screen for office workers, we just have more wiggle room to do that. On the lower end of the spectrum, we do tend to think of 100 as the minimum sample size for quantitative analysis, to be able to talk about data-using percentages.
When the media thinks about survey data, I think most reporters are smart and savvy enough that they understand the difference between a consumer survey and a C-suite survey. If you’re trying to talk to C-suite executives, no one’s going to expect you to have a sample of 1,000. You can still get quite a bit of interest if you talk to just 150 CEOs or whatever. If you try to pitch a survey to the media where you surveyed 150 US adults, you’re going to get laughed at.
Cale Guin: That’s funny that you say that, because as soon as you say 150 consumers, then you’re right. That just sounds like, “Why only 150? Were these the only 150 that answered the way you wanted them to?” We always joke in marketing, too. Three out of four dentists surveyed say that this is the best toothpaste in the world.
Kathy Steinberg – Harris Poll: Even four.
Cale Guin: Yes, but we had to survey 8,000 people to find a group of three out of four. All that stuff seems like it could be BS. I do appreciate the answer because it makes sense. As soon as somebody smart like you says, 150 consumers compared to 150 CEOs, it’s a wildly different prospect. I appreciate that very much. If there’s anything else you have, right now is the time to give it out. Otherwise, I can’t even begin to thank you for your time.
Kathy Steinberg – Harris Poll: It’s been a pleasure. Thank you so much for this opportunity. I’m just looking down at my notes and I think I’ve hit the high marks of everything that I wanted to mention. I just really appreciate the opportunity to share these thoughts and ideas with your audience.
