SBJ/September 26 - October 2, 2005/SBJ In Depth

Agencies roll out new measurement tools as sponsors seek to justify their investments

You won’t find a sports marketer who says the industry’s skill in measuring return on investment hasn’t improved in the past five years. But you also won’t find a sports marketer who says the industry has really figured it out.

And sports marketers aren’t alone. A survey done this year by three broad-based marketing advocacy and research groups found that 60 percent of marketers think it’s important to define, measure and act on return on investment (ROI). But only 20 percent said they’re satisfied with their efforts.

The reasons are many. For starters, some marketing realities are hard to quantify, such as how brand awareness or good will translates to financial return. Or how a good hospitality experience affects the bottom line in subsequent weeks or months.

But even that’s getting ahead of things. The challenge of ROI begins with what ROI is. “You have to determine exactly what you’re measuring, and a methodology for measuring and tracking,” said Jay Kenney, senior vice president for IMG Consulting. “You also have to set a benchmark for what success is. And most importantly, you have to have an internal alignment in senior management sign off on all of the above.”

Indeed, it’s easy for a company to get off the track during the process — at almost any stage. ROI measurement problems can start with a poorly conceived purchase.

“You hear stories about a company that gets into sponsorship and takes the elements and then says, ‘How do we develop the activation program?’” said Steve Lauletta, president of Omnicom’s Radiate group of sports marketing agencies. “We say, develop that [activation strategy] up front and then just go buy [the appropriate] elements. It’s hard work, but the industry is getting better at this.”

Initial purchases are made harder by the fact that, historically, sports properties don’t do a great deal of sophisticated research into their fan demographics, nor do they have incentive to help in the analysis after a deal, in case results are lackluster.

“There is a greater demand at the moment on the owners of sports properties to provide good quality data on their demographics,” said Michael Stirling, an attorney specializing in sports rights negotiations with London law firm Field Fisher Waterhouse, whose clients include Fortune 500 companies and leading Champions League soccer teams. “But we audit the data and we are often very worried about the quality that the rightsholders come up with.”

The ideal is a sharing of good data between sponsor and property. But sponsors also fail to keep the bargain sometimes. “There’s that fear, if [the property] knows how successful we are at driving the bottom line, it could affect fees in the future, so there’s a little bit of hesitancy to share,” said Rob Vogel, president and chief operating officer of The Bonham Group.

Experts say sponsors need to set a benchmark for what they’ll consider to be good ROI, and then lay out a methodology for measuring and tracking those returns.
Oh, and there’s the issue of how to interpret the ROI numbers themselves. “Most people who I see making sponsorship decisions are not necessarily trained in market research and statistical analysis,” said Katie Delahaye Paine, CEO of the analytics firm KDPaine & Partners and publisher of The Measurement Standard newsletter. “You have all these fabulous tools, and most people say, ‘Um, I don’t know how to deal with the numbers.’”

This creates an opportunity for agencies with the right expertise. Analysts see agencies continuing to lead clients through the ROI process, and the agencies see themselves making new investments in ROI measurement technology.

“It will become a lot closer to, if not a daily, then a weekly or monthly management tool,” said Jim Nail, principal analyst at Forrester Research. “And what that will mean is … less and less the agency doing their thing and saying, ‘Put your money here’ and more and more about ‘Here’s where the model says you should put your money.’”

IMG’s Kenney said his firm is looking into developing a suite of online-based ROI tools. “We would also consider buying a company; we’re sort of thinking about developing an alliance or a purchase,” he said. “There’s a future for it as more companies go online with [their analytics]. As a business model it makes sense.” The Bonham Group already has the tools in development, Vogel said.

Indeed, there’s a strong feeling that some of the perplexing facts of ROI measurement can be sorted out by computer-based analytical systems — of course, when they’re accompanied by good planning, wholehearted execution and informed number-crunching.

In this article, we look at some of the strides being made by agencies to develop and improve these systems.

Up-front analysis

As marketers have noted, to maximize return on investment, it helps to make the right investment in the first place. One of the tools now employed on the front end of the sponsorship process is SponsorAid, developed three years ago by the Chicago-based company of the same name, in collaboration with Miller Brewing, which now uses the system to analyze more than 1,000 sponsorship proposals annually. More precisely, Miller lets the system pre-analyze the proposal based on extensive data provided by the property online, at

The property enters the proposal inventory — things such as signs, activation tools, promotional tools in and out of venue, print, TV, radio — and SponsorAid vets the program for overall conformity to Miller’s goals and assigns a dollar value to each of the components listed above, and more.

Panasonic is another company that has turned to sponsorship pre-analysis tools.
There are other pre-analysis products available to sponsors — such as SponsorWise, SponsorDirect and Sponsorium — but SponsorAid president Chris Thompson says SponsorAid is the only product that puts a dollar valuation on each component. And he has a patent application pending to back up the claim.

Rick Hall, vice president at Omnicom’s GMR Marketing, Miller’s sports agency of record, said SponsorAid “gives us a line of sight into the true market value of what Miller’s buying.”

He points to one example when SponsorAid assigned a higher value to the use of a team’s logo rights than Miller would have. “It allowed us to see that there were multiple factors going into the value of the trademark, in terms of the franchise value, its performance on the field, its attendance,” Hall said. “It showed we could really take that trademark to retail and identify with it as a marketing tool.”

Another indication of SporsorAid’s influence on Miller: Former Miller sports marketing boss Steve Lauletta is using SponsorAid as the front end of a two-prong system he’s developing at Radiate. The second prong is a post-sponsorship analysis tool that serves the cause of “being more disciplined about going back and looking at what you said you were going to do,” Lauletta said.

While Miller takes sponsorship assets to market, Thompson is taking SponsorAid to market, as well. He has been angling for single deep partners in each of the remaining seven or eight top sponsorship categories — soft drink, fast food, etc.— but so far hasn’t landed one. “I guess I’ve been spoiled with Miller,” he said with a laugh. “But I’m not out to just sell site licenses to a bunch of companies.”

See also:
League sponsor directory
So he’s putting new energy into developing agency relationships and is adding to SponsorAid the capability to do “one-time applications,” basically a sponsorship valuation snapshot with a one-day turnaround at a cost of between $500 and $1,000. A deeper, Miller-style relationship would run in the mid-five figures, Thompson said.

In the meantime, Thompson has made improvements to SponsorAid, such as adding a user-friendly version for smaller properties. “I had to make it easier for the less-seasoned, say a Saturday night dirt-track racer,” Thompson said.

But even big properties benefit from SponsorAid, he said. “The quicker both sides see what each thinks something is worth, the quicker they can start to talk productively.”

Up-front ... and beyond

SponsorDirect is one of many companies offering online sponsorship pre-analysis tools, and it has recently rolled out version 2.0 of its SponsorPort tool, aiming to facilitate the ROI evaluation on a company’s full portfolio of deals.

Field research can quickly measure the impact of activation and generate sales leads.
Bank of America, Nokia, Dodge and Panasonic are some of the companies that have used earlier versions of SponsorPort.

For Bank of America, there’s immediate ROI. Before automating the pre-analysis, Bank of America had roughly three dozen regional marketers spending up to 18 hours a week vetting proposals. The man-hours have been cut in half, according to Joe Goode, corporate spokesman for Bank of America’s sponsorship group.

In the last few months, SponsorDirect has been introducing 2.0 to clients, and Bank of America and Chevron are two that will be using the new portfolio management and ROI-measurement tools. Goode said it’s too early to comment on the performance, and Chevron marketer Trisha Roche said company policy prevents her from talking about vendors’ products.

But here’s what 2.0 offers users, according to Ken Brenner, president of SponsorDirect: a listing of all the costs that go into a sponsorship; objectives described numerically or qualitatively; the sponsor’s metrics for measuring projected value vs. the actual value derived; and external metrics, such as sales figures and performance.

“We can build algorithms behind the scenes based on a company’s methodology, and we think this is the first time the industry is offering infrastructure for companies to arrive at this kind of measurement and ROI in a centralized platform,” Brenner said.

There’s a flip side to the high claims SponsorDirect is making for the functionality of 2.0. It’s the absolute need for participation by all sponsor employees involved in the marketing process. From executives throwing their weight behind the system to field marketers inputting sales results and updates on execution elements, the system needs extensive data to process.

“Buy-in is so critically important,” said SponsorDirect CEO Mark Rockefeller. “Across an organization you can’t have certain brand managers excited about it and some not.”

Where rubber hits road

Increasingly, marketing agencies are offering field research tools that a sponsor can use to measure the impact of their execution as it’s happening.

DaimlerChrysler has used Event Metrics Co. and Turnkey Sports recently for this. In the latter case, Turnkey stationed roughly a dozen questioners at Historically Black College and Universities football games last season and captured information on their car-buying habits and their likelihood of purchasing DaimlerChrysler products.

Systems on the market can help make sure hospitality projects run efficiently at events and later provide more exact ROI measurements once the events are complete.
The auditors didn’t ask questions about perceptions of DaimlerChrysler’s sponsorship itself. The time was spent in determining whether the fan was a good sales prospect. Immediately after the event, the information — originally collected on PDAs — was uploaded to the company’s data-collection system, administered by agency BBDO.

Soon, dealers had the information on top prospects, and BBDO continued to track sales efficacy at the 30-, 60- and 90-day marks. Melissa Killinger, senior vice president, director of integrated strategic marketing at GlobalHue, Daimler Chrysler’s multicultural marketing agency of record, would not describe the sales results of the data, but she had no qualms about Turnkey’s execution of the program or of the quality of the data.

“Turnkey was well organized, and they did a great job in working with BBDO,” she said. “They had set up systems, and Turnkey followed them.”

Experts say in-event audience polls can be flawed if they’re angling for a true cross-section of fans and their true feelings.

“Some fan sections are not going to feel comfortable doing such a survey, or a high-tech survey in the first place,” said Whitney Wagner, adjunct instructor of marketing at the Warsaw Sports Marketing Center at the University of Oregon, who spent seven years at the NFL in the marketing department. “And then you have the same problem: What does ‘I’m satisfied with this’ mean? How reliable is the question, ‘Would you be willing to pay five dollars more for the experience?’ Just because it comes in real time doesn’t get to the heart of the issue.”

But Turnkey and Chrysler believe they narrowed the criteria enough to get reliable results. “Part of the direction was that they make sure the person is actually educated on the product and then asked politely if they are interested in more information — and if they were in-market, if they could give us their contact information,” said Killinger.

“This was not just lead generation, but a more in-depth profile of the people at the event, so they can tweak their approach, or the whole package, after the event,” said Len Perna, president of Turnkey Sports. “In terms of ROI, you can really figure out where the rubber hits the road: ‘Are they considering us?’”

After the party

Once an agency has a system developed to pre-analyze or manage a sponsorship portfolio, it’s natural for it to try to build on new functionalities, such as SponsorDirect has.

Sponsors are interested in the quality of logo exposure as well as the duration of exposure.
SportsMark’s SportsManager has proved itself as a dependable hospitality-management tool, tracking the myriad administrative details that go into hosting hundreds of guests at a major event, such as the Olympics. But the company is also pushing its post-event analysis tools, which build off the same system. Where SportsManager helped ROI by making hospitality more efficient, the new functionalities aim to make ROI measurement itself more exact.

It starts with a few questions asked of guests before the event, about their perceptions of the brand doing the hosting. After the event, guests are asked via e-mail to answer a few questions about the brand and its event, such as “What did you see at this event and how did the brand come across?” and “Is the brand in tune with contemporary lifestyles?”

The Hartford used SportsManager around its NCAA corporate partnership during last year’s Final Four, including the post-event analysis, married with survey work done by The Hartford.

“The Hartford wanted to increase its corporate customers’ sense that it is a leadership brand in tune with its consumers’ contemporary lifestyles,” said Keith Bruce, CMO of SportsMark. Thanks to the NCAA relationship, and thanks to the hospitality program, as measured by SportsManager, “The Hartford realized a significant growth in its customers’ perceptions of the company as a more contemporary leadership brand.”

And as with some of the other systems profiled here, SportsManager can help a company track eventual sales made with those guests.

Where perception is concerned, Bruce said companies are careful about how they approach guests after the event, and they’re careful with the questions. But he said after a good hospitality experience — SportsMark has also administered Olympics hospitality for Xerox, among its relationships — guests are usually willing to answer a few questions.

On the other hand, how does SportsManager assure that the guests don’t feel obligated to give positive answers? Or are afraid to give negative ones? “There’s always a risk, but there are plenty of different strategies [for encouraging honest responses], depending on the question content,” Bruce said. “Many times, the guests are assured their responses will be anonymous.”

SportsMark is an event-management company that has been tracking hospitality activity online for several years. But more agencies and clients are getting into the act. Paragon Marketing Group rolled out an extensive ticket-management program, called TAP, for its client Continental Airlines this summer, in connection with Web developer American Eagle. The product is now available to corporations for a roughly $30,000 up-front fee, plus maintenance. The Bonham Group has developed its own system.

“The ticket allocation piece, people see it as a necessary evil, but frankly it’s a difficult thing to manage, it’s a nasty beast,” said Robert Madrigal, professor of sports marketing at Oregon’s Warsaw Sports Marketing Center. “And, again, it’s a difficult thing to really analyze what makes the use of a particular piece of hospitality effective.”

Sharpening the picture

Event signage is as much (or more) for television as it is for the fans at the event, but television isn’t always kind to the signs. Partial or obstructed views of logos vex brand marketers, including when they try to measure the value of those exposures.

Joyce Julius & Associates has enhanced its system for measuring the value of logo exposure during television broadcasts.
Joyce Julius & Associates and TNS Sport are two companies that are trying to make the measurement process more exact — Julius with its new Image Identification Technology and TNS Sport with Sportsi, which it rolled out more than a year ago.

Joyce Julius & Associates has developed a huge client roster for its television brand-exposure analysis work, yet you still hear the industry complain of a methodological weakness in how the firm handles partial exposures of a logo. The firm might have solved that perception problem with its Image Identification Technology, powerful software that calculates logo exposure and makes fine distinctions in the quality of that exposure.

“I would use the analogy of watching a baseball game and knowing that the last pitch was a fastball,” said Eric Wright, vice president of research and development for the company. “Then putting a radar gun on the next pitch and seeing it was a 91 mph fastball.”

The technology, developed by UK-based OmniPerception and contracted for on a non-exclusive basis by Joyce Julius, does what the company’s human analysts have always done, but with a pixel-by-pixel exactitude. The technology was developed from forensic tools that searched for face patterns in security videos.

“In basic terms, we can tell the amount of space occupied by a logo, the amount of logo on screen per frame, the percentage of blockage, say by someone standing in front of it, and where on the screen the logo appeared,” Wright said.

It also offers speed. Six weeks worth of human work was recently done so quickly that the company released its U.S. Open tennis results five days after the event.

With the computer to supplement work being done by humans, “Now we can say, ‘Here’s why you received 10, not 15, minutes [worth of exposure], and for those five minutes, look at your clarity rating. We could have done it before, but it would have been labeled subjective,” Wright said. “We believe strongly in our researchers, but now we have percentages and images to back up the concepts that we would have tried to relay.”

TNS’ Sportsi is now being used by some leagues and individual sponsors, and recently gained traction with agencies, said Mark Davis, director of business development. The technology can measure partial and obscured logos, and it produces a numeric score based on how close to perfect the logo is.

Sponsors are demanding a closer accounting of on-screen logos, Davis said. “Since there are so many more opportunities for companies to buy into sports from a media standpoint — so many different sports and ways to be integated into action — there’s more call to measure it. People are not just looking at the commercial strategy but the whole media strategy.”

Radio’s reach

Marketers like to say that ROI measurement has evolved beyond calculating the television cost-per-thousand equivalent of various kinds of brand exposure. But in the radio world, you haven’t always been able to get a dependable cost per thousand to start with.

Turnkey Sports Poll
The following are results of the Turnkey Sports Poll taken in August. The survey covered about 400 senior-level sports industry executives spanning professional and college sports.
Certain sponsors are paying closer attention to measuring return on investment. How will this impact sports properties?
Good for sports
Bad for sports
No impact on sports
Is your organization spending more or less on consumer research compared to three years ago?
Spending is about the same
Spending is up slightly
Spending is up significantly
Spending is down slightly
Spending is down significantly
Did not respond
Source: Turnkey Sports in conjunction with SportsBusiness Journal. Turnkey Sports specializes in instant fan feedback (FanTrak) and custom market research for sports and entertainment. Visit
Arbitron is the first to admit this — especially now that it’s starting an audience measurement system for radio based on next-day phone surveys rather than the traditional 12-week paper diaries.

The new service — called the Custom Sports Survey — is a steppingstone to Arbitron’s “personal people meter” service, which will roll out over two years starting next year and employ pager-sized devices that track listener habits simply by picking up codes in radio signals.

When pre-testing of the PPM suggested that Arbitron’s diaries had seriously undercounted listenership, the company felt it would be profitable to offer the phone service in the meantime. The product rolled out this summer; the Los Angeles Angels are its first client. A 20-game unaided-recall study done this season produced a cumulative rating of 1.6 over a wide range of days and times — twice what the diary had logged for the same male 25-54 audience.

“We’ve always felt that audiences were bigger than the diaries showed, and this was borne out,” said Bob Koontz, station manager at ESPN-owned KSPN-AM in Los Angeles.

The next stage is getting agencies to respect the results, Koontz said. Sales for Arbitron-audited stations have been a loose negotiation in which agencies drove down prices because of the flaws inherent in the diary system, which was generally adequate for broad-stroke ratings over three months but were wildly inaccurate — in both directions — for discrete time slots such as a three-hour Sunday-night baseball game.

“We can’t just ask double our former rates now,” Koontz said. “This still has to prove itself. But we’ve made good inroads in the past few weeks and we haven’t had a bad response from an agency yet.”

The service costs between $50,000 and $65,000 per study, which involves between 1,000 and 1,500 calls. Demographic questions can also be a part of the phone call thanks to Arbitron’s relationship with Scarborough Research, according to John Snyder, vice president of national group services for Arbitron.

Considering that sports teams typically spend less on market research than other consumer businesses of their size, Arbitron is hoping the ability to do radio and demographic research in one fell swoop will prove tempting to teams and their radio partners.

Return to top
Video Powered By - Castfire CMS Powered By - Sitecore

Report a Bug