Facing the data
The pictures showed all the fan emotions and facial expressions one would expect at a sporting event: cheers of jubilation, frowns of frustration, smiles of satisfaction, and even some indifference.
And within all those faces, there was data. Lots and lots of data.
Fancam has built over the last seven years a growing, if still somewhat unknown, business capturing high-resolution images at sports events and concerts. Positioned as an engagement tool where fans could find themselves in a detailed megapixel picture of 50,000 fans, the South African technology company had developed a roster of clients that touched every major college and pro sport and included a collection of blue-chip sponsors underwriting the content.
But a much bigger opportunity still awaited, and over the past year Fancam has pivoted much of its business to mining the data around the faces that make up those images. The Fancam fan offering is now augmented with an analytics component that studies the images for demographic and behavioral trends of attendees, measures fan sentiment using facial recognition technology, and can even register the level of fan attention paid to a video board ad at a given moment.
“Forty thousand megapixels can paint a real picture of clarity on what’s going on in a building,” said Michael Proman, managing director of Fancam’s North American operations. “We’ve always been a content company and will have a content component going forward. But I would say that more than 80 percent of our business development is really data-specific now and our biggest opportunity is on the data side. Facial recognition opens up a lot of insights that legacy data sources simply miss.”
Fancam is not the only firm looking to exploit facial recognition for fan insights or fan experience elements that in turn yield new data. Fueled in part by recent camera, computer processing and imaging advances that make homing in on individual fans in a large venue a far simpler and more accurate task than ever, facial recognition technology is now in many cases also being connected to other elements such as artificial intelligence to seek out new learnings.
“There is significant interest among teams and stadiums in this technology,” said Allen Ganz, director of critical infrastructure solutions for NEC Corp. The company has worked with several sports entities, including the LPGA and NBA, on facial recognition projects. “This is information that is tremendously valuable to any chief marketing officer.
First Look podcast, with facial recognition discussion at the 11:52 mark:
“For example, this is something that can help close that gap between a ticket buyer and the user of that ticket, as those are often not the same people,” Ganz said.
And beyond yielding new business insights to a team, league or venue operator, facial recognition presents enticing possibilities from a fan experience perspective. For example, fans in the not-too-distant future will be immediately identified, individually, as they enter a venue. And depending on their prior relationship with a team, those fans will be welcomed back and presented with a surprise perk such as a free food item tailored to the fans’ known preferences, or access to a luxury club. Or if they are newer fans, that same facial recognition technology will likely be able to use that image and present additional choices to help them tailor their experience.
But facial recognition remains a highly sensitive subject where privacy and legal concerns are paramount, and many U.S. sports entities refuse to even discuss publicly their use of the technology because of those issues, and have frequently tied their vendors to nondisclosure agreements. Fan and industry sentiment is also heavily divided on the subject (see Turnkey Sports Poll).
“There still has been a fair amount of resistance to facial recognition in the United States that we’ve seen, certainly compared to some parts of the world, and what might go into a database,” said Matt Bocko, chief strategy officer for Colosseo USA. The U.S. division of the Slovakian technology company has installed facial recognition systems at several venues in Europe, where the political climate, particularly around data and content issues, is often less piqued.
“It’s always been a tricky subject, walking that line between safety and security,” Bocko said.
Facial recognition in a broad sense has roots going back more than five decades. And in sports, use of the technology takes many of its cues from security settings, where it has been actively deployed for nearly 20 years to secure critical public areas such as airports and national borders.
Much like those other settings, the core concept involves scanning faces with cameras as they enter a venue, and checking those images against some sort of watch list of criminals, wanted individuals, or set of pre-approved entrants.
More recent system improvements have been able to identify matches between an actual person and corresponding images on file in less than a second. Such performance is a quantum leap forward from initial sports-related tests of facial recognition, such as at Super Bowl XXXV in 2001 where facial recognition, at that time hampered by far less powerful computers, struggled to yield clear and timely individual results in a large crowd.
Several of NEC’s recent projects in sports have focused specifically on the security area, including a trial effort with the LPGA to check credentialed media at last year’s ANA Inspiration at Mission Hills Country Club near Palm Springs, Calif., and a similar initiative at this year’s NBA All-Star Game at Los Angeles’ Staples Center.
“We’re very interested in exploring cutting-edge security concepts, and we liked what we have seen so far in this area,” said Lee Zeidman, Staples Center president. Arena staff worked with the NBA during last month’s All-Star Game to check credentialed media with facial recognition, and the concept is now likely to be expanded across the entire L.A. Live sports, entertainment and hotel complex.
But over time, the core influences for facial recognition in sports have centered less on the Department of Homeland Security or other law enforcement agencies, and more on the consumer technology, retail and restaurant industries.
Apple last fall notably introduced the iPhone X, which includes its Face ID technology for users to unlock the device instead of the now-dropped home button. Samsung offers a similar feature for its high-end Galaxy devices, but it hasn’t generated the same level of consumer attention as the splashy arrival of the new iPhone.
“There has been a recent paradigm shift in the marketplace for this technology, and we have Apple to thank for a lot of that,” Ganz said. “This is quickly becoming a much more commonplace and accepted thing.”
Face to order
Popular California-based hamburger chain CaliBurger recently took a similar step as Apple’s Face ID phone unlock feature, piloting a facial recognition-based ordering kiosk that remembers users’ prior orders, loyalty accounts and payment information.
The kiosk, allowing for a seamless ordering process, employs the same NEC NeoFace system used by the LPGA and others in a security context. And it has many sports executives believing it could easily translate to additional consumer applications at stadiums and arenas. And of course, that facial recognition-based ordering system yields large amounts of consumer and transaction data.
“Concepts like CaliBurger and what they’re doing are very exciting, and open up a whole myriad of possibilities for us,” Zeidman said. “Imagine this technology being used to allow access to special VIP clubs, in-suite food ordering, or helping manage periods of peak demand. There are a lot of avenues opening up to us.”
Others in the space see a fan’s face ultimately replacing their tickets, keys, credit cards, and other similar attachments and more seamlessly powering their own experience at a venue.
“I’m really optimistic about what this can do with wait times, and getting into and around a venue,” said Peter Trepp, president and chief executive of California-based FaceFirst, which works with a variety of undisclosed teams and leagues on facial recognition projects, both in security and fan experience settings.
“You think about all the time you’re standing outside a venue waiting to get in, or waiting in line for food. Why couldn’t you just walk in and be recognized as you enter and have that VIP-type of experience unfold for you? That type of situation is coming very quickly,” Trepp said.
But just as more sports industry executives get excited over the promise of facial recognition technology, unease remains about data security and alerting potential threats at a venue to the presence of the systems.
Many facial recognition deals, particularly in a security context, have arrived with strict confidentiality provisions. And as a result, public discussion around those systems is minimal at best. And on the fan experience side, many facial recognition programs have been and will remain opt-in features tied into other voluntary elements such as a team or venue mobile applications.
“Having that level of choice is really important,” Trepp said. “That makes a huge difference in the level of acceptance by fans around this.”
The PGA Tour has explored facial recognition, meeting with several companies, and has imposed a similar threshold on any future implementation.
“It needs to add value to a fan and help them have a better experience,” said Ken Lovell, PGA Tour senior vice president of international marketing. “This is something we’re certainly considering, but we’re really focused on both that value-add to the fan and ensuring that privacy and data security is paramount.”
New York-based 15 Seconds Of Fame relies heavily on facial recognition technology, also generating fan content in collaboration with dozens of pro teams and leagues by extracting video and scoreboard footage of fans for their own use on social media and elsewhere. But it, too, has made its system a fully opt-in proposition.
“We have an opt-in solution and we respect our users’ rights,” said Tom Anderson, 15 Seconds Of Fame chief technology officer. “Some of the other players in the field take an image and utilize biometric profiles of non-users without permission. We distribute fan moments because a fan has opted-in and given us permission to search for their image and deliver them that moment of fame.”