Don’t forget the human factor


The best analytical and AI tools in the world cannot explain the unique weaknesses of humans.

Image: iStock / metamorworks

After years of resisting “fake football”, I eventually joined a neighborhood fantasy football league. I’m a very laid-back soccer fan and I probably couldn’t name 10 active players without thinking for a few minutes, but in the interest of having some neighborhood fun and learning a little more about the game, I started my first team.

I still honestly don’t quite understand fantasy football scoring and all the nuances, but for the unknowns, during a draft process, you select a virtual team from a pool of available players, and each player’s activities on the field that week contribute to your overall result in the team evaluation. For example, if my defense blocks a touchdown, I might get 10 points, while if a running back runs a few yards on my team in another game, I get a fraction of a point. In theory, this piques interest in more teams by giving the fan more players, but at this point it mainly creates confusion as my extremely limited “soccer brain” tries to watch half a dozen simultaneous games.

SEE: Digital Transformation: A CXO Guide (Free PDF) (TechRepublic)

Our league uses the Yahoo! Fantasy Sports app / website, and although it’s been years since I’ve used anything from Yahoo, the app and website are extremely impressive. Most notable for this amateur rank is the breadth and depth of statistics available, from what you would expect about a player’s performance so far to predictions about the outcome of every game in our league. My first game had predicted results for each of my players to two decimal places, predicted totals, and a probability of victory, all updated in real time during the weekend’s games.

I started the day as an underdog, but due to a combination of luck and chance my team has apparently won, unless my kicker who is playing today somehow scores – 13 points. As I studied the app at random intervals on Sunday, I couldn’t help but get the feeling of looking at my stockbroker’s online trading platform. Seemingly precise numbers in red and green, flashing numbers, gave what is basically a roll of wildly complex dice a semblance of digital security.

These messy people

This randomness may seem highly undesirable. After all, nobody wants the unpredictable outcome of a major operation, airplane flight, expensive steak dinner, or even their neighborhood fantasy football league where a few cans of local beer are top prizes. As a technology leader, the strategy to mitigate unpredictability is often automation or, more recently, analytics and AI.

At countless conferences and in the pages of technology books, I’ve heard of a brave new world in which machines make near-perfect decisions and reduce or eliminate human “clutter”. Of course, this is not unprecedented, and machines have proven capable and even superior in everything from flying fighter jets to winning complex games like Go.

However, despite real-time analytics, live data feeds, and a lot more processing power than my skimpy newbie soccer player, the machines couldn’t accurately predict the outcome of my fantasy soccer game. Not only did you miss the win prediction, but the original prediction only gave me a 39% chance of winning.

You’d think this is a totally unfair task to expect a machine to work properly. After all, the outcome of a sporting event can depend on something obvious like the weather, something trivial like what a top athlete had for breakfast. All of this is true, but the danger for technology leaders is the implicit security that comes from visual cues like seemingly precise predictions to the wealth of data fed into a predictive model.

The analytical model that predicted my running back would score 15.89 probably had years of high quality data and may have been developed by some of the best data scientists, but a combination of chance and circumstance has conspired to make this player 4, 90 supplies. Missing 70% is okay for fantasy football, but probably not as good for tasks from transoceanic navigation to sales forecasting.

As a technology leader, it is our job to convey exactly what technologies like AI, analytics and machine learning can and cannot do. These models may have eerie and seemingly magical skills in some areas, but lack skills in others that even a child can perform with ease. These dichotomies become even more difficult when the majority of users, and in the case of neural networks, even the creators or the network, cannot begin to understand how the models work. Whether you aspire to brag about the neighborhood or want to enter a new market, understand the tools at your disposal and the weaknesses and skills of each one.

See also


Leave A Reply