The concept recently got a test run thanks to, of all things, a sports betting website called SportsBettingDime, which enlisted OpenAI’s latest text generator, GPT-3, to see if AI could emulate a head coach. The study (full results here) prompted GPT-3 to generate motivational speeches, handle in-game scenarios, and promote team building, which is among the important functions of a head coach in many sports. The site then asked NBA and NFL superfans how GPT-3 performed. Corny as those movie moments may be, motivational speeches really are a key function of many coaches’ jobs, rallying players with the right dose of reality or inspiration at the right moment. The study used GPT-3 to generate three separate speech snippets and then mixed in two real speeches from coaches. NFL and NBA fans were then asked for their unbiased opinions. “As opposed to tactics and gameplan, speeches are one of the more human responsibilities of a head coach that data can’t answer. Surprisingly, AI seemed to hold its ground in providing motivation and emotional support, according to superfans,” according to the study. In fact, the two of the AI speeches ranked highest in this admittedly loosely controlled test, receiving motivational ratings of 6.82 and 6.47, respectively. However, it must be noted that the third AI speech came in dead last, suggesting marked variability in outcome. Interestingly, the motivational value didn’t necessarily equate to how human the speeches sounded, at least not to the fans who participated in the study. Both of the authentic human speeches were ranked as having likely been given by a real coach (77.8% and 66.3%, respectively), while AI speech realness ranged from 66.4% real-sounding to just 51%. The study also evaluated the AI’s decision-making in game scenarios against the plays the superfans would have called. Scenarios were evaluated first in an NFL situation: Scenario A
On a 4th and 1 with limited time left, AI would call a run play for the first down, specifically calling the running back’s number. Many NFL superfans would disagree: 31.9% would run a quick pass play, although 22.2% agreed with AI on an RB run play. AI’s emphasis on this last drive for the win was to get in range for an FG to tie instead of taking the risk for a TD to win the game. With 50.8% of them agreeing with AI to get in range for an FG and 49.2% disagreeing, NFL superfans were split.
Scenario B
Following a touchdown with no time left and being down 1 point, AI would go for a 2-point conversion to win the game as opposed to a less risky extra point to tie. Again, NFL superfans were pretty split, with 48.2% agreeing with AI and 58.8% disagreeing.
The likewise evaluated play calling in high-pressure NBA scenarios: Scenario A
Down 2 points with 10 seconds left on offense, AI would run a set play, specifically looking for a 2-point jump shot. Most NBA superfans would deviate from that strategy; 28.2% would run a set play for a 3-pointer to win the game, and 19.5% would isolate their star player. Still, 22.2% agreed with AI on a set play for a 2-pointer.
AI believed the best offensive option in this scenario was to go for 2 points, specifically a jump shot. NBA superfans were somewhat split, with 55.5% of them agreeing with AI to go for a tie (2-pointer) and 45.5% disagreeing.
Scenario B
In this defensive scenario, AI would emphasize denying the ball from the other team’s best player and defending without fouling. As their top two emphases lined up with AI, NBA superfans agreed with AI (46.7% agreed with denying the ball from the other team’s best player, and 33.6% agreed with not fouling).
The study results, which, again, is hardly the kind of scientific evaluation needed even to approach answering the question, present some interesting anecdotal takeaways. As a motivational speaker, GPT-3 was surprisingly competent. However, when it came to designing specific play calls, AI seems not to have made the best decision in the eyes of those identified as superfans. However, AI was able to be decisive in scenarios where superfans were split, which points to one possible advantage to AI over humans: computers aren’t emotional. They may be more able to act decisively (and quickly) in crucial moments. Will AI take over from your team’s head coach? Probably no time soon. Should an AI take over when the technology reaches or exceeds a threshold of parity? Depending on how you did last season (or are doing this season), your answer may vary.
