Book summary “Prediction machines”: summary

Introduction

You face a business environment where uncertainty paralyzes decisions and strategy requires constant guesswork. Recent advances in artificial intelligence do not bring general intelligence or thinking machines, but rather a sharp drop in the cost of prediction. This technological shift turns uncertainty into a managed variable, changing how you allocate resources and run operations.

Ajay Agrawal, Joshua Gans, and Avi Goldfarb explain how cheap prediction alters economics, business strategy, and daily work. You will learn to separate prediction from judgment, redesign your workflows, and adjust your organizational boundaries.

How does a drop in prediction costs change your strategy?

• Where do you need human judgment when machines make forecasts?
• How do you integrate prediction tools into existing workflows?
• What happens to organizational boundaries as automation expands?

1. The Economics of Cheap Prediction

The recent wave of artificial intelligence does not bring general intelligence. It brings a critical component of it: cheap prediction.

Ajay Agrawal, Joshua Gans, and Avi Goldfarb are professors at the University of Toronto's Rotman School of Management and founders of the Creative Destruction Lab. They explain that artificial intelligence takes existing data and generates missing information. When you ask Amazon's Alexa a question, the system uses incoming acoustic data to predict the most likely words and responses.

When the price of a foundational input drops significantly, society uses vastly more of it and applies it to entirely new domains.

A collapse in the cost of prediction changes human behavior from deliberate calculation to automatic reliance. William Nordhaus showed that artificial light in the early 1800s cost four hundred times more than today, severely limiting its use before prices collapsed and lit up the world.

Foundations of Prediction

Prediction is the fundamental process of using available data to generate missing information about the past, present, or future. Business applications define prediction as taking data you have to fill in information you do not have. This includes determining if a current credit card transaction is fraudulent, whether a tumor is malignant, or identifying an object in an image.

Avi's credit card provider used spending habits and data to proactively detect a fraudulent transaction in Las Vegas and mail a replacement card without blocking the account or requiring him to call customer service.

Small percentage improvements in prediction accuracy can result in transformational reductions in error rates when starting from already high baselines. Credit card fraud detection rates improved from about 80 percent in the late 1990s to 90-95 percent in 2000, and reached 98 to 99.9 percent with machine learning. Moving from 98 percent to 99.9 percent accuracy means mistakes fall by a factor of twenty, which changes the nature of the activity from incremental to transformational.

The Magic of Prediction

Machine learning models achieved a major breakthrough in 2012 and eventually surpassed human benchmark capabilities in object classification. The ImageNet Challenge tested computer vision across a thousand categories of objects. The error rate plunged from 28 percent in 2010 to 16 percent in 2012 when contestants used deep learning for the first time, eventually beating the human benchmark of roughly 5 percent error by 2015. In the ImageNet Challenge, the best team in 2017 had fewer than half as many mistakes as the human benchmark.

Navigating the impact of artificial intelligence requires building an understanding through a five-layer pyramid ranging from technical foundations to societal trade-offs. The authors organize their economic framework into five sections: the structural organization of the book covering prediction, decision-making, tools, strategy, and society.

To prepare for these changes, you can take specific actions:

• Invest in gathering intelligence on how fast and how far the prediction dial will turn for your sector and applications.
• Invest in developing a thesis about the strategic options created from turning the prediction dial.

2. Data, Accuracy, and Human Limitations

Artificial intelligence requires data in distinct roles to operate while highlighting human statistical shortcomings.

Prediction Requires Data

Machine learning relies on information to function, generate, and improve. You feed input data into the algorithm to receive a prediction. Training data builds the initial algorithm, and feedback data continuously improves performance over time through experience.

Statistically, each additional observation adds less value than the previous one. Economically, however, having superior data can push performance past a critical threshold, capturing disproportionate market rewards.

Search engines like Google, Bing, and DuckDuckGo provide similar results for common queries. Google's data advantage allows it to provide better results for unusual searches, driving market share.

Where Machines Are Poor at Prediction

Psychological biases, heuristics, and the inability to process high-dimensional interaction effects cause professionals to make inconsistent predictions. You can classify prediction settings into four categories based on data availability and causal understanding:

• known knowns with rich data;
• known unknowns with scarce data;
• unknown unknowns with entirely novel events;
• unknown knowns where reverse causality distorts relationships.

In a study of US judges granting bail, a machine learning algorithm predicted reoffending risks so accurately that following its guidance could have reduced the crime rate of released defendants by three-quarters while keeping release numbers constant. Shawn Fanning creating Napster in 1999 introduced free music file sharing, catching the music industry by surprise as an unknown unknown.

Combining human and machine predictions through a cognitive division of labor dramatically reduces error rates by leveraging complementary strengths.

In the Camelyon Grand Challenge detecting metastatic breast cancer from slides, an AI algorithm achieved 92.5 percent accuracy and a human pathologist achieved 96.6 percent. Combining both raised accuracy to 99.5 percent and cut errors by 85 percent.

3. The Anatomy of Decisions and Judgment

Every decision breaks down into multiple parts. You cannot understand the impact of artificial intelligence without looking inside this structure.

A decision consists of six distinct elements where prediction is only one component, meaning cheaper prediction increases the value of human judgment and data collection.

The anatomy of a decision includes:

• input data;
• training;
• prediction;
• judgment;
• action;
• outcome with feedback.

Potential London cabbies spent three years on average learning 'The Knowledge' to master route prediction. Mobile GPS navigation apps provided real-time route predictions to all drivers, commoditizing that prediction superpower.

The Anatomy of a Decision

Machine prediction substitutes for human prediction. Judgment and action remain in human hands as complements.

Better and cheaper prediction increases the overall value and frequency of human judgment. Prediction machines generate the forecast, but they do not provide judgment regarding the relative rewards of taking different actions. As artificial intelligence takes over prediction, you focus entirely on judgment, evaluating payoffs and trade-offs.

Credit card companies like Mastercard, Visa, and American Express predict fraud while using human judgment to weigh the sliding scale of risk, interest rates, and customer satisfaction. Figuring out payoffs and making judgments is costly and requires time, deliberation, or experimentation. Because experimentation involves making mistakes, judgment carries direct costs that slow down decision-making.

Economists J. P. Dubé and Sanjog Misra designed pricing experiments for ZipRecruiter by randomly assigning different prices to customer leads to determine how customers responded. This process achieved a 50 percent daily profit increase when raising prices for certain new customers.

Judging Fraud

When human judgment cannot be easily codified into rules, artificial intelligence can learn to predict human judgment by observing many examples. Instead of writing explicit rules for every complex scenario, systems can be trained by observing what humans do in repeated situations.

Grammarly was founded in 2009 by Alex Shevchenko and Max Lytvyn. It uses machine learning to predict what a human editor would do, correcting grammar and assessing whether stylistic deviations are preferred by human readers.

Hacking the Humans

Prediction machines struggle with rare events and situations where relevant data does not exist. Because prediction relies heavily on historical data, machines fail when encountering rare events or unprecedented strategic shifts where past behavior cannot guide future actions.

Statistician Abraham Wald used modeling to solve a missing-data problem during World War II by advising engineers to add armor to the parts of returning bombers that lacked bullet holes. He inferred that planes hit in those spots elsewhere failed to return.

Research demonstrates that relying on machine predictions reduces variance and can mitigate systemic human biases present in traditional managerial discretion. Working paper 23180 from the National Bureau of Economic Research regarding human decisions and machine predictions revealed that algorithms improve consistency and reduce demographic disparities.

4. Workflows, Complexity, and Tools

Integrating artificial intelligence requires reengineering workflows, handling high stakes, and using structured design canvases.

Traditional automation relied on rigid if-then rules within tightly controlled environments. Prediction machines identify more environmental conditions, allowing agents to navigate complex, uncontrolled settings and execute flexible contingent actions.

Early mail delivery robots required ultraviolet chemical trails on carpeted floors, whereas modern autonomous systems use sophisticated sensors to predict their environment and adjust accordingly. The original Mailmobile cost between 10000 to 12000 dollars in the 1970s, which equals approximately 50000 dollars today. It traveled at a speed of 1 mile per hour and took 20 minutes to complete a mail delivery that took humans 2 hours.

Full automation occurs when the economic returns to machines performing data collection, prediction, judgment, and action exceed the returns of keeping humans in the loop. Rio Tinto deployed 73 self-driving haul trucks in 2016 in the remote Pilbara region of Australia, saving 15 percent in operating costs and running twenty-four hours a day without drivers in daytime temperatures reaching 50 degrees Celsius.

High versus Low Stakes

The value of an AI prediction depends heavily on the stakes involved, which are determined by the expected losses arising from prediction errors.

Predictions fail by degrees and in different directions, such as false positives and false negatives. When stakes are high, the costs of errors outweigh the benefits of cheaper and faster predictions, requiring complementary risk management, insurance, or human oversight.

Amazon product recommendations involve low stakes where a false positive merely results in a returned item and continued shopping, whereas Facebook content moderation involves high stakes where failing to catch offensive material causes severe user dissatisfaction. In the first quarter of 2018, Facebook took down 21 million pieces of adult nudity and sexual activity, took down or labeled 3.5 million pieces of violent content, and removed 2.5 million pieces of hate speech.

Spotify struggled with user-built playlists due to the WTF problem where mismatched songs did not scale, but succeeded by launching Discover Weekly and context-based playlists like music for singing in the car by using 700 songs identified by human editors as likely to be sung in that setting.

Impact of AI Tools on Workflows

The implementation of AI tools can fundamentally change workflows by rendering certain tasks obsolete while creating new ones.

Like classical computers, AI requires businesses to step back, study their objectives and workflows, and identify constituent tasks. Simply introducing AI without workflow redesign fails to yield productivity gains.

Ford reduced its North American accounts payable department by 75 percent from 500 employees by examining its purchasing workflow and using a system to sort difficult cases from easy ones. In contrast, Mazda employed only 5 people in accounts payable in the 1980s, and Mutual Benefit Life used 19 people across 5 departments taking 30 distinct steps to process life insurance applications. Goldman Sachs manages 146 distinct tasks involved in the initial public offering process.

When developing the iPhone in 2006 with a 3.5-inch LCD screen and only 3 weeks given to developers to find a working keyboard solution before launch, Apple engineers deconstructed the physical workflow of typing and used predictive algorithms on a soft keyboard to dynamically expand target touch areas based on the probability of the next letter.

The AI Canvas

The AI Canvas is a structured framework that decomposes decisions within a task to evaluate the potential role of a prediction machine.

The canvas forces clarity by breaking down a decision into seven specific elements, ensuring that organizations understand the exact predictive core, necessary data types, and judgment criteria before deploying AI. Atomwise uses an AI canvas to decompose the molecule selection task, where its AI predicts binding affinity to recommend promising pharmaceutical drug prospects using 38 million public data points on binding affinity as of July 2017.

To build an AI canvas for your own workflow, follow these steps:

• Define the action you are trying to do.
• Identify the core prediction you need to know.
• Determine the judgment used to value outcomes and errors.
• Establish metrics for task success as the outcome.
• Specify the input data needed to run the algorithm.
• Identify the training data needed for the predictive algorithm.
• Determine how feedback from outcomes will be used to improve the algorithm.

5. Strategy, Organization, and Society

Artificial intelligence alters firm boundaries, business strategies, labor markets, and geopolitical competition.

Missing Links in Automation

Complete automation of workflows is often fundamentally constrained by small, seemingly insignificant tasks known as missing links, which can derail the entire automated process if they cannot be solved. Amazon acquired Kiva in 2012 for $775 million to automate bringing shelves to workers. Despite this, Amazon employs forty thousand full-time human pickers and tens of thousands more part-time who handle approximately 120 picks per hour.

How AI Can Change Business Strategy

AI becomes a strategic C-suite agenda item when a business model trade-off is driven by uncertainty, and an AI prediction machine reduces that uncertainty enough to flip the optimal operational strategy. German e-commerce venture Otto used a database of 3 billion past transactions to predict customer demand with 90 percent accuracy, reducing inventory by 20 percent and annual returns by 2 million items.

What to Leave In and What to Leave Out

Prediction machines reduce uncertainty, enabling companies to write more contracts with specific conditions, which alters organizational boundaries by shifting capital equipment and routine labor toward outsourcing. Economists Silke Forbes and Mara Lederman found that major airlines retain control of weather-vulnerable routes instead of outsourcing them to regional partners due to high uncertainty and fast-changing contract costs.

A Whiff of Disruption

An AI-first strategy requires prioritizing long-term learning and data collection over immediate customer experience, revenue, and short-term operational performance. Google and Microsoft announced shifts from mobile-first to AI-first strategies, committing to prioritize intelligence and contextual awareness even if it disrupts other channels.

Liability Risks

AI algorithms can produce unintentional discrimination or disparate impact, creating severe liability risks for companies even when the underlying code is neutral. Latanya Sweeney discovered that searches for black-associated names like Lakisha or Trevon were 25 percent more likely to trigger arrest ads than white-associated names.

Is This the End of Jobs?

Economists view technological advances that make software cheaper to perform tasks as equivalent to opening free trade with a fictitious island of robots. From a free-trade perspective, allowing robots to handle specific tasks frees humans to focus on what they do best, such as judgment and complementary skills. Historical evidence from previous automation shifts shows that while specific jobs change, overall employment does not plummet in the long run.

Conclusion

Artificial intelligence transforms the economic landscape by reducing the cost of prediction, shifting the strategic focus from forecasting the future to exercising judgment.

As prediction becomes abundant and cheap, you must reevaluate your workflows, organizational boundaries, and strategic choices. To navigate this technological shift successfully, you can follow these practical rules:

1. Identify your core predictions by isolating the specific uncertainties that currently limit your operational efficiency and decision-making.
2. Evaluate your prediction dials to determine the optimal trade-off between accuracy, speed, and cost for each specific task.
3. Separate prediction from judgment by recognizing that machines provide probabilities, while humans supply the values, ethics, and preferences.
4. Redesign your workflows around machine outputs rather than merely automating existing human tasks.
5. Protect and develop your proprietary data assets to train better prediction engines and build sustainable competitive advantages.
6. Prepare your organization for changing labor dynamics by focusing human development on areas where judgment and empathy matter most.

10 Key Ideas

1. When the price of a foundational input drops significantly, society uses vastly more of it and applies it to entirely new domains.
2. Moving from 98 percent to 99.9 percent accuracy means mistakes fall by a factor of twenty, which changes the nature of the activity from incremental to transformational.
3. Combining human and machine predictions through a cognitive division of labor dramatically reduces error rates by leveraging complementary strengths.
4. A decision consists of six distinct elements where prediction is only one component, meaning cheaper prediction increases the value of human judgment and data collection.
5. When human judgment cannot be easily codified into rules, artificial intelligence can learn to predict human judgment by observing many examples.
6. The implementation of AI tools can fundamentally change workflows by rendering certain tasks obsolete while creating new ones.
7. The AI Canvas is a structured framework that decomposes decisions within a task to evaluate the potential role of a prediction machine.
8. AI becomes a strategic C-suite agenda item when a business model trade-off is driven by uncertainty, and an AI prediction machine reduces that uncertainty enough to flip the optimal operational strategy.
9. AI algorithms can produce unintentional discrimination or disparate impact, creating severe liability risks for companies even when the underlying code is neutral.
10. Artificial intelligence transforms the economic landscape by reducing the cost of prediction, shifting the strategic focus from forecasting the future to exercising judgment.