Introduction
You use digital tools every day to navigate life, work, and relationships. Yet these applications often fail you, locking you out of accounts or forcing you into rigid identity categories. You might assume these glitches are accidental technical oversights.
Sara Wachter-Boettcher demonstrates that these flaws are deliberate human choices. Technology products quietly encode systemic biases because creators build exclusively for normative experiences. Insular corporate cultures and narrow user models produce systems that ignore vulnerable people.
The book investigates workplace homogeneity, flawed data models, and intrusive forms. You will discover how algorithms enforce discrimination and how platform architectures enable harassment.
- How do technology companies normalize exclusion?
- Why do default settings harm vulnerable users?
- What changes can you demand from creators?
1. Product Blind Spots and Insular Culture
Technology is no longer a discrete sector. It forms core infrastructure powering modern life and democracy. As digital tools embed themselves into daily habits, their design flaws cause widespread harm.
Technology companies foster an insular workplace culture and optimize solely for positive experiences, creating products that fail users facing distress or systemic marginalization.
In late 2015, Pew Research reported that 20 percent of American adults go online almost constantly. When design teams build exclusively for normative experiences, products fail vulnerable users facing grief or medical trauma.
On Christmas Eve 2014, Facebook exposed early programmer Eric Meyer to its automated Year In Review feature. The tool displayed a photo of his six-year-old daughter Rebecca, who had died of brain cancer that year, surrounded by dancing illustrations and party streamers.
Workplace Homogeneity
Companies prioritize superficial entertainment features while underinvesting in critical safety and inclusivity. In March 2016, a JAMA Internal Medicine study revealed that smartphone assistants failed to understand queries like rape or domestic violence. Apple's Siri offered jokes instead of help. In August 2016, Snapchat released an anime-inspired filter that produced buck-toothed, squinty-eyed yellowface caricatures.
The tech industry maintains an insular work culture driven by homogeneity, youth worship, and exclusionary social rituals. Product designer Fatima faced continuous misogyny and was entirely ignored by male colleagues when presenting valid market research for a women's smartwatch. Designer Amélie Lamont heard her manager tell her she was so black she blended into her chair.
Apple's global workforce was 70 percent male in 2014 and barely shifted to 68 percent male by 2016. Technical roles remained 80 and 77 percent male respectively over the same period. Apple reported 9 percent black staff in 2016, dropping to 3 percent in leadership.
Google reported 81 percent male technical employees in 2016, with 1 percent black and 3 percent Hispanic technical employees. Leadership roles across all Google departments were 76 percent male, with 2 percent black and 1 percent Hispanic. Airbnb reported that 10 percent of its staff came from underrepresented groups in 2016, and 5 percent in technical roles.
Tragic Design Failures
Companies blame lack of diversity on a faulty educational pipeline. Top universities turn out black and Hispanic computer science and engineering graduates at twice the rate tech companies hire them, according to a 2014 USA Today analysis. Recruiters consistently ignore qualified candidates at events like the Grace Hopper Celebration of Women in Computing. Kaya Thomas, a black computer science student with app-building and internship experience, was ignored by recruiters from major tech companies there.
Senior leaders make final hiring decisions based on subjective criteria such as shared alma maters, personal referrals, and conformity to company lifestyle gimmicks. Former Facebook recruiters noted that diversity hiring incentives failed because final hiring approvals rested with senior leaders evaluating culture fit.
2. The Myth of the Average User
Designing products around narrow personas and statistical averages alienates diverse users, whereas treating atypical scenarios as stress cases builds robust tools.
You rely on fictional representations of your target audience to make them feel relatable to design teams. These profiles often encode unexamined assumptions about user identity, needs, and lifestyle.
Maggie Delano, a queer woman and MIT PhD candidate not interested in having children, found that period tracking apps like Glow assumed her primary journey was fertility treatment or avoiding pregnancy. Glow launched in 2013 with the mission of helping women get pregnant. By 2014, the company received 17 million dollars in funding and realized half its users wanted to avoid pregnancy.
Personas and Defaults
Attaching demographic details like stock photos, age, and gender to personas causes teams to wrongly conflate demographic averages with user motivations and needs. Client reactions to corporate personas often focus inappropriately on demographic realism rather than underlying goals.
Sara Wachter-Boettcher and her team presented a persona named Linda, a black female CEO, whose demographic details were rejected by a client who insisted a CEO should be an older white man.
Default system settings are designed choices that establish a standard, heavily influencing user behavior and implicitly framing one demographic group as normal while marginalizing others. Because most users never change default settings due to the default effect, these preselected options quietly reinforce cultural biases by presenting dominant groups as standard and others as special.
Seventy percent of New York City cab riders opted for default tipping options on touchscreens, which ranged from 20 to 30 percent. Madeline Messer analyzed 50 endless-runner games and discovered that nearly 90 percent featured free male default characters. Female characters were rarely available as defaults and appeared just 15 percent of the time while often costing extra. The average cost of purchasing a female character in games where they were not free reached 7.53 dollars, which was nearly 29 times the app download cost.
Reframing Edge Cases
The concept of an average user is a statistical myth when applied to human identity and physical dimensions, meaning that designing for the center results in designing for no one. Lt. Gilbert S. Daniels studied over 4000 pilots across 10 physical dimensions and found that not a single one fell within the middle 30 percent for all dimensions. Only 3.5 percent of pilots were average-sized on just three dimensions.
Labeling user needs outside the norm as edge cases dismisses important human scenarios. Re-envisioning atypical scenarios as stress cases allows design teams to evaluate where their product breaks down under pressure and make choices that accommodate a wide spectrum of human experiences.
Libby Bawcombe led a design session at National Public Radio to identify stress cases for news consumers, such as workers with limited phone access or individuals triggered by traumatic news.
Inclusive design and media creation can move past tokenism by focusing on normalization, presenting diverse characters and users as everyday standards rather than exceptions. By starting development without rigid demographic assumptions and treating characters or users as fundamentally human first, creators can make representation reflect the true makeup of the world, where women, people of color, and LGBTQ people combined make up 50 percent of the population.
Shonda Rhimes intentionally casts diverse actors across television shows like Scandal and Grey's Anatomy to normalize television and make it mirror how the world actually looks.
3. Vulnerable Interactions and Identity Erasure
Digital forms and real-name policies force users into vulnerable positions, invading privacy and reinforcing default whiteness while punishing marginalized populations.
Interaction design treats selection menus as minor details that demand quick actions. You ignore the emotional weight of requesting personal information.
Sara Wachter-Boettcher filled out a medical clinic form in the summer of 2014 in Philadelphia when seeking a birth-control refill. The new-patient PDF form asked a binary yes-or-no question about surviving childhood sexual abuse without context, forcing her to either open a traumatic door or lie.
Name Policies and Forms
Platforms enforce real-name policies to prevent trolling. These rules compromise the safety of LGBTQ individuals, political refugees, and stalking victims. Automated moderation flags unconventional names as fake, locking legitimate users out of their accounts.
In the year 2014, Shane Creepingbear, a member of the Kiowa tribe of Oklahoma, was locked out of Facebook on Indigenous Peoples' Day because his name violated platform standards. Other Native Americans such as Robin Kills the Enemy, Dana Lone Hill, and Lance Brown Eyes faced the same exclusion.
Rigid racial drop-down menus flatten complex identities and treat whiteness as the invisible default. During the 2010 US Census, approximately 19 million Latinos did not know how to answer the racial category question because Hispanic was classified as an ethnicity, leading them to select some other race.
Websites require users to select a title or honorific, embedding outdated marital status assumptions. Dr. Louise Selby, a British pediatrician, could not access her gym changing room because the chain's software automatically coded any user with the title doctor as male.
You can fix honorific collection with simple rules:
- You shouldn't ask users for their title.
- If you have to use a title field, make it an optional free-text field and not a drop-down list.
Redesigning Safety Tools
Open-text safety reporting fields allow users to post vague descriptions based on race. This practice encourages racial profiling and neighborhood suspicion. Nextdoor redesigned its Crime and Safety reporting tool in August 2016 after facing boycotts from officials in Oakland, California, resulting in a 75 percent reduction in racial profiling.
Structured form fields reduce biased behavior when you guide user input. You can improve reporting flows by following these steps:
- Focus on behavior, noting what the person was doing that made you suspicious.
- Enter fields about the suspect's race, age, gender, and appearance.
- If entering data in the race field, fill out at least two additional fields about appearance such as hair, top, bottom, or shoes.
Software engineer Aimee Gonzalez-Cameron experienced constant anxiety and system error messages when taking the SAT and GRE. Test registration forms could not fit her hyphenated Hispanic last name, creating persistent digital microaggressions that alienate users.
4. False Intimacy and Surveillance
Trauma and Delight
Tech companies use cheerful animations and forced celebrations to mask underlying data-gathering motives and create a false sense of intimacy. Features like automated memorial recaps and birthday animations distract you from realizing that platforms are primarily collecting personal data, such as exact birth dates, to increase advertising value.
Twitter plastered activist DeRay McKesson's feed with birthday balloons on the exact night he was live-tweeting his arrest during a protest in Baton Rouge.
Automated algorithms designed to resurface memories frequently force grieving or traumatized users to relive painful life events without an easy way to opt out. Because algorithms lack emotional intelligence and context, features meant to spark nostalgia often dredge up tragedies, divorces, or deaths, while companies make the features difficult or impossible to completely disable.
Facebook introduced On This Day and automated Friends Day videos that algorithmically stitched together images of wrecked vehicles and hospital beds set to peppy music for 1 billion users.
Online opt-in forms increasingly rely on passive-aggressive, condescending language to manipulate users into sharing data.
Borrowing tactics from pickup artist communities, companies use negative framing on pop-up close buttons to make you feel inadequate or foolish if you decline to hand over your email addresses. Pop-up windows feature rejection options stating phrases like no thanks, I hate saving money, or I am not interested in being awesome. Designers suffer from inattentional blindness, focusing so narrowly on positive engagement metrics that they fail to anticipate how features will fail vulnerable users. Trained to obsess over pixel-perfect aesthetics and positive emotional engagement, product teams remain oblivious to the harmful or inappropriate contexts in which their designs operate. In the famous invisible gorilla test and a similar study with radiologists, 83 percent of radiologists failed to notice a matchbook-sized gorilla superimposed onto lung scans.
Default Tracking and Inequality
Digital products use deceptive default settings and confusing user interfaces to trick you into sharing more personal data than intended. By eliminating balanced intermediate options and setting privacy or tracking permissions to the most permissive level by default, companies force you into an all-or-nothing choice.
Uber updated its iPhone application to default to tracking user locations at all times even when the app was closed, disabling the middle option of tracking only while using the app.
Tech companies use proxy data to infer missing demographic information, creating self-perpetuating feedback loops that bake bias into algorithms. When companies lack direct demographic information, they use behavioral patterns as stand-ins. These models rely on historical assumptions, which can misclassify you and reinforce systemic discrimination.
Broker Acxiom stored an average of 1,500 data points across 70 marketing clusters for hundreds of millions of consumers in 2012, and Google's privacy policy aggregation frequently misclassified professional women who searched for technical topics as men aged 35 to 44.
Aggressive digital tracking and data harvesting disproportionately harm lower-income and marginalized populations.
Because lower-income individuals rely heavily on mobile devices for internet access, they generate vast troves of location data that expose them to predatory advertising and subprime lending practices. Pew Research Center data showed that in 2016, 1 in 5 Americans making under $30,000 accessed the internet only via mobile phone, compared to 1 in 20 Americans making over $75,000.
5. Algorithmic Inequity and Bias
An algorithm breaks down tedious calculations into discrete steps that computers execute at massive scales. However, if an algorithm is built on faulty assumptions or incorrect steps, the computer will reliably reach the wrong conclusion without recognizing the error. Yelp's algorithm sorts through listings to recommend restaurants by weighing variables such as past user reviews, geographic proximity, and star ratings to produce a prioritized list.
Risk-scoring algorithms and image recognition systems encode historical biases behind sleek interfaces that foster an illusion of scientific neutrality.
Criminal Risk Scoring
Northpointe's Correctional Offender Management Profiling for Alternative Sanctions software uses 137 risk and needs factors, including non-criminal factors like how often someone moved or felt bored, to generate a 1 to 10 recidivism risk score. Because black people are incarcerated at 6 times the rate of white people in the United States, relying on historical arrest data means the system disproportionately flags black defendants as high-risk.
In Broward County, Florida, Bernard Parker, a black man with a prior record, was rated a 10 for recidivism risk by COMPAS and has not been arrested since, whereas Dylan Fugett, a white man with a similar record, was rated a 3 and was subsequently arrested three more times. Out of defendants predicted by COMPAS to commit a future crime, 6 out of 10 actually do so. ProPublica found that black defendants were twice as likely to be falsely flagged as high-risk by COMPAS compared to white defendants.
Training Data Failures
Deep neural networks use learning algorithms to parse training data, recognize patterns across multiple hierarchical layers, and categorize new images. If the training data or product testing fails to include diverse skin tones, the system will misidentify or demean people of color. Jacky Alciné found that Google Photos automatically and repeatedly categorized selfies of him and his friend as gorillas, placing over fifty photos from his set under the animal tag.
The failure of photo technology to accurately render dark skin tones dates back decades to physical photography standards built around white defaults. In the 1950s, Kodak calibration standards known as Shirley Cards depicted a white woman in a high-contrast outfit labeled normal. Film emulsions were manufactured without sensitivity to a dynamic range of brown and reddish skin tones because black customers were not targeted as a major demographic. Kodak only adjusted its film emulsions in the 1970s after furniture retailers and chocolate makers complained that wood grains and dark-chocolate varieties were not rendering correctly.
Word2vec analyzes how frequently words appear near each other in text corpora to build semantic relationships across 3 million word embeddings in the neural network. Because news articles and historical data reflect human biases, these natural language processing embeddings generate vectors linking specific genders to traditional roles, which automated hiring tools can then use to filter out qualified applicants. Talla utilized CV2vec in 2016 to automatically review résumés and match candidates to top-performing staff based on historical patterns.
Tech companies deliberately use clean, minimalist interfaces to foster an illusion of scientific objectivity and machine neutrality, hiding underlying design flaws and human bias. By stripping away editorial markers and presenting data through sleek aesthetics, search engines, recommendation systems, and risk-assessment tools encourage users to blindly trust their outputs as neutral facts rather than questioning their origins. Dylann Roof was radicalized after googling black on white crime and encountering white supremacist propaganda via a search engine interface framed as objective and informative. In 2012, nearly three in four US search engine users said most or all search information was accurate and trustworthy, and two out of three users believed search engines were fair and unbiased. Out of 33,000 people worldwide surveyed by Edelman in fall 2016, 64 percent of respondents trusted search engines for news and information, compared to 57 percent who trusted traditional media.
6. Platform Architecture and Harassment
Default nonreciprocal architectures and automated news feeds facilitate severe online harassment, spread misinformation, and shift moderation burdens onto volunteers.
Twitter enabled anyone to follow and tweet at you without approval. The founders did not anticipate abuse or build safety features into the core product. Critics and former employees described the platform as optimized to accommodate hate speech and designed for maximum ease of trolling due to homogenous leadership. By March 2010, users sent 50 million tweets per day, up from 2.5 million a day in January 2009. In 2011, 25 percent of black American internet users reported being on Twitter, compared to 9 percent of white American internet users. In a 2014 Pew Research Center study, 13 percent of people harassed online deleted a profile or changed a username, and 10 percent left an online forum.
Comedian Lindy West and actress Leslie Jones faced severe, sustained racist and misogynistic abuse and death threats on Twitter. Support teams initially dismissed the reports as not violating rules.
Telling victims of online harassment to ignore trolls is a subtle form of misdirection. This advice assumes harassers will eventually move on, but campaigns can persist for months.
After Milo Yiannopoulos with 388,000 followers was permanently banned from Twitter following a harassment campaign against Leslie Jones, he embraced the ban as a public relations victory. Banning high-profile figures can elevate them as martyrs for the alt-right movement and neofascism.
Reddit and Moderation Burdens
Reddit relied on independent topical forums called subreddits managed entirely by volunteer moderators. Management remained hands-off under the banner of absolute free speech. This model left you unsupported when communities were flooded with hate speech, harassment, and doxing. In 2016, over 250 million unique people visited Reddit each month, and half a billion visits made it the seventh most popular site in the United States.
The subreddit r/blackladies was inundated with racist posts after Ferguson protests. When moderators asked management for proactive solutions, they were told the trolls were not breaking core rules.
Fake News and Move Fast Culture
Facebook replaced human contractors on its Trending team with algorithms after facing accusations of political bias. Without human editorial oversight, the algorithm promoted false stories with sensational headlines to the top of user feeds. In the lead-up to the 2016 US presidential election, top fake stories on Facebook generated 8.7 million shares, comments, and reactions. Top real news stories on Facebook generated 7.4 million shares, comments, and reactions during the same period.
Tech companies operate under the Hacker Way and the motto to move fast and break things. Leadership believes software engineers can solve societal problems through technical code without worrying about consequences. Mark Zuckerberg published a 5,000-word manifesto about the future of Facebook in February 2017 while operating out of One Hacker Way.
7. Meritocracy and Alternative Models
The tech industry's obsession with meritocracy creates toxic work cultures and evades accountability, proving that diverse teams and empathetic leadership yield better outcomes.
Silicon Valley claims its success relies purely on merit. In reality, venture capitalists routinely fund founders from a traditional Silicon Valley cohort. A 2013 Reuters analysis found that 80 percent of 88 tech companies receiving Series A funding from top venture firms had founders from this traditional cohort. Only 10 percent of 187 Silicon Valley startups that received Series A funding in 2016 were woman-led.
GitHub maintained a rug emblazoned with United Meritocracy of GitHub in its waiting room until 2014. Executives like Sequoia Capital partner Greg McAdoo insisted the business is a meritocracy.
Sociologist Michael Young coined the term meritocracy in a 1958 book lampooning the British education system to depict a dystopian society divided by IQ testing. Michael Young wrote a 2001 piece in the Guardian expressing sadness that his satirical warning was ignored.
The Meritocracy Myth
Enshrining meritocracy and hustle culture in a company creates an environment where sexual harassment and ethical violations are protected under the guise of high performance. Uber embedded meritocracy and toe-stepping into its core values, allowing high-performing male employees to escape accountability for harassment. At the time of the writing, Uber had a valuation of $68 billion.
Engineer Susan Fowler faced sexual harassment on her first day at Uber, but human resources refused to punish her manager because he was a top performer. This eventually led her to publish a public blog post that triggered massive fallout, including a boycott campaign where 200,000 people deleted Uber within days. Susan Fowler exposed severe harassment and gender bias at Uber, which led the company to hire Eric Holder to investigate.
Focusing solely on fixing the educational pipeline for diverse tech candidates ignores hostile work environments that drive women and minorities out. In 1984, 37 percent of computer science majors were women, but National Science Foundation numbers from 2014 show that only 18 percent of computer science bachelor degrees went to women. Studies of science, engineering, and technology fields found that more than half of women quit their jobs due to hostile environments and extreme pressures.
Diverse Workplaces
Diverse teams consistently outperform homogenous teams because friction forces members to prepare better. A 2003 study of 177 national banks comparing financial performance and racial diversity demonstrated how inclusion drives stability. Female representation in top management led to an increase of $42 million in firm value.
Labeling diverse businesses under the single umbrella of the tech industry allows companies to evade existing regulations. Uber avoids being regulated as a traditional taxi service or auto loan financer by positioning itself as a technology app.
Building a successful tech company without toxic exceptionalism is possible when leadership prioritizes empathy. Messaging app Slack rejected tech chest-thumping by adopting healthy office mottos and valuing liberal arts perspectives. By mid-2016, Slack reached 3 million daily active users, with 40 percent of management positions held by women and 8 percent of engineers who were Black. CEO Stewart Butterfield asks designers to imagine what users might have experienced in their lives before using the software so the product remains respectful.
10 Key Ideas
- Technology companies foster an insular workplace culture and optimize solely for positive experiences, creating products that fail users facing distress or systemic marginalization.
- Designing products around narrow personas and statistical averages alienates diverse users, whereas treating atypical scenarios as stress cases builds robust tools.
- Digital forms and real-name policies force users into vulnerable positions, invading privacy and reinforcing default whiteness while punishing marginalized populations.
- Online opt-in forms increasingly rely on passive-aggressive, condescending language to manipulate users into sharing data, while aggressive digital tracking disproportionately harms lower-income populations.
- Risk-scoring algorithms and image recognition systems encode historical biases behind sleek interfaces that foster an illusion of scientific neutrality.
- Default nonreciprocal architectures and automated news feeds facilitate severe online harassment, spread misinformation, and shift moderation burdens onto volunteers.
- The tech industry's obsession with meritocracy creates toxic work cultures and evades accountability.
- Diverse teams consistently outperform homogenous teams because friction forces members to prepare better.
- Tech companies evade existing regulations and accountability by positioning themselves merely as technology apps rather than traditional service providers.
- Building a successful tech company without toxic exceptionalism is entirely possible when leadership prioritizes empathy, inclusion, and human well-being over raw scale and speed.