Book summary “Business Analytics”: summary

Introduction

You face a constant challenge when making strategic choices inside data-rich business environments. Raw information flows from every transaction, yet translating these massive inputs into clear insights often feels overwhelming.

Business analytics combines descriptive, predictive, and prescriptive techniques using Microsoft Excel, Analytic Solver, and R to translate raw data into actionable business decisions.

Sanjiv Jaggia structures this book to guide you through the complete analytical process from raw figures to optimal choices. You will master data management, visual summaries, probability concepts, statistical inference, linear regression, data mining, forecasting, and prescriptive optimization.

• How do you classify different types of business data and manage databases effectively?
• Which statistical tools help you test hypotheses and build reliable regression models?
• How can you apply data mining and optimization techniques to solve complex operational problems?

1. Foundations of Business Analytics and Data Management

Effective business analytics begins with understanding data types, measurement scales, and rigorous data wrangling and database management.

Types of Analytics and Data

Business analytics techniques are categorized into descriptive, predictive, and prescriptive analytics based on their function and level of sophistication. Descriptive analytics summarizes past data to answer what happened using tools like reports and dashboards. Predictive analytics uses historical data and models to estimate the likelihood of future outcomes. Prescriptive analytics applies optimization and simulation algorithms to suggest specific actions and decisions.

For an online music streaming company, a descriptive question asks how many country songs were skipped in the first quarter of 2020, while a predictive question identifies the key factors influencing a listener's music choices. Big data is characterized by high volume, velocity, and variety, alongside veracity and generated business value. Volume refers to massive data amounts, velocity represents rapid generation speed, and variety encompasses structured and unstructured formats. Veracity ensures data credibility, while value measures practical insights gained.

Google's artificial intelligence system analyzed 46 billion data points on over 200,000 adult patients to predict hospital visit outcomes and patient death 24 to 48 hours earlier than traditional models. Data is divided into structured data residing in predefined row-column formats and unstructured data lacking such definitions. Structured data capture business transactions and represent about 20 percent of business data. Unstructured data comprise textual reports, emails, social media content, and multimedia files, making up the remaining 80 percent.

Emily Hernandez found 1,420 flight combinations on Orbitz.com that could be structured into a table, while user reviews of hotels constituted unstructured data. Variables are classified as categorical with qualitative labels or numerical with quantitative values, with numerical variables further split into discrete or continuous. Categorical variables use names or labels to represent distinct groups. Numerical variables use meaningful numbers, where discrete variables assume a countable number of values and continuous variables encompass uncountable values within an interval.

A company's stock price or the number of children in a family are discrete numerical variables, whereas a newborn's weight or temperature are continuous variables. Data measurement scales progress in sophistication from nominal and ordinal for categorical variables to interval and ratio for numerical variables. Nominal data differ merely by name or label. Ordinal data can be categorized and ranked, but differences are meaningless. Interval data feature meaningful differences but lack a true zero point. Ratio data possess all interval properties plus a true zero point allowing meaningful ratios.

Hotel survey ratings from 1 to 5 stars are ordinal data, temperature on the Fahrenheit scale is interval data, and monetary sales or profits are ratio data.

Data Management and Wrangling

Data wrangling is the critical and time-consuming process of retrieving, cleansing, integrating, transforming, and enriching raw data to support subsequent analysis. Organizations face significant barriers when they fail to clean and organize big data. Analytics professionals typically spend between 50 and 80 percent of their time on these mundane preparation tasks before actual analytics can be applied. Relational databases use two-dimensional tables with rows and columns to store data, while entity-relationship diagrams model database structures through entities, attributes, and relationships.

Relationships between entities can be one-to-one, one-to-many, or many-to-many. A primary key uniquely identifies each instance of an entity, a foreign key matches the primary key of a related entity, and a composite primary key consists of multiple attributes. Organic Food Superstore uses tables for CUSTOMER, ORDER, ORDER_DETAIL, and PRODUCT connected via primary and foreign keys. Structured Query Language enables efficient data retrieval from relational databases, whereas enterprise data warehouses and data marts integrate data across an organization for managerial decision-making.

Basic SQL statements rely on the Select, From, and Where keywords. Enterprise data warehouses collect data from multiple departments through an ETL process, while smaller-scale data marts often utilize a multidimensional star schema comprising dimension and fact tables. Missing values reduce usable observations and introduce bias, and they can be managed using either the omission strategy or the imputation strategy. The omission strategy deletes observations with missing values, which can lead to severe data loss if missingness is widespread. The imputation strategy replaces missing values with reasonable estimates, such as the simple mean for numerical variables or the most frequent category for categorical variables.

Sarah Johnson analyzes restaurant reviews and applies omission and mean imputation to handle missing customer ratings for ambience, cleanliness, service, and food. You should follow specific preparation guidelines:

• Detect missing values using conditional formatting or missing value detection functions.
• Apply the omission strategy to delete records with missing values if the missing amount is small.
• Apply the imputation strategy to replace missing numerical values with column means and categorical values with predominant categories.

2. Visualizing and Summarizing Business Data

Visual displays and summary measures condense raw categorical and numerical variables into interpretable patterns while guarding against graphical distortions.

Data Visualization Methods

For categorical variables representing names or labels, a frequency distribution counts observations per category, while relative frequency provides proportions calculated by dividing frequency by sample size. Vertical or horizontal bar charts display these proportions through proportional bar lengths.

An urban university surveyed 600 respondents about their commute, finding 273 used public transportation, 141 walked, and 111 biked.

• Group the categorical data into well-defined categories.
• Record the number of observations falling into each category.
• Calculate relative frequency by dividing each category frequency by the sample size.
• Construct a bar chart with bar lengths proportional to category frequencies or relative frequencies.

Unlike categorical variables, numerical variables require constructing intervals or classes. Guidelines dictate that intervals must be mutually exclusive and exhaustive, numbering typically between 5 and 20, with easily recognizable limits derived approximately by dividing the range by the number of intervals.

• Determine the number of intervals, typically ranging from 5 to 20.
• Approximate interval width using the formula (largest observation minus smallest observation) divided by the number of intervals.
• Ensure interval limits are easy to recognize and interpret.
• Verify intervals are mutually exclusive and exhaustive so each observation falls into one and only one interval.

Charts and graphs must follow strict construction guidelines to avoid unintentional or purposeful visual distortions of data.

• Use the simplest graph for a given set of data and avoid unnecessary adornments.
• Clearly mark axes with numbers of their respective scales and label each axis.
• Keep each bar or rectangle of the same width in bar charts and histograms.
• Avoid giving the vertical axis a very high upper limit that compresses data changes.
• Avoid stretching the vertical axis so that data changes appear more pronounced than warranted.

Numerical Summary Measures

Measures of central location attempt to find a typical or central value for numerical data, utilizing the mean, median, and mode.

• Calculate the sample mean by summing all observations and dividing by the sample size n.
• Arrange data in ascending order to find the median as the middle value for odd sample sizes or the average of two middle values for even sample sizes.
• Identify the mode as the most frequently occurring observation.

Dispersion is measured by the range, interquartile range, mean absolute deviation, variance, and standard deviation. William Sharpe's ratio measures excess reward per unit of risk by dividing the difference between an investment's mean return and the risk-free rate by its standard deviation.

• Calculate the range as the difference between the maximum and minimum observations.
• Calculate the interquartile range as IQR = Q3 minus Q1.
• Calculate sample variance as the average of squared differences from the mean divided by n minus 1.
• Take the positive square root of the variance to determine the standard deviation.
• Compute the Sharpe ratio by subtracting the risk-free rate from the mean return and dividing by the standard deviation.

3. Probability Concepts and Statistical Inference

Every probability must be a value between zero and one, and the sum of probabilities of all mutually exclusive and exhaustive events in a sample space must equal one.

Probability rules and sampling distributions provide the theoretical foundation for estimating population parameters and conducting hypothesis tests.

Probability measures the likelihood of an event occurring. A value of zero represents an impossible event, and a value of one indicates a definite event. The complement rule states that the probability of an event's complement is one minus the probability of the event itself. Probabilities are categorized into subjective probabilities based on personal judgment, empirical probabilities based on observed relative frequency, and classical probabilities based on logical analysis of equally likely outcomes.

In a particular city over ten years, 152 out of 180 restaurants opened and closed within a year, yielding an empirical probability of restaurant failure of 152/180 equals 0.84.

Anthony calculates a 75 percent chance of getting an A in Statistics, 55 percent in Managerial Economics, and 40 percent in both, leading to an 90 percent chance of getting an A in at least one course.

• Define relevant events and their associated prior probabilities.
• Identify the conditional probabilities for each scenario state.
• Compute joint probabilities by multiplying prior probabilities by conditional probabilities.
• Calculate the denominator total probability by summing all joint probabilities.
• Divide the specific joint probability by the total probability to find the posterior probability.

In a lie-detector test with a 99 percent truth-telling prior probability and 95 percent test reliability with a 0.5 percent false-positive rate, the posterior probability of telling the truth when a lie is detected drops to 0.3426.

Estimation and Hypothesis Testing

A population parameter is an unknown constant describing a population, whereas a sample statistic is a variable calculated from a random sample that serves as an estimator. The expected value of the sample mean equals the population mean, and its standard error equals the population standard deviation divided by the square root of the sample size. The Central Limit Theorem states that the sampling distribution of the sample mean becomes approximately normal for any population as long as the sample size is sufficiently large.

A pizza chain manager analyzes 16-inch pizzas with a population standard deviation of 0.8 inches, showing that samples of size 4 have a lower standard error of 0.4 than samples of size 2 with a standard error of 0.565.

A confidence interval provides a range of values calculated from sample statistics that contains the unknown population parameter with a specified level of confidence.

• Specify the point estimate for the population parameter.
• Determine the sample size and degrees of freedom if using the t distribution.
• Find the appropriate reliability factor corresponding to the desired confidence level.
• Calculate the margin of error by multiplying the reliability factor by the standard error.
• Add and subtract the margin of error from the point estimate to establish the interval limits.

For 100 Amazon Prime customers with annual expenditures, a 95% confidence interval is constructed using Excel and R to show average expenditures fall between $1,240.24 and $1,373.64.

Hypothesis testing resolves conflicts between a null hypothesis representing the status quo and an alternative hypothesis representing a new claim or challenge. The p-value approach measures the probability of observing a sample result at least as extreme as the one obtained, assuming the null hypothesis is true.

• Identify the relevant population parameter of interest.
• Determine whether the test is one-tailed or two-tailed.
• Formulate the null hypothesis to include an equality sign representing the status quo.
• Formulate the alternative hypothesis to reflect the claim to be established.

• Specify the null and alternative hypotheses.
• Specify the significance level alpha representing the allowed probability of a Type I error.
• Calculate the value of the test statistic using sample data and hypothesized parameters.
• Calculate the exact p-value based on the test statistic and the alternative hypothesis.
• Compare the p-value to alpha and state the final conclusion and interpretation.

4. Linear Regression and Advanced Modeling

Regression models quantify relationships between variables, evaluate goodness of fit, and incorporate nonlinear transformations and interaction effects.

Regression analysis is widely used in predictive analytics to assess relationships between variables and predict target outcomes based on predictors. Relationships can be deterministic, where the response is uniquely determined by predictors as in physical sciences, or stochastic, which include a random error term due to omitted unquantifiable factors. Models can be simple with one predictor or multiple with several predictors. Fiona Schmidt uses College Scorecard data from 116 colleges to predict post-college earnings based on cost, graduation rate, debt repayment percentage, and city location.

Dummy variables represent categorical data numerically as 1 or 0, but you must avoid the dummy variable trap by using one fewer dummy than total categories. The category assigned a value of 0 is called the reference or benchmark category against which others are compared. Including as many dummy variables as categories creates perfect multicollinearity, making estimation impossible. A government researcher analyzing retail sales and GNP by quarters uses three dummy variables for quarters one through three, leaving the fourth quarter as the reference category.

• Ensure the number of dummy variables representing a categorical variable is one less than the number of categories if the model includes an intercept.

Model fit is assessed using the standard error of the estimate, the coefficient of determination, and the adjusted R-squared. The standard error of the estimate measures residual dispersion in the same units as the response variable. R-squared indicates the proportion of sample variation explained by the model, but adjusted R-squared must be used for comparing models with different numbers of predictors because it penalizes adding variables. When comparing three models of post-college earnings, Model 3 is preferred because it achieves the lowest standard error of the estimate and the highest adjusted R-squared of 0.4292, meaning 42.92 percent of variation is explained.

Hypothesis testing in regression evaluates both the joint significance of all predictors and the individual significance of specific coefficients. Joint significance uses an F-test to determine if any predictor has a linear relationship with the response variable, based on the ratio of mean square regression to mean square error. Individual significance uses a t-test to determine if a specific slope coefficient differs from a hypothesized value. Evaluating Model 3 for college earnings at a 0.05 significance level shows that the predictor variables are jointly significant, with an exact p-value of 7.56 x 10^-13.

• Divide the computer-generated p-value in half when specifying a one-tailed test with a hypothesized value of zero.

Valid OLS estimation and hypothesis testing rely on core classical linear regression assumptions regarding the error term and predictors. Assumptions require linearity in parameters, an expected error term of zero, no perfect multicollinearity, homoskedasticity, no serial correlation, no endogeneity, and a normally distributed error term. Violations compromise unbiasedness or invalidate standard error estimates. A study on age and happiness reveals a nonlinear pattern where happiness decreases up to age fifty and increases thereafter, violating the linearity assumption across 24 interviewed individuals.

• Plot residuals against predictor variables, predicted values, or time sequentially to inspect model assumptions and detect departures.

Violations of classical assumptions can be addressed through variable transformations, dropping redundant predictors, or using robust standard errors. Multicollinearity can be remedied by dropping collinear variables or checking variance inflation factors. Changing variability and serial correlation can be handled by calculating robust standard errors in software packages like R, since OLS point estimates remain unbiased even when standard errors are distorted. In examining convenience store sales against square footage, residual plots show fanning out, indicating changing variability, which is corrected using robust standard errors via R's sandwich package.

• Install and load the sandwich package in R to compute robust standard errors when heteroskedasticity or serial correlation is present.
• Use the vcovHC or NeweyWest function in R to calculate robust standard errors for the variance-covariance matrix.
• Take the square root of the diagonal values of the variance-covariance matrix to obtain corrected standard errors.

Nonlinear Models and Cross-Validation

Cross-validation partitions sample data into training and validation sets to independently assess a model's predictive power on unseen data and prevent overfitting.

Interaction variables allow the partial effect of a predictor variable on the response variable to depend on the value of another predictor variable. While standard linear regression assumes constant partial effects, incorporating a product term of two predictors captures interactions between dummy variables, dummy and numerical variables, or two numerical variables. The slope coefficient of the interaction term measures the enhancement or reduction in effect when both variables are present. In a study of business graduates, the interaction variable between an MIS concentration and a statistics minor showed a positive coefficient of 3491.5, yielding an adjusted R-squared of 0.8029 for Model 2 compared to 0.7901 for Model 1.

• Create a new data column for the interaction variable by multiplying the column entries of the two predictor variables.
• Include the interaction term in the regression equation alongside the individual predictor variables.
• Estimate the model using ordinary least squares and interpret the combined slope coefficients.

Quadratic regression models capture nonlinear relationships where the slope changes in magnitude and sign, creating U-shaped or inverted U-shaped curves. By adding a squared predictor variable to the linear framework, you can model phenomena where an independent variable's effect initially increases and then decreases, or vice versa. Ioannes Papadopoulos examined the relationship between wages, education, and age for 80 workers, finding that the quadratic model produced an adjusted R-squared of 0.8257 compared to 0.6088 for the linear model, with 51 years as the optimal age at which hourly wages are maximized.

• Create a squared data column containing the squared values of the numerical predictor.
• Estimate the quadratic regression model including both the linear and squared terms.
• Calculate the partial effect approximated by b1 plus 2 times b2 times x.
• Find the optimal value of x by solving b1 plus 2 times b2 times x equals zero.

Logarithmic transformations convert changes in variables into percentage changes, enabling the modeling of nonlinear curves while keeping linear estimation frameworks. Common transformations include log-log models, logarithmic models, and exponential models. In evaluating rental data in Ann Arbor, Michigan, Model 4 log-transformed rent and square footage, achieving an R-squared of 0.7554 based on actual rent values rather than log rent.

• Transform selected variables into natural logarithms using the natural log function.
• Estimate the chosen regression model.
• For models with logged response variables, calculate predictions by exponentiating the sum of the predicted log value and half of the squared standard error of the estimate.
• Compute R-squared for log-transformed response models by squaring the sample correlation coefficient between the observed response and the corrected predicted values.

The linear probability model applies standard linear regression to binary response variables to predict probabilities of success, though it suffers from unbounded probability estimates. Analyzing 30 recent mortgage applications, the linear probability model predicted loan approval based on down payment percentage and income-to-loan ratio, yielding an accuracy rate of 83.33 percent with a default cutoff of 0.5. Logistic regression uses a nonlinear S-shaped logistic specification estimated via maximum likelihood to constrain predicted probabilities strictly between 0 and 1. In the same mortgage study, the logistic regression model correctly classified 86.67 percent of the 30 observations, outperforming the linear probability model.

• Designate the binary choice outcome as 1 for success and 0 for failure.
• Estimate the regression equation using ordinary least squares with binary response data.
• Compute predicted probabilities by multiplying coefficients by predictor values.
• Calculate accuracy rates by comparing binary predictions against actual outcomes using a 0.5 cutoff.
• Specify the logistic regression model linking the log odds of the binary response to predictors.
• Estimate parameters using maximum likelihood estimation in statistical software.
• Evaluate model performance using accuracy rate, sensitivity, and specificity based on a selected cutoff.

Overfitting occurs when a model conforms too closely to sample quirks, failing on new data. In analyzing project manager salaries, comparing models with the holdout method using 150 training and 50 validation observations revealed that simpler Model 1 outperformed complex Model 2 in validation root mean squared error, reversing the training set conclusion. With sufficiently large sample sizes like the 209 single-family houses sampled in Ames, Iowa, standard tests of significance lose practical meaning because sampling variability approaches zero.

• Partition the original sample dataset into training and validation sets or into k distinct subsets.
• Estimate competing models using only the training set observations.
• Generate predictions for the validation set using the training set estimates.
• Calculate the root mean squared error for linear models or accuracy rate for binary models on the validation set.
• Select the preferred model based on superior validation performance rather than training fit.
• Rely on economic intuition and model validation rather than tests of significance when confronting big data.

5. Supervised and Unsupervised Data Mining

Data mining applies analytical techniques to uncover hidden structures and patterns in data, serving as a building block for machine learning and artificial intelligence. While artificial intelligence simulates human cognitive functions and machine learning employs self-learning algorithms without human intervention, data mining provides the underlying analytical process to find patterns and relationships.

Data mining algorithms uncover hidden structures through supervised classification and prediction, as well as unsupervised clustering and association rules.

Uber uses machine learning algorithms to predict rider demand and dispatch drivers, whereas a public transportation department might use data mining for traffic data and AI for congestion issues.

Supervised Learning Methods

The Cross-Industry Standard Process for Data Mining methodology provides a six-phase lifecycle framework for conducting data mining projects from business understanding to deployment. Practitioners use it for its holistic approach, spending about 80 percent of project time on data understanding, cleansing, transformation, and preparation leading up to modeling.

• Business understanding: understand the project objectives and gather situational context.
• Data understanding: collect relevant data and conduct preliminary analyses.
• Data preparation: perform record and variable selection, data wrangling, and cleansing.
• Modeling: select and execute data mining techniques and set parameter values.
• Evaluation: evaluate competing models against specific criteria to select the best one.
• Deployment: develop actionable recommendations and a strategy for monitoring and feedback.

Data mining algorithms are divided into supervised techniques where a target variable is identified and unsupervised techniques where no target variable is specified. Supervised techniques relate predictor variables to a known target variable for classification or prediction, whereas unsupervised techniques focus on exploratory data analysis, dimension reduction, and pattern recognition without a target variable. A marketing manager classifies customers as buyers or nonbuyers using supervised data mining, while retail companies use unsupervised pattern recognition to segment customers.

Data partitioning divides a data set into training, validation, and optional test sets to detect overfitting, while oversampling adjusts the class distribution for rare target classes. Training sets build the model, validation sets evaluate performance and fine-tune complexity, and test sets assess final performance on unseen data. Oversampling intentionally overweights rare classes in the training set to prevent models from defaulting to the majority class. An online retailer sends promotional emails to 10,000 potential customers with only 200 responses, resulting in a rare target response rate of 2 percent.

• Partition 60 percent of data into a training set and 40 percent into a validation set, or use a 70/30 split.
• For three-way partitioning, use 50 percent training, 30 percent validation, and 20 percent test data.
• Apply oversampling only to the training data set while keeping validation and test sets at original class proportions.

Unsupervised Learning and Association

Unsupervised data mining discovers hidden structures and patterns without requiring a known response variable. Unlike supervised learning that relies on a target variable to train models, unsupervised techniques allow computers to explore data and group observations without specific analyst guidance. Aliyah Williams used cluster analysis to group candy bars by their nutritional facts in her vending machine business.

Hierarchical clustering uses an iterative process to build a tree of clusters via agglomerative or divisive approaches. Agglomerative clustering works bottom-up by starting with each observation as its own cluster and merging them, whereas divisive clustering works top-down by starting with all observations in one cluster and separating them.

• Select variables that are numerical or categorical or mixed data.
• Measure dissimilarity using Euclidean or Manhattan distance, matching, Jaccard's coefficient, or Gower's coefficient.
• Apply z-score standardization to make numerical variables independent of scale.
• Choose a linkage method to evaluate distance between clusters.
• Inspect the resulting dendrogram or banner plot to determine the number of clusters.

6. Time Series Forecasting and Prescriptive Analytics

Time series data consists of sequential observations ordered over time and are characterized by trend, seasonal, cyclical, and random components. The trend component shows long-term upward or downward movements. The seasonal component represents repetitions over a year or less. The cyclical component captures multi-year wavelike business cycles. The random component captures unexplained movements.

Apple's quarterly revenue data from 2010 to 2018 shows a persistent upward movement alongside a seasonal pattern where revenue peaks during the holiday quarter.

Time series models project historical trends into the future, while simulation and optimization prescribe optimal business decisions under constraints.

Time Series Forecasting

Cross-validation for time series data requires sequential partitioning into early training sets and later validation sets to preserve intrinsic temporal patterns. You evaluate competing models using validation forecast errors via MSE, MAD, and MAPE.

• Split the time series sequentially into early training sets and later validation sets.
• Evaluate competing models using validation forecast errors via MSE, MAD, and MAPE.
• Reestimate the selected preferred model using the entire combined dataset to make future forecasts.

The moving average technique smooths out random variations by averaging a fixed number of the most recent observations. As a new observation becomes available, the oldest is dropped and the average moves forward. Simple exponential smoothing continually updates the level of a time series by assigning exponentially decreasing weights to older observations.

• Compute the average from a fixed number of the most recent observations.
• Include the newest observation and drop the oldest observation as new data becomes available.

Regression models can capture both long-term linear trends and seasonal variations using a time variable and seasonal dummy variables. Nonlinear regression models such as exponential, quadratic, and cubic trend models capture accelerated growth or directional reversals in a time series. Advanced smoothing methods like Holt and Holt-Winters incorporate trend and seasonality into recursive exponential smoothing equations.

• Define three dummy variables representing the first three quarters, using the fourth quarter as reference.
• Estimate the linear regression model incorporating the trend variable and seasonal dummy variables.

Simulation and Optimization

Prescriptive analytics uses simulation and optimization algorithms to advise on what businesses should do by quantifying the effects of different actions. Monte Carlo simulation models the risk and uncertainty of real-world processes by using random sampling to mimic the odds of all possible outcomes.

In 1994, Orange County, California relied on an average low interest rate for investments, resulting in a $2 billion loss and municipal bankruptcy.

Linear programming is a subset of optimization techniques used to maximize or minimize a linear objective function subject to linear resource constraints. In linear programming, binding constraints utilize all available resources and directly affect the optimal solution, whereas nonbinding constraints feature unused slack or surplus. Linear integer programming is utilized when decision variables must take on whole numbers rather than fractional values to reflect real-world operational realities.

• Open the spreadsheet model and ensure the Solver add-in is activated.
• Designate decision variable cells and set up objective function formulas using SUMPRODUCT.
• Launch Solver, assign the objective cell, and select Max or Min.
• Add constraints including binary or integer restrictions for the decision variables.
• Choose Simplex LP as the solving method and execute Solver.

10 Key Ideas

1. Business analytics professionals spend between 50 and 80 percent of their time on mundane data preparation tasks like cleansing, integrating, and transforming raw information.
2. Charts and graphs must follow strict construction guidelines to avoid unintentional or purposeful visual distortions of numerical and categorical data.
3. Probability rules and sampling distributions provide the theoretical foundation for estimating unknown population parameters and conducting formal hypothesis tests.
4. Hypothesis testing resolves conflicts between a null hypothesis representing the status quo and an alternative hypothesis representing a new claim or challenge.
5. Valid Ordinary Least Squares estimation and hypothesis testing rely on core classical linear regression assumptions regarding the error term and predictors.
6. Logistic regression uses a nonlinear S-shaped logistic specification estimated via maximum likelihood to constrain predicted probabilities strictly between 0 and 1.
7. Data partitioning divides a dataset into training, validation, and optional test sets to detect overfitting, while oversampling adjusts the class distribution for rare target classes.
8. Hierarchical clustering uses an iterative process to build a tree of clusters via agglomerative bottom-up approaches or divisive top-down approaches.
9. Cross-validation for time series data requires sequential partitioning into early training sets and later validation sets to preserve intrinsic temporal patterns.
10. Linear programming is a subset of optimization techniques used to maximize or minimize a linear objective function subject to linear resource constraints.