Multiple Regression

Key Takeaways

• Multicollinearity — the presence of high correlation among the independent variables in a regression model — is the most common and damaging statistical problem in petroleum engineering multiple regression applications, because when two or more predictors are strongly correlated (such as lateral length and proppant volume in unconventional well performance analysis, which tend to both increase together in larger completions), the regression algorithm cannot reliably separate their individual effects and the estimated coefficients for the correlated predictors have very large standard errors that make them statistically unreliable; the practical consequence is that the regression may indicate that lateral length has no significant effect on production rate even though both lateral length and proppant volume individually improve production, simply because the model cannot distinguish their separate effects when they always vary together; principal component regression (PCR) and partial least squares (PLS) regression address multicollinearity by transforming the correlated predictors into orthogonal components before fitting the regression model, at the cost of interpretability of the resulting coefficients; variable selection methods (stepwise regression, LASSO regularization, ridge regression) provide alternative approaches to managing multicollinearity by either excluding redundant predictors or shrinking their coefficients toward zero.
• The R-squared statistic (coefficient of determination) measures the proportion of variance in the dependent variable that is explained by the regression model, ranging from 0 (the model explains none of the outcome variability) to 1 (the model explains all the variability perfectly): in petroleum well performance regression, R-squared values of 0.3-0.6 are common because the variability in well production rates is driven by a combination of controllable completion parameters (which the regression may capture reasonably) and uncontrollable geological heterogeneity (reservoir thickness, natural fracture density, organic content variations) that the available predictors cannot fully characterize; R-squared values close to 1 in a petroleum dataset are suspicious and usually indicate overfitting (where the model has been tuned to fit the training data too closely at the expense of predictive validity on new data), particularly when the number of predictors is large relative to the number of observations; the adjusted R-squared (which penalizes for the number of predictors and cannot increase by adding uninformative variables) and the cross-validated R-squared (which evaluates predictive performance on data not used in fitting) are more reliable measures of model quality than the raw R-squared that always increases when predictors are added.
• Heteroscedasticity (non-constant variance of the regression residuals across the range of the predicted values) is common in petroleum production data because well production rates span multiple orders of magnitude and the variance of production often increases with the mean production rate (larger wells have both higher average production and higher production variability); ordinary least squares (OLS) regression assumes constant residual variance and provides inefficient estimates when heteroscedasticity is present, with the confidence intervals for regression coefficients being systematically incorrect (usually too narrow for high-production observations and too wide for low-production observations); heteroscedasticity is detected by plotting the regression residuals against the predicted values (fan-shaped patterns indicate increasing variance with prediction) and is addressed by log-transforming the dependent variable (converting the regression to a multiplicative model that is often more appropriate for production data with exponential decline), by weighted least squares regression (which assigns lower weight to high-variance observations), or by using robust standard errors (which correct the coefficient standard errors for heteroscedasticity without changing the coefficient estimates themselves).
• Production performance regression in unconventional wells attempts to disentangle the effects of geological parameters (net pay, TOC, brittleness, natural fracture density) from completion parameters (stage count, proppant volume, fluid volume, cluster spacing) on initial production rate and EUR (estimated ultimate recovery), but this separation is complicated by the fact that completion designs tend to be optimized for the specific geological conditions (operators use different completion designs in different parts of a field specifically because the geology differs), creating the same endogeneity problem that plagues observational regression studies in economics and medical research; the ideal approach is a controlled experiment (randomized completion design variations in geologically similar wells), but this is rarely practical in commercial oil and gas operations; the practical workaround is careful selection of the comparison dataset (restricting the analysis to wells in a geologically homogeneous fairway where geological variability is minimized) and including as many geological covariates as are available (from logs, cores, seismic attributes) to control for the geological variation that might otherwise confound the completion parameter effects.
• Neural network and machine learning methods have largely displaced traditional multiple regression for complex petroleum prediction problems (well performance from completion and geological parameters, seismic attribute to property prediction) because they can capture nonlinear and interactive effects between predictors that linear regression misses; however, multiple regression retains important advantages including interpretability (the coefficient of each predictor has a direct physical meaning that can be assessed against domain knowledge), statistical inference (p-values and confidence intervals provide a quantified measure of uncertainty about coefficient estimates), and sample efficiency (regression can produce useful estimates from relatively small datasets while neural networks typically require large training datasets to avoid overfitting); the appropriate choice between multiple regression and machine learning methods depends on the size and quality of the available dataset, the complexity of the true functional relationship being modeled, and the relative importance of prediction accuracy versus model interpretability in the specific application.