Polynomial Models

Polynomial Models, represented by functions of the form \[y = a_n x^n + a_{n-1} x^{n-1} + \ldots + a_1 x + a_0,\] are foundational in modelling curvilinear relationships. In this formulation, each \(a_i\) (where \(i = 0, 1, \ldots, n\)) denotes the coefficient corresponding to the \(i\)-th term of the polynomial, and \(x\) is the independent or exploratory variable. The degree \(n\) of the polynomial determines the model’s complexity, with higher degrees allowing for more intricate curve shapes.

These models are particularly useful in capturing the non-linear dynamics often observed in real-world data. For instance, a quadratic model (where \(n = 2\)) can describe simple parabolic trends, while higher-degree models, such as cubic (\(n = 3\)) or quartic (\(n = 4\)), enable the representation of more complex and varied behaviours (Figure 1).

However, increasing the degree \(n\) also increases the risk of overfitting, a phenomenon where the model adapts too closely to the specificities of the training data, including noise, at the expense of generalizability to new data (Figure 2). Overfitting leads to models that perform poorly in predictive scenarios, failing to capture the true underlying trend.

Polynomial models are extensively applied across various disciplines. In physics, they are instrumental in modelling motion under uniform acceleration, among other phenomena. In economics, polynomial trends are fitted to time series data to understand market dynamics. In biological sciences, these models aid in interpreting growth rates and simple gene expression patterns. The interpretation of the coefficients \(a_i\) can provide significant insights; for instance, in the quadratic model \(y = ax^2 + bx + c\), the sign of \(a\) determines the direction in which the parabola opens, offering crucial information about the nature of the relationship being modelled (Figure 3).

In practical applications, the selection of the polynomial degree \(n\) is critical. It is a balance between capturing the complexity of the data and avoiding overfitting (Figure 2). Techniques such as cross-validation, where the data is divided into training and testing sets, can be used to determine the optimal degree of the polynomial. Additionally, statistical measures like the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC) are often employed to select the most appropriate model by balancing model fit and complexity.

In summary, Polynomial Models are a versatile and powerful tool in statistical modelling. Their ability to approximate complex functions with a relatively straightforward mathematical formulation makes them a fundamental component in various fields of data analysis.

While traditional polynomial models are highly effective, they sometimes lack the flexibility required for certain types of data, particularly those exhibiting asymmetric trends. This limitation led to the development of fractional polynomial models, which extend the concept of polynomial models by allowing for fractional exponents. This advancement provides a greater ability to fit a wider range of curves and is especially useful in cases where the relationship between variables is not adequately captured by integer exponents.

Fractional Polynomial Models

Fractional Polynomials represent an advanced evolution of traditional polynomial models, marked by their use of non-integer, real-number exponents in the independent variable. Mathematically, they are expressed as \[y = \beta_0 + \beta_1 x^{p_1} + \beta_2 x^{p_2} + \ldots + \beta_n x^{p_n},\] where \(x\) is the independent variable, \(\beta_i\) are coefficients, and \(p_i\) are the variable powers. These powers, unlike the integer-only powers in traditional polynomials, can include any real number like \(0.5\), \(-1\), or \(2.3\). This flexibility significantly broadens the modeling capability of polynomials, allowing for more precise fitting to complex and asymmetric data patterns. Terms such as \(x^{-1}\) and \(x^{0.5}\), representing the reciprocal and square root of \(x\) respectively, enable the modeling of relationships that exhibit dramatic changes over different ranges of \(x\), a task challenging for standard polynomial models with integer exponents.

The utility of fractional polynomials extends to various fields, notably in medical statistics and biological data analysis, where data patterns often defy symmetry. They are particularly adept in modeling phenomena like dose-response curves in pharmacokinetics and progression rates of diseases, where the response changes in a non-linear fashion. Such flexibility makes them invaluable in scenarios where data exhibit complex, non-standard behaviors.

However, the complexity of fractional polynomials can pose interpretational challenges, and like their traditional counterparts, they are susceptible to overfitting. This risk is heightened with the inclusion of multiple fractional terms or higher degrees, necessitating careful model selection and validation processes. Methods such as cross-validation or the use of information criteria like AIC or BIC are often employed to balance model fit against the risk of overfitting.

Spline Models

Splines are a class of mathematical functions used in statistical modeling, defined as piecewise polynomials that are joined at specific points called knots. Mathematically, if we denote a spline function by \(S(x)\), it can be represented as: \[S(x) = \begin{cases} P_1(x) & \text{for } x \leq k_1, \\ P_2(x) & \text{for } k_1 < x \leq k_2, \\ \vdots \\ P_n(x) & \text{for } x > k_{n-1}, \end{cases}\] where \(P_i(x)\) are polynomial functions of degree \(d\), typically represented as \[P_i(x) = a_{i0} + a_{i1}x + a_{i2}x^2 + \ldots + a_{id}x^d\] for each piece of the spline, with \(a_{i0}, a_{i1}, \ldots, a_{id}\) being the coefficients that vary from one segment of the spline to another. The knots \(k_1, k_2, \ldots, k_{n-1}\) are specific values in the domain of \(x\) where these polynomial pieces meet. Splines ensure continuity and smoothness at these knots by enforcing that both the function and its derivatives up to degree \(d-1\) are continuous across the knots. This mathematical formulation allows splines to flexibly model a wide range of functions by varying the number and position of the knots as well as the degree of the polynomial pieces.

Splines, with their diverse forms, can have a variety of applications, each type distinguished by its unique features and mathematical formulation. Linear splines represent the most basic form, where each segment \(P_i(x)\) is a linear function, typically \(P_i(x) = a_{i0} + a_{i1}x\). These are straightforward to compute and are used in applications requiring simple, piecewise linear approximations.

Cubic splines elevate the complexity and smoothness, with each \(P_i(x)\) being a third-degree polynomial: \[P_i(x) = a_{i0} + a_{i1}x + a_{i2}x^2 + a_{i3}x^3\]. Their widespread adoption is attributed to their ability to model smooth, continuous curves, making them highly suitable for applications in curve fitting, computer-aided design, and modelling non-linear relationships in data.

B-splines, or basis splines, while they can also be cubic, offer a more general and flexible approach. They are defined not by the splines themselves, but by a set of basis functions, which are piecewise polynomials of a specified degree. The key advantage of B-splines is their local control; adjustments in one part of the spline affect only a limited region around that part. This is due to their basis functions having minimal support, which means each function is non-zero only over a small interval.

For B-splines, the mathematical representation shifts from individual polynomial pieces to a sum of basis functions, each weighted by a coefficient. These basis functions are defined over the knots and have local support. The B-spline representation can be expressed as:

\[S(x) = \sum_{i=0}^{n+d} B_{i,d}(x) \cdot c_i\]

Here, \(B_{i,d}(x)\) are the B-spline basis functions of degree \(d\), and \(c_i\) are the coefficients. The basis functions \(B_{i,d}(x)\) are defined recursively, starting with degree 0 (piecewise constant functions) and building up to the desired degree. The recursive definition for a B-spline basis function of degree \(d\) is given by:

\[ B_{i,0}(x) = \begin{cases} 1 & \text{if } k_i \leq x < k_{i+1}, \\ 0 & \text{otherwise}, \end{cases} \]

\[ B_{i,d}(x) = \frac{x - k_i}{k_{i+d} - k_i} B_{i,d-1}(x) + \frac{k_{i+d+1} - x}{k_{i+d+1} - k_{i+1}} B_{i+1,d-1}(x) \]

The knots \(k_0, k_1, \ldots, k_{n+d}\) in the case of B-splines extend beyond the range of the data to ensure that the spline is well-defined at the boundaries. This expansion of the knot sequence provides a framework for the B-spline basis functions to cover the entire domain of the data. Unlike traditional splines, where each polynomial piece is defined explicitly, B-splines construct the spline function as a linear combination of these basis functions, providing a high degree of flexibility and local control over the shape of the spline. This formulation allows for efficient computation and adjustments in a localized manner, making B-splines a powerful tool for modelling complex data in various applications.

Citation

1. For attribution, please cite this work as:

2. BibTeX citation

@misc{oliveira2024polynomials,
  author = {Oliveira, Thiago},
  title = {Exploring Polynomial, Fractional Polynomial, and Spline Models},
  year = {2024}
}

Did you find this page helpful? Consider sharing it 🙌