Fitting Methods
Numerous algorithms have been developed over decades to tackle the task of finding the best fit to data.
In the conventional approach, the choice of the appropriate method relies on the specific
characteristics of the data and the intended application.
However, there exist universal methods that are applicable to the majority of cases.
But let's first take a look at the most common options.
Linear and Polynomial Fitting
The simplest approaches of linear and polynomial regression are the most popular,
regardless of their drawbacks. Linear fitting works fine only in two cases:
- Data shows evidence of linear behavior → you can use a simple
equation.
- Data is big enough so that the interpolation error is relatively small → you can use piecewise linear fitting.
Linear Function
Piecewise Linear Fit
Some nonlinear relations in data can be fitted with polynomials. Simple polynomial fitting uses equations like
.
Polynomial fitting and piecewise polynomial regression work better in cases when data show evidence of nonlinear behavior,
and the number of data points is not sufficient to neglect the errors obtained by linear models.
However, polynomial fitting often leads to the issue of overfitting, especially for high-order polynomials,
causing the resulting curve to oscillate between data points, resulting in a drastic increase of the interpolation error.
Cubic Polynomial Fit
Overfitting
Spline Functions
Obtaining smooth curves is important when data contains measurement noise or natural variability.
This is a perfect application for approximation with spline functions, and particularly smoothing splines.
Smoothing Splines
A smoothing spline is a flexible non-parametric method for evaluating splines as a linear combination of
basis functions (B-splines). Each B-spline is nonzero over only a subset of the data range, determined by the knot locations.
Together, a linear combination of B-splines can represent complex shapes while maintaining control and smoothness,
avoiding excessive oscillations.
Smoothing Spline Fitting
The solution for a smoothing spline fit is based on minimizing a cost function that combines a goodness-of-fit term and a penalty term.
The penalty term is based on the curvature of the curve, which helps control the level of smoothness (s-factor).
The smoothing spline finds the optimal balance between fitting the data closely and keeping the function simple, preventing overfitting.
The smoothing spline algorithm is available in the SciPy library
'interpolate'.
It is also SplineCloud’s default fitting method. The curve shape can be adjusted via a set of parameters:
- Smoothness factor – controls the number of knots, and eventually the curvature of the spline.
- Degree (order) of the underline B-splines.
- Knot positioning – can be used to tune local curvature in some regions.
Least Squares
This is a different method for finding the optimal spline function to fit a set of data points.
It works by minimizing the sum of the squares of the offsets (residuals) of data points from the curve.
Still, the target of the fitting algorithm is similar and is aimed at finding the balance between smoothness and adhering closely
to the data points.
SplineCloud also supports the Least Squares method, so you can choose what works best for your data.
Least Squares Spline Fitting
Interpolating Splines
Interpolating splines in fact is just an edge case for smoothing splines, for which a smoothing factor equals to 0,
meaning no smoothing is applied and a maximum number of knots is selected.
Use this method if you need to obtain an exact fit for your data.
Spline Interpolation
Other Methods
In addition to polynomials and splines, mathematicians have developed numerous specialized functions for curve fitting.
Methods like Fourier series, Taylor polynomials, Chebyshev polynomials, Rational Thiele interpolation and others
excel at fitting certain data types.
Fourier series fit periodic data by superimposing sine waves.
Taylor polynomials approximate local behavior via point derivatives.
Chebyshev polynomials minimize error through orthogonal bases.
Thiele interpolation uses a rational function that is expressed as a continued fraction.
However, each method shines best only for particular data characteristics and their complexity and narrow focus limit broader applicability.
Splines, on the contrary, stand out for offering a flexible and generalizable approach.
They make no assumptions about function form or derivatives.
The piecewise nature of splines allow both global and local control, which makes them an excellent default choice
before considering more exotic methods.
The simplicity and intuitiveness of spline construction and manipulation make them accessible to experts and non-experts alike.
And by selecting spline methods, consider parametric splines,
as they put the flexibility of the shape and ease of use to the next level.
