[α₁⁰ α₁¹ α₁² ⋯ α₁ⁿ⁻¹]
[α₂⁰ α₂¹ α₂² ⋯ α₂ⁿ⁻¹]
[α₃⁰ α₃¹ α₃² ⋯ α₃ⁿ⁻¹]
[ ⋮ ⋮ ⋮ ⋱ ⋮ ]
[αm⁰ αm¹ αm² ⋯ αmⁿ⁻¹]
Ex:
M = [1, 2, 3], n = 4
[1⁰ 1¹ 1² 1³] [1 1 1 1 ]
V = [2⁰ 2¹ 2² 2³] = [1 2 4 8 ]
[3⁰ 3¹ 3² 3³] [1 3 9 27]
https://en.wikipedia.org/wiki/Vandermonde_matrix
/**
* Weighted linear least squares fitting using Matrix algebra (Polynomial).
*
* Generalizing from a straight line (first degree polynomial) to a kᵗʰ degree polynomial:
* y = a₀ + a₁x + ⋯ + akxᵏ
*
* Leads to equations in matrix form:
* [n Σxᵢ ⋯ Σxᵢᵏ ] [a₀] [Σyᵢ ]
* [Σxᵢ Σxᵢ² ⋯ Σxᵢᵏ⁺¹] [a₁] [Σxᵢyᵢ ]
* [ ⋮ ⋮ ⋱ ⋮ ] [ ⋮ ] = [ ⋮ ]
* [Σxᵢᵏ Σxᵢᵏ⁺¹ ⋯ Σxᵢ²ᵏ ] [ak] [Σxᵢᵏyᵢ]
*
* This is a Vandermonde matrix:
* [1 x₁ ⋯ x₁ᵏ] [a₀] [y₁]
* [1 x₂ ⋯ x₂ᵏ] [a₁] [y₂]
* [⋮ ⋮ ⋱ ⋮ ] [ ⋮ ] = [ ⋮]
* [1 xn ⋯ xnᵏ] [ak] [yn]
*
* Can write as equation:
* y = Xa
*
* Solve by premultiplying by transpose Xᵀ:
* XᵀWy = XᵀWXa
*
* Invert to yield vector solution:
* a = (XᵀWX)⁻¹XᵀWy
*
* (http://mathworld.wolfram.com/LeastSquaresFittingPolynomial.html)
*
* For reference, the traditional way to do least squares:
* _ _ __
* x y - xy _ _
* m = _________ b = y - mx
* _ __
* (x)² - x²
*
* @param array $ys y values
* @param array $xs x values
* @param array $ws weight values
*
* @return Matrix [[m], [b]]
*/
public function leastSquares(array $ys, array $xs, array $ws, int $order = 1) : Matrix
{
// y = Xa
$X = new VandermondeMatrix($xs, $order + 1);
$y = new ColumnVector($ys);
$W = new DiagonalMatrix($ws);
// a = (XᵀWX)⁻¹XᵀWy
$Xᵀ = $X->transpose();
$beta_hat = $Xᵀ->multiply($W)->multiply($X)->inverse()->multiply($Xᵀ)->multiply($W)->multiply($y);
return $beta_hat;
}
/**
* Evaluate for x
* Use the smoothness parameter α to determine the subset of data to consider for
* local regression. Perform a weighted least squares regression and evaluate x.
*
* @param number $x
*
* @return number
*/
public function evaluate($x)
{
$α = $this->α;
$λ = $this->λ;
$n = $this->n;
// The number of points considered in the local regression
$Δx = Single::abs(Single::subtract($this->xs, $x));
$αᵗʰΔx = Average::kthSmallest($Δx, $this->number_of_points - 1);
$arg = Single::min(Single::divide($Δx, $αᵗʰΔx * max($α, 1)), 1);
// Kernel function: tricube = (1-arg³)³
$tricube = Single::cube(Single::multiply(Single::subtract(Single::cube($arg), 1), -1));
$weights = $tricube;
// Local Regression Parameters
$parameters = $this->leastSquares($this->ys, $this->xs, $weights, $λ);
$X = new VandermondeMatrix([$x], $λ + 1);
return $X->multiply($parameters)[0][0];
}
