Multivariate Local Fitting with General Basis Functions

In this paper we combine the concepts of local smoothing and fitting with basis functions for multivariate predictor variables. We start with arbitrary basis functions and show that the asymptotic variance at interior points is independent of the choice of the basis. Moreover we calculate the asymptotic variance at boundary points. We are not able to compute the asymptotic bias since a Taylor theorem for arbitrary basis functions does not exist. For this reason we focus on basis functions without interactions and derive a Taylor theorem which covers this case. This theorem enables us to calculate the asymptotic bias for interior as well as for boundary points. We demonstrate how advantage can be taken of the idea of local fitting with general basis functions by means of a simulated data set, and also provide a data-driven tool to optimize the basis.

The author is grateful to Gerhard Tutz (LMU, Dep. of Statistics) for helpful inspirations during this work, Daniel Rost (LMU, Math. Inst.) for contributions concerning the extendibility of Taylor’s theorem and to the referees for many suggestions which improved this paper. This work was finished while the author passed a term at the University São Paulo. Many thanks especially to Carmen D. S. de André and Júlio M. Singer (USP) for their support in various fields.

Parts of this work were presented and discussed at the Euroworkshop on Statististical Modelling (2001). There a variety of valuable comments and suggestions were given that enhanced the paper.

Authors and Affiliations

1. Institut für Statistik, Ludwig Maximilians Universität, Akadeniiestr. 1, 80799, München, Germany

   Jochen Einbeck

Appendix

A Regularity conditions

• (A1) The kernel K is bounded with compact support, ∫ uu^TK(u)du = μ₂I_d, where μ₂ is a scalar and I_d the d × d identity matrix. In addition, all odd-order moments of K vanish, i.e. \(\int {u_1^{{l_1}}\; \cdots \;u_d^{{l_d}}K\left( u \right)du = 0} \) for all nonnegative integers l₁, …, l_d with an odd sum.

• (A2) The point x is ∈ supp(f). At x, σ² is continuous, f is continuously differentiable and all second-order derivatives of m are continuous. Further f(x) > 0, σ²(x) > 0.

• (A3) The sequence of bandwidth matrices H^1/2 is such that n⁻¹∣H∣^−1/2 and each entry of H tends to zero as n → ∞.

• (A4) For a boundary point x, there exists a value on the boundary of supp(f) with x = x_b + H^1/2c, where c is a fixed element of supp(K), and a convex set \({\cal C}\) with nonnull interior containing x_b, such that \(x \in {\cal C}\).

  (A5) At x, all basis functions are continuously differentiable (for variance expressions in Theorem 1,3,4) resp. twice continuously differentiable (for bias expressions in Theorem 3 and 4). In either case, the point x is non-singular for all basis functions, i.e. ∇φ_j(x) ≠ 0 for j = 1, …, q.

For explanations and interpretations of conditions (Al) to (A4) see Ruppert & Wand (1994).

B Proofs

B.1 Proof of Theorem 1

Let 1 be a matrix of appropriate dimension having only entries equal to 1, further let

Note that for any u ∈ ℝ^d

holds. Let \({C_{x,H}} = \left\{ {t:{H^{ - 1/2}}\left( {t - x} \right) \in {{\cal D}_{x,H}}} \right\}\). For interior and boundary points we derive

and analogously

Substituting (19) and (20) into (6) leads to (8). In the special case of an interior point we have \({M_x} = \left( {\begin{array}{*{20}c} 1 \; 0 \\ 0 \; {{\mu _2}{{\bf{I}}_d}} \\ \end{array} } \right)\) and \({N_x} = \,\left( {\begin{array}{*{20}c} {{\nu _0}} \; 0 \\ 0 \; {\int u{u^T}\;{K^2}\left( u \right)du} \\ \end{array} } \right)\). Thus (8) reduces to

B.2 Proof of Theorem 2

We introduce the function M : [0, 1] → ℝ,

Then we have M(0) = m(x) and M(1) = m(z). We apply the univariate Taylor theorem on the function M ∈ C^p+1([0,1]) and obtain

where

Using the Inverse Function Theorem we obtain

Repeated application of the chain rule on M = m ○ y_Φ leads to

Applying the latter formulas in (23) and substituting ζ = y_Φ(τ) proves the allegation.

B.3 Proof of Theorem 3

The proof is kept shortly since it follows mainly the ideas of the corresponding proof for multivariate local linear fitting, see Ruppert & Wand (1994).

Asymptotic Bias

First note that, applying (10), we have

with

and S_m(x) = o(Q_m(x)). Plugging (24) into (5) shows that

Let W_i = K_H(X_i − x). Using matrix algebra (see e.g. Fahrmeir & Hamerle (1984)) we derive

and

so that substituting (26) and (27) into (25) proves (11).

Asymptotic variance

Similar like above we obtain

where

Plugging this result and (26) into (6) leads to (12).

B.4 Proof of Theorem 4

Let

Asymptotic bias

Note that

where C_x,h was defined in the proof of Theorem 1. Using the first step in (27),

holds. Assuming (A4), M_x is nonsingular and we have

where \(\mu_{x}^{11}=\left(\mu_{x, 11}-\mu_{x, 12} \mu_{x, 22}^{-1} \mu_{x, 21}\right)^{-1}, \mu_{x}^{12}=-\left(\mu_{x, 12} / \mu_{x, 11}\right) \mu_{x}^{22}\)and \(\mu_{x}^{22}=\left(\mu_{x, 22}-\mu_{x, 21} \mu_{x, 12} / \mu_{x, 11}\right)^{-1}\). Then substituting (28) and (29) into (25) and noticing that

yields formula (14).

Asymptotic variance

Similar considerations like in (28) lead to

With (6), (28) and (30) we get

what had to be proven.

Reprints and permissions