B.4. The Euclidean Space

As described in Chapter A, the Euclidean space $\R^d=R\times\cdots\times\R$ is the set of all ordered $d$-tuples or vectors over the real numbers $\R$ (when $d=1$ we refer to the vectors as scalars). We denote a vector in $\R^d$ when $d>1$ in bold and refer to its scalar components via subscripts. For example the vector $\bb x=(x_1,\ldots,x_d)$ has $d$ scalar components $x_i\in\R$, $i=1,\ldots,d$. Together with the Euclidean distance \[ d(\bb x,\bb y) = \sqrt{\sum_{i=1}^d (x_i-y_i)^2},\] the Euclidean space is a metric space $(\R,d)$ (we prove later later in this chapter that the Euclidean distance above is a valid distance function).

The Euclidean norm is the most popular norm. The more general $L_p$ norm \begin{align*} \|\bb x\|_p=\left(\sum_{i=1}^d |x_i|^p\right)^{1/p}, \qquad p\geq 1, \end{align*} has the following special cases \begin{align*} \|\bb x\|_2 &=\|\bb x\| \qquad \text{ (the Euclidean norm)}\\ \|\bb x \|_1 &= \sum_{i=1}^d |x_i|\\ \|\bb x\|_{\infty} &= \max\{|x_1|,\ldots,|x_d|\} \qquad \text{ (achieved by letting } p\to\infty). \end{align*}

The $L_p$ norm can be further generalized as follows.

Weighted norms are convenient for emphasizing some dimensions over others. For example, if $\diag(\bb w)$ is an all zero matrix, except for its diagonal \[ \diag(\bb w) = \begin{pmatrix} w_1 &0 &\cdots& 0\\ 0 & w_2&\cdots&0\\ \vdots & \vdots & \ddots &\vdots\\ 0 & \cdots & 0& w_d \end{pmatrix}, \] the corresponding weighted $L_2$ norm is \[ \|\bb x\|_{2,\diag(\bb w)} = \sqrt{\sum_{i=1}^d w_i^2 x_i^2}.\]

The propositions above confirm that the Euclidean space $(\R^d,d)$, where \[ d(\bb x,\bb y) = \sqrt{\sum_{i=1}^d (x_i-y_i)^2},\] is a metric space.

Figure B.4.1 below shows contour lines of four $L_p$ norms in the left column and four $L_{p,W}$ norms in the right column, where $W=\diag(2,1)$. A non-diagonal matrix $W$ would result in rotated versions of the figures in the right column.

The R code below generates the left and right columns of the figure.

s = seq(-1, 1, length.out = 50)
R = expand.grid(x1 = s, x2 = s)
# generate left column (Lp norms)
D = rbind(R, R, R, R)
D$Norm[1:2500] = abs(D$x1[1:2500]) + abs(D$x2[1:2500])
D$Norm[2501:5000] = ((abs(D$x1[2501:5000]))^1.5 + (abs(D$x2[2501:5000]))^1.5)^(2/3)
D$Norm[5001:7500] = ((abs(D$x1[5001:7500]))^2 + (abs(D$x2[5001:7500]))^2)^0.5
D$Norm[7501:10000] = pmax(abs(D$x1[7501:10000]), abs(D$x2[7501:10000]))
D$p = c(rep("Lp norm, p=1", 2500), rep("Lp norm, p=1.5",
    2500), rep("Lp norm, p=2", 2500), rep("Lp norm, p = inf",
    2500))
ggplot(D, aes(x1, x2, z = Norm)) + facet_grid(p ~ .) +
    stat_contour(bins = 4)

# generate right column (weighted Lp norms)
D = rbind(R, R, R, R)
D$Norm[1:2500] = abs(2 * D$x1[1:2500]) + abs(D$x2[1:2500])
D$Norm[2501:5000] = ((2 * abs(D$x1[2501:5000]))^1.5 +
    (abs(D$x2[2501:5000]))^1.5)^(2/3)
D$Norm[5001:7500] = ((2 * abs(D$x1[5001:7500]))^2 +
    (abs(D$x2[5001:7500]))^2)^(1/2)
D$Norm[7501:10000] = pmax(abs(2 * D$x1[7501:10000]),
    abs(D$x2[7501:10000]))
D$p = c(rep("weighted Lp norm, p=1", 2500), rep("weighted Lp norm, p=1.5",
    2500), rep("weighted Lp norm, p=2", 2500), rep("weighted Lp norm, p=inf",
    2500))
ggplot(D, aes(x1, x2, z = Norm)) + facet_grid(p ~ .) +
    stat_contour(bins = 4)

The convergence problems mentioned in Example~B.4.3 leads to the common practice of defining the metric structure on $\R^{\infty}$ using the distance function $\bar d$ in Example B.4.4 rather than the Euclidean distance. In fact, whenever we refer to the metric structure of $\R^{\infty}$ we will assume the metric structure of $\bar d$ derived above. This metric structure is commonly referred to in the literature as the product topology of $\R^{\infty}$.

The metric structure of the Euclidean space simplifies some of the properties described in the previous chapter.

Note that above we are denoting the function using bold-face $\bb f$ to indicate that $\bb f(\bb x)$ is a vector.

Note that in the first and third cases above, we have $k=1$, and in the last case above, we have $k=d=1$.

The concept of compactness (Definition B.1.6) has some important consequences (Propositions B.3.5 and B.3.6). The general definition of a compact space (Definition B.1.6) is hard to verify, but the following simple condition is veru useful for verifying compactness in $\R^d$. A proof is available in (Rudin, 1976).