Inner product

Definition

An inner product on a vector space $V$ over a field is a function $⟨\cdot ,\cdot ⟩:V^2\to F$ that satisfies the followings:

For arbitrary $\mathbf{x},\mathbf{y},\mathbf{z}\in V$ and $c\in \mathbb{R}$,

(1) $⟨a\mathbf{x}+\mathbf{y},\mathbf{z}⟩=a⟨\mathbf{x},\mathbf{z}⟩+⟨\mathbf{y},\mathbf{z}⟩$ 　(Linearity for the first argument)

(2) $⟨\mathbf{x},\mathbf{y}⟩=\overline{⟨\mathbf{y},\mathbf{x}⟩}$ 　(Conjugate symmetry)

(3) $⟨\mathbf{x},\mathbf{x}⟩\ge 0;$ $⟨\mathbf{x},\mathbf{x}⟩=0$ if and only if $\mathbf{x}=0$ 　(Positive semidefiniteness)

Examples

Standard inner product

Definition

A standard inner space in ${\mathbb{R}}^n$ is defined for $\mathbf{x}=(x_1,\cdots ,x_n)$ and $\mathbf{y}=(y_1,\cdots ,y_n)$ as

$$⟨\mathbf{x},\mathbf{y}⟩={\mathbf{x}}^{\top }\mathbf{y}=\sum _{i=1}^n{x}_i{y}_i.$$

Note that for the Euclidean norm,

$$\Vert \mathbf{x}{\Vert }_2=\sqrt{⟨\mathbf{x},\mathbf{x}⟩}.$$

Angle between vectors

The standard inner product on ${\mathbb{R}}^n$ is related to the angle between two vectors $\mathbf{x},\mathbf{y}$. Let $\theta$ be the angle between the two vectors and let $\mathbf{z}=\mathbf{x}-\mathbf{y}$. Then, applying the second law of cosine gives

$$\Vert \mathbf{z}{\Vert }_2^2=\Vert \mathbf{x}{\Vert }_2^2+\Vert \mathbf{y}{\Vert }_2^2-2\Vert \mathbf{x}{\Vert }_2\Vert \mathbf{y}{\Vert }_2\cos \theta .$$

Now substitute $\mathbf{z}=\mathbf{x}-\mathbf{y}$.

\lVert \mathbf{x}-\mathbf{y} \rVert_2^2 &= (\mathbf{x}-\mathbf{y})^\top(\mathbf{x}-\mathbf{y}) \\ &= \mathbf{x}^\top \mathbf{x} + \mathbf{y}^\top\mathbf{y} -2\mathbf{x}^\top\mathbf{y} \\ &= \lVert \mathbf{x} \rVert_2^2 + \lVert \mathbf{y} \rVert_2^2 -2\mathbf{x}^\top\mathbf{y} \\ &= \lVert \mathbf{x} \rVert_2^2 + \lVert \mathbf{y} \rVert_2^2 -2\lVert \mathbf{x} \rVert_2 \lVert \mathbf{y} \rVert_2\cos\theta \end{align*}$$ Hence,

\cos\theta = \dfrac{\mathbf{x}^\top\mathbf{y}}{\lVert \mathbf{x} \rVert_2 \lVert \mathbf{y} \rVert_2}.