Numerical range

In the mathematical field of linear algebra and convex analysis, the numerical range or field of values or Wertvorrat or Wertevorrat of a complex $n \times n$ matrix A is the set $W (A) = {\frac{𝐱^{*} A 𝐱}{𝐱^{*} 𝐱} | 𝐱 \in ℂ^{n}, 𝐱 \neq 0} = {⟨ 𝐱, A 𝐱 ⟩ | 𝐱 \in ℂ^{n}, ‖ 𝐱 ‖_{2} = 1}$ where $𝐱^{*}$ denotes the conjugate transpose of the vector $𝐱$ . The numerical range includes, in particular, the diagonal entries of the matrix (obtained by choosing x equal to the unit vectors along the coordinate axes) and the eigenvalues of the matrix (obtained by choosing x equal to the eigenvectors).

Equivalently, the elements of $W (A)$ are of the form $tr (A P)$ , where $P$ is a Hermitian projection operator from $ℂ^{n}$ to a one-dimensional subspace.

In engineering, numerical ranges are used as a rough estimate of eigenvalues of A. Recently, generalizations of the numerical range are used to study quantum computing.

A related concept is the numerical radius, which is the largest absolute value of the numbers in the numerical range, i.e. $r (A) = sup {| λ | : λ \in W (A)} = sup_{‖ x ‖_{2} = 1} | ⟨ 𝐱, A 𝐱 ⟩ | .$

Properties

Let sum of sets denote a sumset.

General properties

The numerical range is the range of the Rayleigh quotient.
(Hausdorff–Toeplitz theorem) The numerical range is convex and compact.
$W (α A + β I) = α W (A) + {β}$ for all square matrix $A$ and complex numbers $α$ and $β$ . Here $I$ is the identity matrix.
$W (A)$ is a subset of the closed right half-plane if and only if $A + A^{*}$ is positive semidefinite.
The numerical range $W (\cdot)$ is the only function on the set of square matrices that satisfies (2), (3) and (4).
$W (U A U^{*}) = W (A)$ for any unitary $U$ .
$W (A^{*}) = W (A)^{*}$ .
If $A$ is Hermitian, then $W (A)$ is on the real line. If $A$ is anti-Hermitian, then $W (A)$ is on the imaginary line.
$W (A) = {z}$ if and only if $A = z I$ .
(Sub-additive) $W (A + B) \subseteq W (A) + W (B)$ .
$W (A)$ contains all the eigenvalues of $A$ .
The numerical range of a $2 \times 2$ matrix is a filled ellipse.
$W (A)$ is a real line segment $[α, β]$ if and only if $A$ is a Hermitian matrix with its smallest and the largest eigenvalues being $α$ and $β$ .

Normal matrices

If $A$ is normal, and $x \in span (v_{1}, \dots, v_{k})$ , where $v_{1}, \dots, v_{k}$ are eigenvectors of $A$ corresponding to $λ_{1}, \dots, λ_{k}$ , respectively, then $⟨ x, A x ⟩ \in hull (λ_{1}, \dots, λ_{k})$ .
If $A$ is a normal matrix, then $W (A)$ is the convex hull of its eigenvalues.
If $α$ is a sharp point on the boundary of $W (A)$ , then $α$ is a normal eigenvalue of $A$ .

Numerical radius

$r (\cdot)$ is a unitarily invariant norm on the space of $n \times n$ matrices.
$r (A) \leq ‖ A ‖_{op} \leq 2 r (A)$ , where $‖ \cdot ‖_{op}$ denotes the operator norm.
$r (A) = ‖ A ‖_{op}$ if (but not only if) $A$ is normal.
$r (A^{n}) \leq r (A)^{n}$ .

Proofs

Most of the claims are obvious. Some are not.

General properties

Proof of (13)

If $A$ is Hermitian, then it is normal, so it is the convex hull of its eigenvalues, which are all real.

Conversely, assume $W (A)$ is on the real line. Decompose $A = B + C$ , where $B$ is a Hermitian matrix, and $C$ an anti-Hermitian matrix. Since $W (C)$ is on the imaginary line, if $C \neq 0$ , then $W (A)$ would stray from the real line. Thus $C = 0$ , and $A$ is Hermitian.

The following proof is due to

Proof of (12)

The elements of $W (A)$ are of the form $tr (A P)$ , where $P$ is projection from $ℂ^{2}$ to a one-dimensional subspace.

The space of all one-dimensional subspaces of $ℂ^{2}$ is $ℙ ℂ^{1}$ , which is a 2-sphere. The image of a 2-sphere under a linear projection is a filled ellipse.

In more detail, such $P$ are of the form $\frac{1}{2} I + \frac{1}{2} [\begin{matrix} \cos 2 θ & e^{i ϕ} \sin 2 θ \\ e^{- i ϕ} \sin 2 θ & - \cos 2 θ \end{matrix}] = \frac{1}{2} [\begin{matrix} 1 + z & x + i y \\ x - i y & 1 - z \end{matrix}]$ where $x, y, z$ , satisfying $x^{2} + y^{2} + z^{2} = 1$ , is a point on the unit 2-sphere.

Therefore, the elements of $W (A)$ , regarded as elements of $ℝ^{2}$ is the composition of two real linear maps $(x, y, z) \mapsto \frac{1}{2} [\begin{matrix} 1 + z & x + i y \\ x - i y & 1 - z \end{matrix}]$ and $M \mapsto tr (A M)$ , which maps the 2-sphere to a filled ellipse.

Proof of (2)

$W (A)$ is the image of a continuous map $x \mapsto ⟨ x, A x ⟩$ from the ${ℙ ℂ}^{n}$ , so it is compact.

Given two complex nonzero vectors $x, y$ , let $P_{x}, P_{y}$ be their corresponding Hermitian projectors from $ℂ^{n}$ to their respective spans. Let $P$ be the Hermitian projector to the span of both. We have that $P^{*} A P$ is an operator on $Span (x, y)$ .

Therefore, the “restricted numerical range” of $P^{*} A P$ , defined by ${Tr (P^{*} A P P_{z}) : z \in Span (x, y), z \neq 0}$ , is a closed ellipse, according to (12). It is also the case that if $z \in Span (x, y)$ is nonzero, then $Tr (P^{*} A P P_{z}) = Tr (A P P_{z} P) = Tr (A P_{z}) \in W (A)$ . Therefore, the restricted numerical range is contained in the full numerical range of $A$ .

Thus, if $W (A)$ contains $Tr (A P_{x}), Tr (A P_{y})$ , then it contains a closed ellipse that also contains $Tr (A P_{x}), Tr (A P_{y})$ , so it contains the line segment between them.

Proof of (5)

Let $W$ satisfy these properties. Let $W_{0}$ be the original numerical range.

Fix some matrix $A$ . We show that the supporting planes of $W (A)$ and $W_{0} (A)$ are identical. This would then imply that $W (A) = W_{0} (A)$ since they are both convex and compact.

By property (4), $W (A)$ is nonempty. Let $z$ be a point on the boundary of $W (A)$ , then we can translate and rotate the complex plane so that the point translates to the origin, and the region $W (A)$ falls entirely within $ℂ^{+}$ . That is, for some $ϕ \in ℝ$ , the set $e^{i ϕ} (W (A) - z)$ lies entirely within $ℂ^{+}$ , while for any $t > 0$ , the set $e^{i ϕ} (W (A) - z) - t I$ does not lie entirely in $ℂ^{+}$ .

The two properties of $W$ then imply that $e^{i ϕ} (A - z) + e^{- i ϕ} (A - z)^{*} ⪰ 0$ and that inequality is sharp, meaning that $e^{i ϕ} (A - z) + e^{- i ϕ} (A - z)^{*}$ has a zero eigenvalue. This is a complete characterization of the supporting planes of $W (A)$ .

The same argument applies to $W_{0} (A)$ , so they have the same supporting planes.

Normal matrices

Proof of (1), (2)

For (2), if $A$ is normal, then it has a full eigenbasis, so it reduces to (1).

Since $A$ is normal, by the spectral theorem, there exists a unitary matrix $U$ such that $A = U D U^{*}$ , where $D$ is a diagonal matrix containing the eigenvalues $λ_{1}, λ_{2}, \dots, λ_{n}$ of $A$ .

Let $x = c_{1} v_{1} + c_{2} v_{2} + \dots + c_{k} v_{k}$ . Using the linearity of the inner product, that $A v_{j} = λ_{j} v_{j}$ , and that ${v_{i}}$ are orthonormal, we have:

$⟨ x, A x ⟩ = \sum_{i, j = 1}^{k} c_{i}^{*} c_{j} ⟨ v_{i}, λ_{j} v_{j} ⟩ = \sum_{i = 1}^{k} {| c_{i} |}^{2} λ_{i} \in hull (λ_{1}, \dots, λ_{k})$

Proof (3)

By affineness of $W$ , we can translate and rotate the complex plane, so that we reduce to the case where $\partial W (A)$ has a sharp point at $0$ , and that the two supporting planes at that point both make an angle $ϕ_{1}, ϕ_{2}$ with the imaginary axis, such that $ϕ_{1} < ϕ_{2}, e^{i ϕ_{1}} \neq e^{i ϕ_{2}}$ since the point is sharp.

Since $0 \in W (A)$ , there exists a unit vector $x_{0}$ such that $x_{0}^{*} A x_{0} = 0$ .

By general property (4), the numerical range lies in the sectors defined by: $Re (e^{i θ} ⟨ x, A x ⟩) \geq 0 for all θ \in [ϕ_{1}, ϕ_{2}] and nonzero x \in ℂ^{n} .$ At $x = x_{0}$ , the directional derivative in any direction $y$ must vanish to maintain non-negativity. Specifically:
${\frac{d}{d t} Re (e^{i θ} ⟨ x_{0} + t y, A (x_{0} + t y) ⟩) |}_{t = 0} = 0 \forall y \in ℂ^{n}, θ \in [ϕ_{1}, ϕ_{2}] .$ Expanding this derivative:
$Re (e^{i θ} (⟨ y, A x_{0} ⟩ + ⟨ x_{0}, A y ⟩)) = 0 \forall y \in ℂ^{n}, θ \in [ϕ_{1}, ϕ_{2}] .$

Since the above holds for all $θ \in [ϕ_{1}, ϕ_{2}]$ , we must have: $⟨ y, A x_{0} ⟩ + ⟨ x_{0}, A y ⟩ = 0 \forall y \in ℂ^{n} .$

For any $y \in ℂ^{n}$ and $α \in ℂ$ , substitute $α y$ into the equation: $α ⟨ y, A x_{0} ⟩ + α^{*} ⟨ x_{0}, A y ⟩ = 0.$ Choose $α = 1$ and $α = i$ , then simplify, we obtain $⟨ y, A x_{0} ⟩ = 0$ for all $y$ , thus $A x_{0} = 0$ .

Numerical radius

Proof of (2)

Let $v = \arg {max}_{‖ x ‖_{2} = 1} | ⟨ x, A x ⟩ |$ . We have $r (A) = | ⟨ v, A v ⟩ |$ .

By Cauchy–Schwarz, $| ⟨ v, A v ⟩ | \leq ‖ v ‖_{2} ‖ A v ‖_{2} = ‖ A v ‖_{2} \leq ‖ A ‖_{o p}$

For the other one, let $A = B + i C$ , where $B, C$ are Hermitian. $‖ A ‖_{o p} \leq ‖ B ‖_{o p} + ‖ C ‖_{o p}$

Since $W (B)$ is on the real line, and $W (i C)$ is on the imaginary line, the extremal points of $W (B), W (i C)$ appear in $W (A)$ , shifted, thus both $‖ B ‖_{o p} = r (B) \leq r (A), ‖ C ‖_{o p} = r (i C) \leq r (A)$ .

Generalisations

Higher-rank numerical range

The numerical range is equivalent to the following definition: $W (A) = {λ \in ℂ : P M P = λ P for some Hermitian projector P of rank 1} .$ This allows a generalization to higher-rank numerical ranges, one for each $k = 1, 2, 3, \dots$ : $W_{k} (A) = {λ \in ℂ : P M P = λ P for some Hermitian projector P of rank k} .$ $W_{k} (A)$ is always closed and convex, but it might be empty. It is guaranteed to be nonempty if $k < n / 3 + 1$ , and there exists some $A$ such that $W_{k} (A)$ is empty if $k \geq n / 3 + 1$ .

Bibliography

Books

Bonsall, F.F.; Duncan, J. (1971), Numerical Ranges of Operators on Normed Spaces and of Elements of Normed Algebras, Cambridge University Press, ISBN 978-0-521-07988-4
Bonsall, F.F.; Duncan, J. (1973), Numerical Ranges II, Cambridge University Press, ISBN 978-0-521-20227-5
Horn, Roger A.; Johnson, Charles R. (1991), Topics in Matrix Analysis, Cambridge University Press, Chapter 1, ISBN 978-0-521-46713-1.
Horn, Roger A.; Johnson, Charles R. (1990), Matrix Analysis, Cambridge University Press, Ch. 5.7, ex. 21, ISBN 0-521-30586-1
Bhatia, Rajendra (1997). Matrix analysis. Graduate texts in mathematics. New York Berlin Heidelberg: Springer. ISBN 978-0-387-94846-1.
Gustafson, Karl E.; Rao, Duggirala K. M. (1997). Numerical Range: The Field of Values of Linear Operators and Matrices. Universitext. New York, NY: Springer. doi:10.1007/978-1-4613-8498-4. ISBN 978-0-387-94835-5. ISSN 0172-5939.

Papers

Toeplitz, Otto (1918). "Das algebraische Analogon zu einem Satze von Fejér" (PDF). Mathematische Zeitschrift (in German). 2 (1–2): 187–197. doi:10.1007/BF01212904. ISSN 0025-5874.
Hausdorff, Felix (1919). "Der Wertvorrat einer Bilinearform". Mathematische Zeitschrift (in German). 3 (1): 314–316. doi:10.1007/BF01292610. ISSN 0025-5874.
Choi, M.D.; Kribs, D.W.; Życzkowski (2006), "Quantum error correcting codes from the compression formalism", Rep. Math. Phys., 58 (1): 77–91, arXiv:quant-ph/0511101, Bibcode:2006RpMP...58...77C, doi:10.1016/S0034-4877(06)80041-8, S2CID 119427312.
Dirr, G.; Helmkel, U.; Kleinsteuber, M.; Schulte-Herbrüggen, Th. (2006), "A new type of C-numerical range arising in quantum computing", Proc. Appl. Math. Mech., 6: 711–712, doi:10.1002/pamm.200610336
Li, C.K. (1996), "A simple proof of the elliptical range theorem", Proc. Am. Math. Soc., 124 (7): 1985, doi:10.1090/S0002-9939-96-03307-2.
Keeler, Dennis S.; Rodman, Leiba; Spitkovsky, Ilya M. (1997), "The numerical range of 3 × 3 matrices", Linear Algebra and Its Applications, 252 (1–3): 115, doi:10.1016/0024-3795(95)00674-5.
Johnson, Charles R. (1976). "Functional characterizations of the field of values and the convex hull of the spectrum" (PDF). Proceedings of the American Mathematical Society. 61 (2). American Mathematical Society (AMS): 201–204. doi:10.1090/s0002-9939-1976-0437555-3. ISSN 0002-9939.

References

↑ ""well-known" inequality for numerical radius of an operator". StackExchange.
↑ "Upper bound for norm of Hilbert space operator". StackExchange.
↑ "Inequalities for numerical radius of complex Hilbert space operator". StackExchange.
↑ Hilary Priestley. "B4b hilbert spaces: extended synopses 9. Spectral theory" (PDF). Archived from the original (PDF) on 2014-12-25. In fact, ‖T‖ = max(−m_T , M_T) = w_T. This fails for non-self-adjoint operators, but w_T ≤ ‖T‖ ≤ 2w_T in the complex case.
↑ Davis, Chandler (June 1971). "The Toeplitz-Hausdorff Theorem Explained". Canadian Mathematical Bulletin. 14 (2): 245–246. doi:10.4153/CMB-1971-042-7. ISSN 0008-4395.
↑ Choi, Man-Duen; Kribs, David W.; Życzkowski, Karol (October 2006). "Higher-rank numerical ranges and compression problems". Linear Algebra and Its Applications. 418 (2–3): 828–839. arXiv:math/0511278. doi:10.1016/j.laa.2006.03.019.
↑ Li, Chi-Kwong; Sze, Nung-Sing (2008). "Canonical Forms, Higher Rank Numerical Ranges, Totally Isotropic Subspaces, and Matrix Equations". Proceedings of the American Mathematical Society. 136 (9): 3013–3023. arXiv:0706.1536. doi:10.1090/S0002-9939-08-09536-1. ISSN 0002-9939. JSTOR 20535511.
↑ Woerdeman, Hugo J. (2008-01-01). "The higher rank numerical range is convex". Linear and Multilinear Algebra. 56 (1–2): 65–67. doi:10.1080/03081080701352211. ISSN 0308-1087.
↑ Li, Chi-Kwong; Poon, Yiu-Tung; Sze, Nung-Sing (2009-06-01). "Condition for the higher rank numerical range to be non-empty". Linear and Multilinear Algebra. 57 (4): 365–368. arXiv:0706.1540. doi:10.1080/03081080701786384. ISSN 0308-1087.

[1] ""well-known" inequality for numerical radius of an operator". StackExchange.

[2] "Upper bound for norm of Hilbert space operator". StackExchange.

[3] "Inequalities for numerical radius of complex Hilbert space operator". StackExchange.

[4] Hilary Priestley. "B4b hilbert spaces: extended synopses 9. Spectral theory" (PDF). Archived from the original (PDF) on 2014-12-25. In fact, ‖T‖ = max(−m_T , M_T) = w_T. This fails for non-self-adjoint operators, but w_T ≤ ‖T‖ ≤ 2w_T in the complex case.

[5] Davis, Chandler (June 1971). "The Toeplitz-Hausdorff Theorem Explained". Canadian Mathematical Bulletin. 14 (2): 245–246. doi:10.4153/CMB-1971-042-7. ISSN 0008-4395.

[6] Choi, Man-Duen; Kribs, David W.; Życzkowski, Karol (October 2006). "Higher-rank numerical ranges and compression problems". Linear Algebra and Its Applications. 418 (2–3): 828–839. arXiv:math/0511278. doi:10.1016/j.laa.2006.03.019.

[7] Li, Chi-Kwong; Sze, Nung-Sing (2008). "Canonical Forms, Higher Rank Numerical Ranges, Totally Isotropic Subspaces, and Matrix Equations". Proceedings of the American Mathematical Society. 136 (9): 3013–3023. arXiv:0706.1536. doi:10.1090/S0002-9939-08-09536-1. ISSN 0002-9939. JSTOR 20535511.

[8] Woerdeman, Hugo J. (2008-01-01). "The higher rank numerical range is convex". Linear and Multilinear Algebra. 56 (1–2): 65–67. doi:10.1080/03081080701352211. ISSN 0308-1087.

[9] Li, Chi-Kwong; Poon, Yiu-Tung; Sze, Nung-Sing (2009-06-01). "Condition for the higher rank numerical range to be non-empty". Linear and Multilinear Algebra. 57 (4): 365–368. arXiv:0706.1540. doi:10.1080/03081080701786384. ISSN 0308-1087.