Parikh's theorem

In theoretical computer science, Parikh's theorem states that if one looks only at the number of occurrences of each terminal symbol in a context-free language, without regard to their order, then the language is indistinguishable from a regular language. It is useful for deciding that strings with a given number of terminals are not accepted by a context-free grammar. It was first proved by Rohit Parikh in 1961 and republished in 1966.

Definitions and formal statement

Let $Σ = {a_{1}, a_{2}, \dots, a_{k}}$ be an alphabet. The Parikh vector of a word $w$ is defined as its image under the function $p : Σ^{*} \to ℕ^{k}$ , given by $p (w) = (| w |_{a_{1}}, | w |_{a_{2}}, \dots, | w |_{a_{k}})$ where $| w |_{a_{i}}$ denotes the number of occurrences of the symbol $a_{i}$ in the word $w$ .

A subset of $ℕ^{k}$ is said to be linear if it is of the form $u_{0} + ℕ u_{1} + \dots + ℕ u_{m} = {u_{0} + t_{1} u_{1} + \dots + t_{m} u_{m} ∣ t_{1}, \dots, t_{m} \in ℕ}$ for some vectors $u_{0}, \dots, u_{m}$ . A subset of $ℕ^{k}$ is said to be semi-linear if it is a union of finitely many linear subsets.

Theorem—Let $L$ be a context-free language or a regular language, let $P (L)$ be the set of Parikh vectors of words in $L$ , that is, $P (L) = {p (w) ∣ w \in L}$ . Then $P (L)$ is a semi-linear set.

If $S$ is any semi-linear set, then there exists a regular language (which a fortiori is context-free) whose Parikh vectors is $S$ .

In short, the image under $p$ of context-free languages and of regular languages is the same, and it is equal to the set of semilinear sets.

Two languages are said to be commutatively equivalent if they have the same set of Parikh vectors. Thus, every context-free language is commutatively equivalent to some regular language.

Proof

The second part is easy to prove.

Proof

Given semi-linear set $S$ , to construct a regular language whose set of Parikh vectors is $S$ .

$S$ is a union of 0 or more linear sets. Since the empty language is regular, and union of regular languages is regular, it suffices to prove that any linear set is the set of Parikh vectors of a regular language.

Let $S = {u_{0} + t_{1} u_{1} + \dots + t_{m} u_{m} ∣ t_{1}, \dots, t_{m} \in ℕ}$ , then it is the set of Parikh vectors of ${z_{0}} \cdot (\cup_{i = 1}^{m} {z_{i}})^{*}$ , where each $z_{i}$ has Parikh vector $u_{i}$ .

The first part is less easy. The following proof is credited to Goldstine.

First we need a small strengthening of the pumping lemma for context-free languages:

Lemma—If $L$ is generated by a Chomsky normal form grammar, then $\exists N \geq 1$ , such that the following is true.

For any $k \geq 1$ , and for any $w \in L$ with $| w | \geq N^{k}$ , given a derivation $S \Rightarrow^{*} u A v$ of $w$ from the axiom $S$ , we can split $w$ into segments $w = u x_{1} \dots x_{k} z y_{k} \dots y_{1} v$ , and pick a nonterminal symbol $A$ , such that we have:

$| x_{i} y_{i} | \geq 1$ for all $i$ , and $| x_{1} \dots x_{k} z y_{k} \dots y_{1} | \leq N^{k}$

and the derivation $S \Rightarrow^{*} u A v$ can be split in the following (with the same parse tree, in particular using precisely the same nonterminals):

$S \Rightarrow^{*} u A v \forall i, A \Rightarrow^{*} x_{i} A y_{i} A \Rightarrow^{*} z$

The proof is essentially the same as the standard pumping lemma: use the pigeonhole principle to find $k$ copies of some nonterminal symbol $A$ in the longest path in the shortest derivation tree.

Now we prove the first part of Parikh's theorem, making use of the above lemma.

Proof

First, construct a Chomsky normal form grammar for $L$ .

For each finite nonempty subset of nonterminals $U$ , define $L_{U}$ to be the set of sentences in $L$ such that there exists a derivation that uses every nonterminal in $U$ , no more and no less. It is clear that $L = \cup_{U} L_{U}$ , so it suffices to prove that each $p (L_{U})$ is a semilinear set.

Now fix some $U$ , and let $k = | U |$ . We construct two finite sets $F, G$ , such that $p (L_{U}) = p (F \cdot G^{*})$ , which is obviously semilinear.

For notational clarity, write

\Rightarrow_{U}^{*}

to mean "there exists a derivation using no more (but possibly less) than nonterminals in

U

. With that, we define

F, G

as follows:

$F = {w \in L_{U} : | w | < N^{k}}$ $G = {x y : 1 \leq | x y | \leq N^{k} and there exists A \in U such that A \Rightarrow_{U}^{*} x A y}$

To prove $p (L_{U}) \subset p (F \cdot G^{*})$ , we induct on the length of $w \in L_{U}$ .

If

| w | < N^{k}

, then

w \in F

, so

p (w) \in p (F \cdot G^{*})

. Otherwise, since

w \in L_{U}

, there is a derivation of

w

using precisely the elements of

U

. By the strengthened pumping lemma, this derivation can be rewritten to be of the form

$S \underset{d_{0}}{\overset{*}{\Rightarrow}} u A v \underset{d_{1}}{\overset{*}{\Rightarrow}} u x_{1} A y_{1} v \underset{d_{2}}{\overset{*}{\Rightarrow}} \dots \underset{d_{k}}{\overset{*}{\Rightarrow}} u x_{1} \dots x_{k} A y_{k} \dots y_{1} v \underset{d_{k + 1}}{\overset{*}{\Rightarrow}} u x_{1} \dots x_{k} z y_{k} \dots y_{1} v$

where

A \in U

,

1 \leq | x_{i} y_{i} |

, and

| x_{1} \dots x_{k} z y_{k} \dots y_{1} | \leq N^{k}

.

Since there are only

k - 1

elements in

U ∖ {A}

, but there are

k

sub-derivations

d_{1}, ..., d_{k}

in the middle, picking one witnessing subderivation to use each of the nonterminals in

U ∖ {A}

, we may delete one sub-derivation

d_{i}

and obtain a shorter

w^{'}

that still uses all the nonterminals of

U

, i.e., which is still in

L_{U}

, with

$p (w) = p (u z v) + p (x_{1} y_{1}) + \dots + p (x_{k} y_{k}) = p (w^{'}) + p (x_{i} y_{i})$

By induction,

p (w^{'}) \in p (F \cdot G^{*})

, and by construction,

x_{i} y_{i} \in G

, so

p (w) \in p (F \cdot G^{*})

.

To prove $p (L_{U}) \supset p (F \cdot G^{*})$ , consider an element $w \in F \cdot G^{*}$ . We need to show that $p (w) \in p (L_{U})$ . We induct on the minimal number of factors from $G$ that is needed to identify $w$ as an element of $F \cdot G^{*}$ .

If no factor from

G

is needed, then

w \in F \subset L_{U}

.

Otherwise,

w = w^{'} x y

for some

w^{'} \in F \cdot G^{*}

and

x y \in G

, where

w^{'}

requires less factors from

G

than

w

.

By induction,

p (w^{'}) = p (w^{″})

for some

w^{″} \in L_{U}

. By construction of

G

, there exists some

A \in U

such that

A \Rightarrow_{U}^{*} x A y

.

By construction of

L_{U}

, the symbol

A

appears in a derivation of

w^{″}

using exactly all of

U

. Then we can interpolate

A \Rightarrow_{U}^{*} x A y

into that derivation to obtain some

w^{‴} \in L_{U}

such that

$p (w^{‴}) = p (w^{″}) + p (x y) = p (w^{'}) + p (x y) = p (w)$

Strengthening for bounded languages

A language $L$ is bounded if $L \subset w_{1}^{*} \dots w_{k}^{*}$ for some fixed words $w_{1}, \dots, w_{k}$ . Ginsburg and Spanier gave a necessary and sufficient condition, similar to Parikh's theorem, for bounded languages.

Call a linear set stratified, if in its definition for each $i \geq 1$ the vector $u_{i}$ has the property that it has at most two non-zero coordinates, and for each $i, j \geq 1$ if each of the vectors $u_{i}, u_{j}$ has two non-zero coordinates, $i_{1} < i_{2}$ and $j_{1} < j_{2}$ , respectively, then their order is not $i_{1} < j_{1} < i_{2} < j_{2}$ . A semi-linear set is stratified if it is a union of finitely many stratified linear subsets.

Ginsburg-Spanier—A bounded language $L$ is context-free if and only if ${(n_{1}, \dots, n_{k}) ∣ w_{1}^{n_{1}} \dots w_{k}^{n_{k}} \in L}$ is a stratified semi-linear set.

Significance

The theorem has multiple interpretations. It shows that a context-free language over a singleton alphabet must be a regular language and that some context-free languages can only have ambiguous grammars^{[further explanation needed]}. Such languages are called inherently ambiguous languages. From a formal grammar perspective, this means that some ambiguous context-free grammars cannot be converted to equivalent unambiguous context-free grammars.

References

1 2 Kozen, Dexter (1997). Automata and Computability. New York: Springer-Verlag. ISBN 3-540-78105-6.
↑ Håkan Lindqvist. "Parikh's theorem" (PDF). Umeå Universitet.
↑ Parikh, Rohit (1961). "Language Generating Devices". Quarterly Progress Report, Research Laboratory of Electronics, MIT.
↑ Parikh, Rohit (1966). "On Context-Free Languages". Journal of the Association for Computing Machinery. 13 (4): 570–581. doi:10.1145/321356.321364. S2CID 12263468.
↑ Goldstine, J. (1977-01-01). "A simplified proof of parikh's theorem". Discrete Mathematics. 19 (3): 235–239. doi:10.1016/0012-365X(77)90103-0. ISSN 0012-365X.
↑ Ginsburg, Seymour; Spanier, Edwin H. (1966). "Presburger formulas, and languages". Pacific Journal of Mathematics. 16 (2): 285–296. doi:10.2140/pjm.1966.16.285.

[kozen-1] 1 2 Kozen, Dexter (1997). Automata and Computability. New York: Springer-Verlag. ISBN 3-540-78105-6.

[2] Håkan Lindqvist. "Parikh's theorem" (PDF). Umeå Universitet.

[3] Parikh, Rohit (1961). "Language Generating Devices". Quarterly Progress Report, Research Laboratory of Electronics, MIT.

[4] Parikh, Rohit (1966). "On Context-Free Languages". Journal of the Association for Computing Machinery. 13 (4): 570–581. doi:10.1145/321356.321364. S2CID 12263468.

[5] Goldstine, J. (1977-01-01). "A simplified proof of parikh's theorem". Discrete Mathematics. 19 (3): 235–239. doi:10.1016/0012-365X(77)90103-0. ISSN 0012-365X.

[6] Ginsburg, Seymour; Spanier, Edwin H. (1966). "Presburger formulas, and languages". Pacific Journal of Mathematics. 16 (2): 285–296. doi:10.2140/pjm.1966.16.285.