Equivalently, using the notation for vincular patterns, a Baxter permutation is one that avoids the two dashed patterns and .
For example, the permutation in (written in one-line notation) is not a Baxter permutation because, taking
, and , this permutation violates the first condition.
In general, has the following formula:
In fact, this formula is graded by the number of descents in the permutations, i.e., there are Baxter permutations in with descents.
Other properties
The number of alternating Baxter permutations of length is , the square of a Catalan number, and of length is
.
The number of doubly alternating Baxter permutations of length and (i.e., those for which both and its inverse are alternating) is the Catalan number .
Baxter introduced Baxter permutations while studying the fixed points of commuting continuous functions. In particular, if and are continuous functions from the interval to itself such that for all , and for finitely many
in , then:
the number of these fixed points is odd;
if the fixed points are then and act as mutually-inverse permutations on
and ;
the permutation induced by on uniquely determines the permutation induced by
on ;
under the natural relabeling , , etc., the permutation induced on is a Baxter permutation.
↑Ackerman, Eyal; Barequet, Gill; Pinter, Ron Y. (2006), "A bijection between permutations and floorplans, and its applications", Discrete Applied Mathematics, 154 (12): 1674–1684, doi:10.1016/j.dam.2006.03.018, MR2233287.