Exercise 14.2.15

Proof.

Let $i$ be a fixed integer in $\{1,\dots ,n\}$. Since $G$ is transitive, $G\cdot i=\{1,\dots ,n\}$. The Fundamental Theorem of group actions gives $n=|G\cdot i|=(G:G_i),$ thus $|G_i|n=|G|$.

Given a subgroup $G_i\subset H\subset G$, let $\{{\tau }_1=e,\dots ,{\tau }_m\}$ be a complete system of representatives of the left cosets $\mathit{\tau H},\tau \in G$, where $m=(G:H)$, so that ${\tau }_{1}H,\dots ,{\tau }_{m}H$ partition $G$.

Consider

As $G={\tau }_{1}H\cup \cdots \cup {\tau }_{m}H$, then

Now we show that this union is disjoint.

If $u\in R_j\cap R_k=({\tau }_{j}H)\cdot i\cap ({\tau }_{k}H)\cdot i$, then

Then ${h{}^{\prime }}^{-1}{\tau }_k^{-1}{\tau }_{j}h)(i)=i$, thus

hence ${\tau }_{j}h={\tau }_k{h}^{\prime }{h}^{″},h,h^{\prime },h^{″}\in H$. This shows that ${\tau }_{j}H={\tau }_{k}H$, thus $j=k$. This proves

Now we prove that every $\sigma \in G$ preserves the block structure:

If $\sigma \in G$ and $R_j=({\tau }_{j}H)\cdot i$, then $\sigma {\tau }_{j}H$ is a left coset, thus $\sigma {\tau }_{j}H={\tau }_{k}H$ for some index $k$, and

Since $G$ is transitive, all $R_j$ have same cardinality $l$, and $n=\mathit{lm}$.

To conclude, $R_1,\dots ,R_m$ have same cardinality, partition $\{1,\dots ,n\}$, and every $\sigma \in G$ preserves the block structure. If $l>1$ and $m>1$, then $G$ is imprimitive.

Hence, if we assume that $G$ is primitive, either $l=1$ or $m=1$.

$\text{∙}$ If $l=1$, then for all indices $k$, $({\tau }_{k}H)\cdot i=\{{\tau }_k(i)\}$. With $k=1$ and ${\tau }_k=e$, we obtain $H\cdot i=\{i\}$, which shows that $H\subset G_i$, thus $H=G_i$.

$\text{∙}$ If $m=1$, then $(G:H)=m=1$, thus $H=G$.

This proves that there is no subgroup $H$ such that $G_i⊊H⊊G$ : $G_i$ is maximal with respect to inclusion.

Conversely, suppose that $G$ is imprimitive, with respect to the blocks $R_1,\dots ,R_m$, where $m>1$. Since $G$ is transitive, all $R_j$ have the same cardinality $|R_j|=l>1$.

If $i\in \{1,\dots ,n\}$ is some fixed integer, there is some index $j,1\le j\le m$ such that $i\in R_j$. Now, consider the subgroup

Then $G_i\subset H$: if $\sigma \in G_i$, then $\sigma (i)=i\in R_j$, thus $\sigma (R_j)=R_j$. Moreover

$\text{∙}$ $H\ne G$: Since $m>1$, there is some $R_k\ne R_j$, and some $w\in R_k$. Since $G$ is transitive, there is some $\sigma \in G$ such that $\sigma (i)=w$, so that $\sigma (R_j)=R_k$. Then $\sigma \notin H$, and $H\ne G$.

$\text{∙}$ $G_i\ne H$: Since $|R_j|>1$, there is some $i^{\prime }\ne i$ in the same block $R_j$. Since $G$ is transitive, there is some ${\sigma }^{\prime }\in G$ such that ${\sigma }^{\prime }(i)=i^{\prime }$, so that $\sigma (R_j)=R_j$. Then $\sigma \in H$, but $\sigma \notin G_i$, so $G_i\ne H$.

To conclude, the subgroup $H$ satisfies

This proves that $G_i$ is not maximal with respect to inclusion.

We have proved that $G$ is primitive if and only if the isotropy subgroups of $G$ are maximal with respect to inclusion. □