Exercise 12.1.11

Proof. As $H$ is a subgroup of $A_n$, by Exercise 9, there exists $\phi \in A_n$ such that $H=H(\phi )$. Let ${\mathcal{}O}_{\phi }$ be the orbit of $\phi$ under the action of $A_n$:

and let $G$ the subgroup of $A_n$ defined by

Then $G\subset H({\phi }_1)=H$. We show that $G$ is normal in $A_n$.

Let $\tau \in A_n$ and $\sigma \in G$. Fix $i$ between 1 and $s$. Then $\tau \cdot {\phi }_i\in {\mathcal{}O}_{\phi }$, so $\tau \cdot {\phi }_i={\phi }_j$ for some $j\in \text{[[}1,s\text{]]}$. Then

so ${\tau }^{-1}\mathit{\sigma \tau }\in G$. Since $A_n$ is a simple group for $n\ge 5$, $G=\{e\}$ or $G=A_n$. Since $G\subset H$ and $H\subset A_n,H\ne A_n$, then $G\ne A_n$, therefore $G=\{e\}$.

$H=H(\phi )={\mathrm{Stab}}_{A_n}(\phi )$, therefore $s=|{\mathcal{}O}_{\phi }|=(A_n:H)$.

If we suppose that $(A_n:H)<n$, then $s<n$. Then $s\le n-1$, therefore $s!\le (n-1)!<n!∕2$. Since there are $n!∕2$ permutations in $A_n$, and only $s$ permutations of $\{{\phi }_1,{\phi }_2,\dots ,{\phi }_s\}$ there exist two distinct permutations ${\tau }_1,{\tau }_2\in A_n$ such that

So $e\ne {\tau }_2^{-1}{\tau }_1\in G$, $G\ne \{e\}$: this is a contradiction. This proves $(A_n:H)\ge n$. □