Exercise 4.6.7 ($A$ is the unique (nontrivial) minimal normal subgroup of $S_\Omega$)

Proof. Let $H$ be any nontrivial normal subgroup of $S_{\mathrm{\Omega }}$.

We prove first that $H\cap D\ne \{1\}$ (this is the less easy part).

$\bullet$ Suppose first that there is some $\lambda \in H\cap D$ such that ${\lambda }^2\ne 1$. Then there exists $a\in \mathrm{\Omega }$ such that $c={\lambda }^2(a)\ne a$. Put $b=\lambda (a)$, so that $c=\lambda (b)$. Then $a,b,c$ are distinct elements of $\mathrm{\Omega }$.

Consider the permutation $\xi ={\lambda }^{-1}(bc)\lambda {(bc)}^{-1}$. Since $H⊴S_{\mathrm{\Omega }}$ and $\lambda \in H$, we obtain $\xi ={\lambda }^{-1}[(bc)\lambda {(bc)}^{-1}]\in H$. If $x\notin \{a,b,c\}$, then

so $\xi \in H\cap D$. Moreover,

so $\xi \ne 1$. In this case, $H\cap D\ne \{1\}$.

$\bullet$ It remains the case where ${\lambda }^2=1$ for all $\lambda \in H$. Since $H\ne \{1\}$, there is some $\lambda \in H$ such that $\lambda \ne 1$ and ${\lambda }^2=1$. Then there exists $a\in \mathrm{\Omega }$ such that $b=\lambda (a)\ne a$ and $\lambda (b)=a$. If $\lambda (x)=x$ for all $x\notin \{a,b\}$, then $\lambda =(ab)\in H\cap D-\{1\}$. Otherwise there is some $c\notin \{a,b\}$ such that $d=\lambda (c)\ne c$. Then

where $\mu$ fixes the four distinct elements $a,b,c,d$, so $\mu$ commutes with the transpositions $(ab),(cd),(ac),(ad)$.

Consider the permutation $\chi ={\lambda }^{-1}(bc)\lambda {(bc)}^{-1}$. Since $H⊴S_{\mathrm{\Omega }}$, $\chi \in H$. Moreover,

thus

In both cases,

Consider the subgroup $\tilde{H}=H\cap D$, where $D$ is defined as in Exercise 6. We prove $A\le \tilde{H}$ (so $A\le H$).

Since $A$ is simple by Exercise 6, and $\tilde{H}\cap A=H\cap A$ is normal in $A$, we obtain

We must exclude $\tilde{H}\cap A=\{1\}$. Suppose that $\tilde{H}\cap A=\{1\}$. If $\sigma ,\tau \in \tilde{H}-\{1\}$, then $\sigma$ and $\tau$ are odd (this makes sense because $\tilde{H}\le D$), therefore $\mathit{𝜎𝜏}$ is even, thus $\mathit{𝜎𝜏}\in \tilde{H}\cap A=\{1\}$, so $\tau ={\sigma }^{-1}$. In particular ${\sigma }^2=1$, so $\sigma ={\sigma }^{-1}$ and $\tau =\sigma$. This shows that $\tilde{H}-\{1\}=\varnothing$ or $\tilde{H}-\{1\}=\{\sigma \}$ for some $\sigma$, or equivalently $\tilde{H}=\{1\}$ or $\tilde{H}=\{1,\sigma \}$, so $\tilde{H}$ has at most two elements. Since $\tilde{H}=H\cap D\ne \{1\}$ by (1), we obtain

Then $\sigma$ has order $2$, so $\sigma =(ij)(\cdots )\cdots \in D$ is a (finite) product of disjoint transpositions. Consider the transposition $\tau =(jk)$, where $k\ne i,k\ne j$. Then

is also a product of disjoint transpositions. Therefore $\tau \ne 1$, and $\mathit{𝜏𝜎}{\tau }^{-1}\ne \sigma$, since $\sigma (i)\ne \mathit{𝜏𝜎}{\tau }^{-1}(i)$, hence $\mathit{𝜏𝜎}{\tau }^{-1}\notin \tilde{H}$. This contradicts $\tilde{H}⊴A$. This proves $\tilde{H}\cap A\ne \{1\}$. Therefore, by (2), we obtain $\tilde{H}\cap A=A$, so $A\le \tilde{H}\le H$.

(If $A$ and $A^{\prime }$ are both nontrivial minimal normal subgroup of $S_{\mathrm{\Omega }}$, then $A\le A^{\prime }$ and $A^{\prime }\le A$, so $A=A^{\prime }$.)

In conclusion, $A$ is the unique nontrivial minimal normal subgroup of $S_{\mathrm{\Omega }}$. □