Exercise 4.5.53 (If $G$ is any non-abelian group in which every proper subgroup is abelian then $G$ is not simple)

Proof. Reasoning by contradiction, let $G$ be a counterexample to the assertion of the statement. Then $G$ is a simple non-abelian group, and every proper subgroup of $G$ is abelian.

• Since $G\ne \{1\}$, $G$ contains some maximal subgroup $M$. We write $g_{1}M{g}_1^{-1},\dots ,g_{k}M{g}_k^{-1}$ the distinct conjugate subgroups of $M$.

  By Exercise 4.3.24, $G\ne \underset{i=1}{\overset{k}{\bigcup <!--\; FUNCTION\; APPLICATION\; -->}}{g}_{i}M{g}_i^{-1}$. Therefore there is some $h\in G$ such that $h\notin \underset{i=1}{\overset{k}{\bigcup <!--\; FUNCTION\; APPLICATION\; -->}}{g}_{i}M{g}_i^{-1}$. Then the subgroup $⟨h⟩$ is contained in some maximal subgroup $N$. Moreover $N\ne g_{i}M{g}_i^{-1}$ for all indices $i$, since $h\in N$ but $h\notin g_{i}M{g}_i^{-1}$, so $M$ and $N$ are not conjugate subgroups. We write $h_{1}N{h}_1^{-1},\dots ,h_{l}N{h}_l^{-1}$ the distinct conjugate subgroups of $N$. Then

  $$\begin{array}{lll}\hfill G-\{1\}\supseteq \underset{i=1}{\overset{k}{\bigcup }}(g_{i}M{g}_i^{-1}-\{1\})\cup \underset{j=1}{\overset{l}{\bigcup }}(h_{j}N{h}_j^{-1}-\{1\}).& & \hfill \text{(1)}\end{array}$$
• Note that $\mathit{𝑔𝑀}{g}^{-1}$ is a maximal subgroup for every $g\in G$: $\mathit{𝑔𝑀}{g}^{-1}$ is a proper subgroup of $G$, and if $\mathit{𝑔𝑀}{g}^{-1}\le H$, where $H$ is some subgroup of $G$, then $M\le g^{-1}\mathit{𝐻𝑔}$. Since $M$ is maximal, $g^{-1}\mathit{𝐻𝑔}=M$ or $g^{-1}\mathit{𝐻𝑔}=G$. In the latter case $H=G$, and in the former case $H=\mathit{𝑔𝑀}{g}^{-1}$. This proves that $\mathit{𝑔𝑀}{g}^{-1}$ is a maximal subgroup of $G$. Similarly $\mathit{h𝑁}{h}^{-1}$ is a maximal subgroup of $G$ for all $h\in G$.

• The subgroups $g_{i}M{g}_i^{-1}$ and $h_{j}N{h}_j^{-1}$ for $1\le i\le k$ and $1\le j\le l$ are distinct maximal subgroups of the simple group $G$, so they are abelian subgroups. By Exercise 52, the intersection of any two such subgroups is trivial. Therefore the union in (1) is a disjoint union. This shows that

  $$\begin{array}{llll}\hfill |G|-1& \ge \underset{i=1}{\overset{k}{\sum }}|g_{i}M{g}_i^{-1}-\{1\}|+\underset{j=1}{\overset{l}{\bigcup }}|h_{j}N{h}_j^{-1}-\{1\})|,& \hfill & \end{array}$$

  so

  $$\begin{array}{lll}\hfill |G|-1& \ge k(|M|-1)+l(|N|-1).& \hfill \text{(2)}\end{array}$$

• The maximal subgroup $M$ cannot be trivial, otherwise, for some $g\ne 1$, $M=\{1\}<⟨g⟩$ implies $G=⟨g⟩$, thus $G$ would be cyclic, but $G$ is not abelian by hypothesis. Since $M\le N_G(M)$, where $M$ is maximal, then $N_G(M)=M$ or $N_G(M)=G$. In the latter case, $M⊴G$. This is impossible since $G$ is simple and $M$ is a proper nontrivial subgroup of $G$. This proves $N_G(M)=M$, and similarly $N_G(N)=N$.

  Then the number $k$ of distinct conjugates of $M$ is $k=|G:N_G(M)|=|G:M|$, and we have a similar result for $N$:

  $$k=|G:M|,l=|G:N|.$$

• Then (2) gives

  $$\begin{array}{llll}\hfill |G|-1& \ge \frac{|G|}{|M|}(|M|-1)+\frac{|G|}{|N|}(|N|-1)& \hfill & \\ \hfill & =|G|\left(2-\frac{1}{|M|}-\frac{1}{|N|}\right)& \hfill & \\ \hfill & \ge |G|,& \hfill & \end{array}$$

  because $2-\frac{1}{|M|}-\frac{1}{|N|}\ge 1$ if and only if $\frac{1}{|M|}+\frac{1}{|N|}\le 1$, and this last assertion is true, because $M$ and $N$ are not trivial, so $|M|\ge 2$ and $|N|\ge 2$.

• $|G|-1\ge |G|$ is a contradiction, which proves that if $G$ is any non-abelian group in which every proper subgroup is abelian then $G$ is not simple.