Exercise 3.1.25 ($g N g^{-1} \subseteq N$ does not imply $gNg^{-1} = N$: counterexample in $\mathrm{GL}_n(\mathbb{Q})$)

Proof. By definition, a subgroup $N$ of $G$ is normal in $G$ if and only if $\mathit{gN}{g}^{-1}=N$ for all $g$ in $G$.

(a)

If $N⊴G$, then $\mathit{gN}{g}^{-1}=N$. A fortiori $\mathit{gN}{g}^{-1}\subseteq N$.

Conversely, suppose that $\mathit{gN}{g}^{-1}\subseteq N$ for all $g\in G$. For any $g\in G$, $\mathit{gN}{g}^{-1}\subseteq N$. Moreover $g^{-1}\in G$, so $g^{-1}N{(g^{-1})}^{-1}\subseteq N$, i.e., $g^{-1}\mathit{Ng}\subseteq N$, therefore $N\subseteq \mathit{gN}{g}^{-1}$.

(If $n\in N$, then $g^{-1}\mathit{ng}\in g^{-1}\mathit{Ng}\subseteq N$, so $m=g^{-1}\mathit{ng}\in N$ and $n=\mathit{gm}{g}^{-1}\in \mathit{gN}{g}^{-1}$. This proves $N\subseteq \mathit{gN}{g}^{-1}$.)

Since $\mathit{gN}{g}^{-1}\subseteq N$ and $N\subseteq \mathit{gN}{g}^{-1}$, we obtain $\mathit{gN}{g}^{-1}=N$ for all $g$ in $G$. In conclusion,

$$N⊴G⟺\forall <!--\; FUNCTION\; APPLICATION\; -->g\in G,\mathit{gN}{g}^{-1}\subseteq N.$$

(b)

We verify that $N$ is a subgroup of $G$. First $I_2\in N$, so $N\ne \varnothing$, and if $A=\left(\begin{array}{cc}\hfill 1\hfill & \hfill a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)$, then $\det <!--\; FUNCTION\; APPLICATION\; -->(A)=1$, thus $A\in {\mathrm{GL}}_2(\mathbb{Q})$. So $N\subseteq {\mathrm{GL}}_2(\mathbb{Q})$.

Let $A=\left(\begin{array}{cc}\hfill 1\hfill & \hfill a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)$ and $B=\left(\begin{array}{cc}\hfill 1\hfill & \hfill b\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)$ be any elements of $N$, so that $a,b\in \mathbb{Z}$.

Since $-a\in \mathbb{Z}$ and $a+b\in \mathbb{Z}$, we obtain

$$\begin{array}{llll}\hfill A^{-1}& =\left(\begin{array}{cc}\hfill 1\hfill & \hfill -a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\in N,& \hfill & \\ \hfill \mathit{AB}& =\left(\begin{array}{cc}\hfill 1\hfill & \hfill a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 1\hfill & \hfill b\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)=\left(\begin{array}{cc}\hfill 1\hfill & \hfill a+b\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\in N,& \hfill & \end{array}$$

so $N$ is a subgroup of $G$ (note that $N\simeq \mathbb{Z}$).

If $A=\left(\begin{array}{cc}\hfill 1\hfill & \hfill a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)(a\in \mathbb{Z})$ is any element of $N$, and $g=\left(\begin{array}{cc}\hfill 2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)$, then

$$\begin{array}{llll}\hfill \mathit{gA}{g}^{-1}& =\left(\begin{array}{cc}\hfill 2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 1\hfill & \hfill a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 1∕2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)& \hfill & \\ \hfill & =\left(\begin{array}{cc}\hfill 2\hfill & \hfill 2a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 1∕2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)& \hfill & \\ \hfill & =\left(\begin{array}{cc}\hfill 1\hfill & \hfill 2a\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right).& \hfill & \end{array}$$

Since $2a\in \mathbb{Z}$, $\mathit{gA}{g}^{-1}\in N$. This shows that

$$\mathit{gN}{g}^{-1}\subseteq N.$$

But for $A_0=\left(\begin{array}{cc}\hfill 1\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\in N$,

$$\begin{array}{llll}\hfill g^{-1}{A}_{0}g& =\left(\begin{array}{cc}\hfill 1∕2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 1\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)& \hfill & \\ \hfill & =\left(\begin{array}{cc}\hfill 1∕2\hfill & \hfill 1∕2\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\left(\begin{array}{cc}\hfill 2\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)& \hfill & \\ \hfill & =\left(\begin{array}{cc}\hfill 1\hfill & \hfill 1∕2\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right).& \hfill & \end{array}$$

Therefore $g^{-1}{A}_{0}g\notin N$, since $1∕2\notin \mathbb{Z}$. Hence $g^{-1}\mathit{Ng}⊈N$, thus $g^{-1}\mathit{Ng}\ne N$, or equivalently $N\ne \mathit{gN}{g}^{-1}$. This shows that $g$ does not normalize $N$, even if $\mathit{gN}{g}^{-1}\subseteq N$.