Exercise 3.5.6 (Comparison of the classes of $ax^2 + bxy +cy^2$ and $ax^2 - bxy +cy^2$)

Proof. Put $f_1=(1,-1,2)$ and $g_1=(1,1,2)$. Then $f_1$ is not a reduced form. The algorithm of reduction gives $f_1\cdot T=g_1$, where $T=\left(\begin{array}{cc}\hfill 1\hfill & \hfill 1\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right)\in \Gamma$. Explicitly,

This shows that $x^2+\mathit{xy}+2y^2$ is properly equivalent to $x^2-\mathit{xy}+2y^2$.

The forms $f_2=(3,1,4)$ and $g_2=(3,-1,4)$ are two reduced forms, and are positive definite. By Section 3.7, in this case, the reduced form in a given equivalence class is unique, so $f_2$ and $g_2$ are not properly equivalent.

The direct proof is cumbersome: assume that there is some $A=\left(\begin{array}{cc}\hfill a\hfill & \hfill b\hfill \\ \hfill c\hfill & \hfill d\hfill \end{array}\right)\in {\mathrm{GL}}_2(\mathbb{Z})$such that $g_2=f_2\cdot A$. Then

Therefore

The first equation gives by completing the square

Thus $47c^2\le 36$, so $c=0,a=\pm 1$.

The third equation gives

Then $47d^2\le 48$, thus $|d|\le 1$. If $d=0$, then $3b^2=4$. This is impossible, so $d=\pm 1$. Then $b(3b+d)=0$, and $3b\pm 1=0$ is impossible, thus $b=0$ (and $d=\pm 1$).

Then the second equation with $b=c=0$ gives $\mathit{ad}=-1$, where $a=\pm 1$, thus $d=-a$. The only matrices $A\in {\mathrm{GL}}_2(\mathbb{Z})$ such that $g_2=f_2\cdot A$ are

Then $\det <!--\; FUNCTION\; APPLICATION\; -->(A)=-1$, so $A\notin \Gamma ={\mathrm{SL}}_2(\mathbb{Z})$. This shows that $3x^2+\mathit{xy}+4y^2$ and $3x^2-\mathit{xy}+4y^2$ are not properly equivalent. □