Exercise 3.7.7* (Smallest values of $f(x,y)$)

Proof. By hypothesis $f(x,y)=a{x}^2+\mathit{bxy}+c$ is a reduced positive definite formform, so

and in both cases

(and $1\le a<c$).

Suppose that $x\wedge y=1$. We prove that either $f(x,y)>a+|b|+c$, or $f(x,y)\in V=\{a,c,a-b+c,a+b+c\}$.

• Suppose that $y=0$. Then $x=\pm 1$, and $f(x,y)=a\in V$.

• Suppose that $|y|=1$.

  • If $x=0$, then $f(x,y)=c\in V$.
  • If $x=\pm 1$, then $f(x,y)=a\pm b+c\in V$.

  • If $x=\pm 2$, then

    $$\begin{array}{llll}\hfill f(x,y)& =4a\pm 2b+c& \hfill & \\ \hfill & \ge 4a-2|b|+c& \hfill & \\ \hfill & \ge a+|b|+c,& \hfill & \end{array}$$

    because this last inequality is equivalent to $3a\ge 3|b|$.

    If $a>|b|$, then $f(x,y)>a+|b|+c$, and we are done. Otherwise $b=\pm a$, but (1) shows that $b\ne -a$, so $a=b$, and $f=(a,a,c)$.

    In this case,

    $$\begin{array}{llll}\hfill f(2,1)=f(-2,-1)& =6a+c>2a+c=a+|b|+c,& \hfill & \\ \hfill f(2,-1)=f(-2,1)& =2a+c=a+|b|+c\in V.& \hfill & \end{array}$$

  • If $|x|\ge 3$, then

    $$\begin{array}{llll}\hfill |2\mathit{ax}+\mathit{by}|& \ge |2\mathit{ax}|-|\mathit{by}|& \hfill & \\ \hfill & \ge 6a-|b|& \hfill & \\ \hfill & \ge 5a.& \hfill & \end{array}$$

    Therefore

    $$\begin{array}{llll}\hfill 4\mathit{af}(x,y)& ={(2\mathit{ax}+\mathit{by})}^2-d{y}^2& \hfill & \\ \hfill & \ge 25a^2-d& \hfill & \\ \hfill & =25a^2+4\mathit{ac}-b^2& \hfill & \\ \hfill & >4a(a+|b|+c),& \hfill & \end{array}$$

    because

    $$\begin{array}{llll}\hfill 25a^2+4\mathit{ac}-b^2>4a(a+|b|+c)& ⟺4a|b|+b^2<21a^2,& \hfill & \end{array}$$

    and by (2), $4a|b|+b^2\le 5a^2<21a^2$.

    Since $a>0$, this shows that $f(x,y)>a+|b|+c$.

• Suppose that $|y|=2$. Then $x$ is odd.

  • If $|x|=1$, then

    $$\begin{array}{llll}\hfill f(x,y)& =a\pm 2b+4c& \hfill & \\ \hfill & \ge a-2|b|+4c& \hfill & \\ \hfill & \ge a+|b|+c,& \hfill & \end{array}$$

    because this last inequality is equivalent to $3|b|\le 3c$.

    If $|b|<c$, then $f(x,y)>a+|b|+c$, and we are done. Otherwise, $b=\pm c$, and by (1), this implies $a=b=c$, and $f=(a,a,a)$. Then

    $$\begin{array}{llll}\hfill f(1,2)=f(-1,-2)& =7a>3a=a+|b|+c,& \hfill & \\ \hfill f(1,-2)=f(-1,2)& =3a=a+|b|+c\in V.& \hfill & \end{array}$$
  • $|x|=2$ is impossible since $x$ is odd.

  • If $|x|\ge 3$, then

    $$\begin{array}{llll}\hfill |2\mathit{ax}+\mathit{by}|& \ge |2\mathit{ax}|-|\mathit{by}|& \hfill & \\ \hfill & \ge 6a-2|b|& \hfill & \\ \hfill & \ge 4a.& \hfill & \end{array}$$

    Therefore

    $$\begin{array}{llll}\hfill 4\mathit{af}(x,y)& ={(2\mathit{ax}+\mathit{by})}^2-d{y}^2& \hfill & \\ \hfill & \ge 16a^2-4d& \hfill & \\ \hfill & =16a^2+16\mathit{ac}-4b^2& \hfill & \\ \hfill & >4a(a+|b|+c),& \hfill & \end{array}$$

    because

    $$\begin{array}{lll}\hfill 16a^2+16\mathit{ac}-4b^2>4a(a+|b|+c)⟺4b^2+4a|b|<12a^2+12\mathit{ac}& & \hfill \end{array}$$

    and by (2),

    $$4b^2+4a|b|\le 8a^2<12a^2+12\mathit{ac}.$$

• Suppose that $|y|\ge 3$. Then by (3.3),

  $$\begin{array}{llll}\hfill 4\mathit{af}(x,y)& ={(2\mathit{ax}+\mathit{by})}^2-d{y}^2& \hfill & \\ \hfill & \ge -d{y}^2& \hfill & \\ \hfill & \ge -9d=36\mathit{ac}-9b^2& \hfill & \\ \hfill & >4a(a+|b|+c),& \hfill & \end{array}$$

  because

  $$\begin{array}{llll}\hfill 36\mathit{ac}-9b^2>4a(a+|b|+c)& ⟺4a^2+4a|b|+9b^2<32\mathit{ac},& \hfill & \end{array}$$

  and by (2), $4a^2+4a|b|+9b^2\le 17a^2\le 17\mathit{ac}<32\mathit{ac}$.

  Since $a>0$, this shows that $f(x,y)>a+|b|+c$.

In all cases, either $f(x,y)>a+|b|+c$, or $f(x,y)\in V=\{a,c,a-b+c,a+b+c\}$. In other words, if $\gcd (x,y)=1$ and $f(x,y)\le a+|b|+c$, then $f(x,y)$ must be one of the numbers $a$, $c$, $a-|b|+c$ or $a+|b|+c$. □