Exercise 7.1.1

For the original function, $F=\mathit{ReLU}(x)+\mathit{ReLU}(y)+\mathit{ReLU}(z)$, we have 8 linear pieces, i.e. 

1.

$F=x+y+z$, where $x>0,y>0,z>0$

• $F=x+y$, where $x>0,y>0,z<0$
• $F=x+z$, where $x>0,y<0,z>0$
• $F=x$, where $x>0,y<0,z<0$
• $F=y+z$, where $x<0,y>0,z>0$
• $F=y$, where $x<0,y>0,z<0$
• $F=z$, where $x<0,y<0,z>0$
• $F=0$, where $x<0,y<0,z<0$

If we add the $4$th fold of $\mathit{ReLU}(x+y+z)$, its fold plane is $x+y+z=0$, for this fold plane to cut the original pieces, it requires solution for the fold plane in each piece. It’s clear to see that no solution exists for the first ($x>0,y>0,z>0$) and last piece ($x<0,y<0,z<0$). The new fold plane doesn’t cut through them. So there are a total of $2\times 6+2=14$ pieces.

The $14$ pieces are:

1.

$F=2x+2y+2z$, where $x>0,y>0,z>0$

• $F=2x+2y+z$, where $x>0,y>0,z<0,x+y+z>0$
• $F=x+y$, where $x>0,y>0,z<0,x+y+z<0$
• $F=2x+y+2z$, where $x>0,y<0,z>0,x+y+z>0$
• $F=x+z$, where $x>0,y<0,z>0,x+y+z<0$
• $F=2x+y+z$, where $x>0,y<0,z<0,x+y+z>0$
• $F=x$, where $x>0,y<0,z<0,x+y+z<0$
• $F=x+2y+2z$, where $x<0,y>0,z>0,x+y+z>0$
• $F=y+z$, where $x<0,y>0,z>0,x+y+z<0$
• $F=x+2y+z$, where $x<0,y>0,z<0,x+y+z>0$
• $F=y$, where $x<0,y>0,z<0,x+y+z<0$
• $F=x+y+2z$, where $x<0,y<0,z>0,x+y+z>0$
• $F=z$, where $x<0,y<0,z>0,x+y+z<0$
• $F=0$, where $x<0,y<0,z<0$