Eliminating the mixed xy term in conics
To remove the mixed $xy$ term from a conic, you simply rotate the axes by an angle $\alpha$ such that $$\tan 2\alpha = \frac{B}{A - C}$$ where $A, B, C$ are the coefficients of the general conic equation. The rotation angles that satisfy this condition are $$ \alpha = \tfrac{1}{2} \arctan \left(\frac{B}{A - C}\right) + \tfrac{k\pi}{2} $$
There is always a rotation angle that will eliminate the mixed $xy$ term from the equation of a conic.
To determine this angle:
- Identify the coefficients $(A, B, C)$ in the general conic equation: $$ Ax^2 + Bxy + Cy^2 + Dx + Ey + F = 0 $$
- Apply the formula for the rotation angle: $$ \tan 2\alpha = \frac{B}{A - C} $$
Note. If the denominator $A - C$ is zero, the tangent is undefined (the angle is vertical). In this special case, the condition simplifies to $$ \cos 2\alpha = 0 \quad \Longrightarrow \quad 2\alpha = \tfrac{\pi}{2} + k\pi $$
- Compute the angle of rotation $ \alpha $ and choose one solution: $$ \alpha = \tfrac{1}{2} \arctan \left(\frac{B}{A - C}\right) + \tfrac{k\pi}{2} $$
- Rotate the axes by the chosen angle $ \alpha $: $$ \begin{cases} x = x'\cos\alpha - y'\sin\alpha \\ \\ y = x'\sin\alpha + y'\cos\alpha \end{cases}$$
- Rewrite the conic equation. The mixed term $ x'y' $ will disappear.
Worked Example
Consider the following conic equation, which contains a mixed term:
$$ 4x^2 + 8xy + 4y^2 + \sqrt{2} x - \sqrt{2} y = 0 $$
Here the coefficients are: $$ A=4, \ B=8, \ C=4, \ D=\sqrt{2}, \ E=-\sqrt{2}, \ F=0$$
Now, calculate the angle that removes $xy$ using the formula:
$$ \tan 2\alpha = \frac{B}{A-C} $$
Since $A-C=0 $, $\tan 2\alpha$ is undefined.
In this situation, we use the alternative condition $ \cos 2\alpha=0$, which gives:
$$ 2\alpha = \arccos 0 $$
$$ 2 \alpha = \frac{\pi}{2}+k\pi $$
$$ \alpha=\frac{\pi}{4}+k\frac{\pi}{2} $$
For simplicity, let’s take $\alpha=\frac{\pi}{4}$ and perform the rotation:
$$ \begin{cases} x = x'\cos\alpha + y'\sin\alpha \\ \\ y = -x'\sin\alpha + y'\cos\alpha \end{cases} $$
$$ \begin{cases} x = x'\cos \frac{\pi}{4} + y'\sin \frac{\pi}{4} \\ \\ y = -x'\sin \frac{\pi}{4} + y'\cos \frac{\pi}{4} \end{cases} $$
Since $ \cos \frac{\pi}{4} = \frac{\sqrt{2}}{2} $ and $ \sin \frac{\pi}{4} = \frac{\sqrt{2}}{2} $:
$$ \begin{cases} x = x' \frac{\sqrt{2}}{2} + y' \frac{\sqrt{2}}{2} \\ \\ y = -x' \frac{\sqrt{2}}{2} + y' \frac{\sqrt{2}}{2} \end{cases} $$
$$ \begin{cases} x = \frac{\sqrt{2}}{2} (x' + y') \\ \\ y = \frac{\sqrt{2}}{2} ( y' -x' ) \end{cases} $$
Substituting $ x $ and $ y $ into the original equation:
$$ 4x^2 + 8xy + 4y^2 + \sqrt{2} x - \sqrt{2} y = 0 $$
Now expand the quadratic terms $ 4x^2, 8xy , 4y^2 $ as well as the linear terms $ \sqrt{2} x, \sqrt{2} y $:
- $ 4x^2 = 4 ( \frac{\sqrt{2}}{2} (x' + y') )^2 = 2x'^2+4x'y'+2y'^2 $
- $ 8xy = 8 ( \frac{\sqrt{2}}{2} (x' + y') ) ( \frac{\sqrt{2}}{2} ( y' -x' ) ) = 4y'^2 -4x'^2 $
- $ 4y^2 = 4 ( \frac{\sqrt{2}}{2} (y' -x') )^2 = 2x'^2-4x'y'+2y'^2 $
- $ \sqrt{2} x = \sqrt{2} \frac{\sqrt{2}}{2} (x' + y') = x'+y' $
- $ - \sqrt{2} y = - \sqrt{2} \frac{\sqrt{2}}{2} (y' -x' ) = x'-y' $
Putting everything together, the equation becomes:
$$ (2x'^2+4x'y'+2y'^2) + ( 4y'^2 -4x'^2 ) + ( 2x'^2 -4x'y'+2y'^2 ) + (x'+y' ) + (x'-y' ) = 0 $$
$$ 2x'^2+4x'y'+2y'^2 + 4y'^2 -4x'^2 + 2x'^2 -4x'y'+2y'^2 + x'+y' + x'-y' = 0 $$
$$ x^2 (2-4+2) + y'^2 (2 +4 +2) + x'y' (4-4) + 2x' = 0 $$
$$ 0 \cdot x^2 + 8 \cdot y'^2 + 0 \cdot x'y' + 2x' = 0 $$
$$ 8 y'^2 + 2x' = 0 $$
In the rotated system with $\alpha=\frac{\pi}{4}$, the mixed term $ xy $ has vanished.
$$ 2x' = - 8 y'^2 $$
$$ x'=-4 y'^2 $$
After rotation, the parabola has its vertex at the origin, its axis parallel to the $x'$-axis, and it opens to the left:

Proof
We begin with the general equation of a conic:
$$ Ax^2 + Bxy + Cy^2 + Dx + Ey + F = 0 $$
Our task is to determine the rotation angle $ \alpha $ that eliminates the coefficient $ B $, which multiplies the mixed term $ xy $.
In analytic geometry, a rotation of the axes is described by:
$$ \begin{cases} x = x'\cos\alpha - y'\sin\alpha \\ y = x'\sin\alpha + y'\cos\alpha \end{cases} $$
Substituting $ x $ and $ y $ into the conic equation, we expand term by term:
- The term $ Ax^2 $: $$ \begin{aligned} Ax^2 &= A\,(x' \cos \alpha - y' \sin \alpha)^2 \\ &= A\,(x'^2 \cos^2 \alpha - 2x'y' \sin \alpha \cos \alpha + y'^2 \sin^2 \alpha) \end{aligned} $$
- The term $ Bxy $: $$ \begin{aligned} Bxy &= B\,(x' \cos \alpha - y' \sin \alpha)(x' \sin \alpha + y' \cos \alpha) \\ &= B\,(x'^2 \cos \alpha \sin \alpha + x'y'(\cos^2 \alpha - \sin^2 \alpha) - y'^2 \sin \alpha \cos \alpha) \end{aligned} $$
- The term $ Cy^2 $: $$ \begin{aligned} Cy^2 &= C\,(x' \sin \alpha + y' \cos \alpha)^2 \\ &= C\,(x'^2 \sin^2 \alpha + 2x'y' \sin \alpha \cos \alpha + y'^2 \cos^2 \alpha) \end{aligned} $$
- The term $ Dx $: $$ \begin{aligned} Dx &= D\,(x'\cos\alpha - y'\sin\alpha) \\ &= (D\cos\alpha)\,x' + (-D\sin\alpha)\,y' \end{aligned} $$
- The term $ Ey $: $$ \begin{aligned} Ey &= E\,(x'\sin\alpha + y'\cos\alpha) \\ &= (E\sin\alpha)\,x' + (E\cos\alpha)\,y' \end{aligned} $$
Now collect the coefficients of the mixed term $ x'y' $:
$$ B' = -2A x'y' \sin \alpha \cos \alpha + B x'y' ( \cos^2 \alpha - \sin^2 \alpha) + 2C x'y' \sin \alpha \cos \alpha $$
$$ B' = x'y' \cdot \big( -2A \sin \alpha \cos \alpha + B ( \cos^2 \alpha - \sin^2 \alpha) + 2C \sin \alpha \cos \alpha \big) $$
By the zero-product property, $ B' = 0 $ if and only if one of the two factors is zero.
The first factor, $ x'y'=0 $, is trivial. The relevant condition comes from the second factor:
$$ - 2A \sin \alpha \cos \alpha + B ( \cos^2 \alpha - \sin^2 \alpha) + 2C \sin \alpha \cos \alpha = 0 $$
Rearranging terms gives:
$$ (2C - 2A) \sin \alpha \cos \alpha + B ( \cos^2 \alpha - \sin^2 \alpha) = 0 $$
$$ (C - A) \cdot 2 \sin \alpha \cos \alpha + B ( \cos^2 \alpha - \sin^2 \alpha) = 0 $$
Using the double-angle identities, $ 2 \sin \alpha \cos \alpha = \sin 2 \alpha $ and $ \cos^2 \alpha - \sin^2 \alpha = \cos 2 \alpha $, we obtain:
$$ (C - A) \sin 2 \alpha + B \cos 2 \alpha = 0 $$
$$ (C - A) \sin 2 \alpha = - B \cos 2 \alpha $$
Dividing through by $ \cos 2 \alpha $ yields:
$$ (C - A) \frac{ \sin 2 \alpha }{ \cos 2 \alpha } = - B $$
Since $ \tfrac{\sin \theta}{\cos \theta} = \tan \theta $, it follows that:
$$ (C - A) \tan 2 \alpha = - B $$
$$ \tan 2 \alpha = \frac{- B }{C-A } $$
Multiplying numerator and denominator by $ -1 $ gives:
$$ \tan 2 \alpha = \frac{ B }{A-C } $$
Thus, whenever $ \tan 2 \alpha = \tfrac{ B }{A-C } $, the coefficient $ B' $ of the mixed term is zero. This establishes the condition required to eliminate the mixed term.
From this relation we have:
$$ 2\alpha = \arctan \left(\frac{B}{A - C}\right) + k\pi, \qquad k \in \mathbb{Z} $$
Which leads to:
$$ \alpha = \tfrac{1}{2} \arctan \left(\frac{B}{A - C}\right) + \tfrac{k\pi}{2} $$
In other words, there is not a single solution for $\alpha$, but rather an entire family of angles differing by $\tfrac{\pi}{2}$.
For practical purposes, it is customary to choose the solution lying between $-\tfrac{\pi}{4}$ and $\tfrac{\pi}{4}$.
And this completes the proof.
