The Hard Case: Factoring Quadratics with a Box