Upper triangular parts of conjugacy classes of nilpotent matrices with finite number of $B$-orbits

We consider the intersection of the conjugacy class of a nilpotent matrix with the space of upper triangular matrices. We give necessary and sufficient conditions for this intersection to be a union of finitely many orbits for the action by conjugation of the group of invertible upper triangular matrices.