A complex mode-locking (or entrainment) structure underlying the nonlinear whirling phenomenon of a horizontal Jeffcott rotor with a discontinuous nonlinearity (bearing clearance) was identified. A winding number is introduced as a measure of the ratio between two frequencies involved in the aperiodic whirling motions of the rotor system considered. Utilizing the winding number map, it was revealed that the alternating periodic and quasi-periodic responses take place according to the Farey number tree. The winding number varies in the form of the so-called “Devil’s staircase” as a certain system parameter varies. From the mode-locking pattern in the parameter space of the forcing amplitude and frequency, it was observed that as the forcing amplitude increases, the size of each locking interval increases so that its growth takes place in the form of “Arnol’d tongues,” where the winding number remains a rational number. Moreover, inside each locking zone, i.e., each “Arnol’d tongue,” there exist many smaller tongues similar to the main tongue, in which a sequence of period-doubling bifurcations leading to chaos occurred. The boundaries of each locking zone was obtained using a fixed-point algorithm along with the Floquet theory for checking the stability of the periodic solutions. The winding numbers were estimated utilizing a fixed-point algorithm modified to obtain quasi-periodic responses. A jump phenomenon was also observed by tracking multiple periodic solutions for several parameters of the rotor system.