The objective of the paper is to explain a modern Hilbert transform method for analysis and identification of mechanical non-linear vibration structures in the case of quasiperiodic signals. This special kind of periodicity arises in experimental vibration signals. The method is based on the Hilbert transform of input and output signals in a time domain to extract the instantaneous dynamic structure characteristics. The paper focuses on the dynamic analysis and identification of three groups of dynamics systems: • Forced vibrations of linear and non-linear SDOF systems excited with quasiperiodic force signal. • Combined forced vibrations of quasiperiodic time varying linear and non-linear SDOF systems excited with harmonic signal. • Combined self-excited and forced vibrations of non-linear SDOF systems excited with harmonic signal. The study focuses on signal processing techniques for nonlinear system investigation, which enable us to estimate instantaneous system dynamic parameters (natural frequencies, damping characteristics and their dependencies on a vibration amplitude and frequency) for different kinds of system excitation.