Cables are prone to vibration due to their low inherent damping characteristics. Recently, negative stiffness dampers have gained attentions, because of their promising energy dissipation ability. The viscous inertial mass damper (termed as VIMD hereinafter) can be viewed as one realization of the inerter. It is formed by paralleling an inertial mass part with a common energy dissipation element (e.g., viscous element) and able to provide pseudo-negative stiffness properties to flexible systems such as cables. A previous study examined the potential of IMD to enhance the damping of stay cables. Because there are already models for common energy dissipation elements, the key to establish a general model for IMD is to propose an analytical model of the rotary mass component. In this paper, the characteristics of the rotary mass and the proposed analytical model have been evaluated by the numerical and experimental tests. First, a series of harmonic tests are conducted to show the performance and properties of the IMD only having the rotary mass. Then, the mechanism of nonlinearities is analyzed, and an analytical model is introduced and validated by comparing with the experimental data. Finally, a real-time hybrid simulation test is conducted with a physical IMD specimen and cable numerical substructure under distributed sinusoidal excitation. The results show that the chosen model of the rotary mass part can provide better estimation on the damper’s performance, and it is better to use it to form a general analytical model of IMD. On the other hand, the simplified damper model is accurate for the preliminary simulation of the cable responses.