We consider unidirectionally coupled ring networks of neurons with inhibitory connections. It is known that when a number of inhibitory neurons is even, the system never has stable oscillatory modes. However, we observed oscillatory modes in such a case. In this paper, we clarify the oscillation mechanism in a state space. We investigate the relationship between these oscillatory modes and the unstable periodic solutions (UPSs) generated by the Hopf bifurcations. As a result we find that these transient oscillatory modes are generated by trajectories closing to the UPS and staying around it for a long time. We also confirm this phenomenon in a simple electrical circuit using inverting operational amplifiers.