To avoid the one-sidedness of resilience evaluation for automatic operation systems in single fault scenarios, the Monte Carlo simulation method is used to simulate the randomness of system disturbance and performance degradation, and provide a probability-based resilience evaluation method for automatic operation systems. Taking the Yanfang line as an example, the fault data of the signal and integrated supervisory control systems in the past three years are counted, and the fault frequency and fault recovery time distribution of each piece of equipment are calculated. The performance degradation of the equipment is classified according to the failure consequence of the equipment, and the probability of each performance degradation level is determined according to the failure data. The “virtual value of the state” is set for each performance level for quantification. The weight of the output data stream of the device is the same as the “virtual value of the state” of the device. The fluctuation of the system performance is represented by the change in the weight of the total data stream of the system. Using the “Zobel” resilience quantization method, the estimated value of the system resilience is 0.9893 after 105 simulations. In addition, 105 perturbation simulations are performed for each device, and the resilience probability cumulative distribution of the system for each device is obtained. The results indicate that the resilience distribution of the system is wide when CI, ATP, and ZC are disturbed, whereas the system resilience exhibits a low value when the switch machine and track circuit are disturbed. In general, the system resilience is relatively stable, remaining above 0.97.