In order to avoid the one sidedness of resilience evaluation for automatic operation system in single fault scenario, Monte Carlo simulation method is used to simulate the randomness of system disturbance and performance degradation, and provide a probability based resilience evaluation method for automatic operation system. Taking Yanfang line as an example, the fault data of signal and integrated supervisory control systems in the past three years are counted, and the fault frequency and fault recovery time distribution of each equipment are calculated. According to the failure consequence of the equipment, the performance degradation of the equipment is classified, and according to the failure data, the probability of each performance degradation level is counted. The "virtual value of state" is set for each performance level to quantify. The weight of the output data stream of the device is the same as the "virtual value of state" of the device. The fluctuation of the system performance is represented by the change of the weight of the total data stream of the system. Using the "Zobel" resilience quantization method, after 100000 simulation, the estimated value of the system resilience is 0.9893. At the same time, 100000 perturbation simulations were carried out for each device, and the resilience probability cumulative distribution of the system for each device was obtained. The results show that the resilience distribution of the system is wide when CI, ATP and ZC are disturbed, and the system resilience may appear low value when the switch machine and track circuit are disturbed, the system resilience is relatively stable, and the system resilience remains above 0.97.