The prediction of the maximum depth of the scour hole formed downstream of overflow dams is critical in determining the safety of hydraulic structures. Most of the conventional formulae are not able to consider complex hydraulic and morphologic conditions. A new formula for estimating the maximum depth of the scour hole based on computational fluid dynamics (CFD), which can be used to simulate the complicated phenomenon, is proposed. The relationship between the maximum velocity in numerical simulations and the maximum scour depth is reflected in this formula, which is established using the Levenberg–Marquardt (LM) algorithm. The validity of this proposed formula is discussed by comparing this formula with three other conventional formulae. The prediction formula based on CFD is applied to the Wuqiangxi Dam, and the absolute deviation of the predicted maximum scour depth (35.44 m) from the measured depth (36.00 m) is 0.56 m.