Phosphorus (P) in a water body is mainly controlled by the interaction between surface sediment and the overlying water column after the complete control of external pollution. Significant enhancement of P in a water body would cause eutrophication of lakes. Thus, a better understanding is needed of the occurrences of P between the sediment and water column in eutrophic lakes. Here, we measured total phosphorus (TP) and major elements (Fe, Al, Ca, Mn, Si) in the water column, and total nitrogen, organic matter, TP and major oxides (Fe2O3, Al2O3, CaO, SiO2) in surface sediment of Chaohu Lake, a continuously eutrophic lake. The results showed that the rank of TP levels was western lake > eastern lake > southern lake. There were significantly positive correlations between TP (including water TP and sedimentary TP) and Fe, Al, Mn, while the correlation coefficients between water TP and sedimentary TP were −0.43, −0.41 and 0.18 for the western, eastern and southern lake respectively. The negative and significant correlations of water TP and sedimentary TP may indicate that the risk of sedimentary P release was great in the western and eastern lake during algae bloom sedimentation, while the southern lake showed weak P exchange between the sediment and water column.