The United States Environmental Protection Agency mandates that community water systems (CWSs), or drinking water utilities, provide annual consumer confidence reports (CCRs) reporting on water quality, compliance with regulations, source water, and consumer education. While certain report formats are prescribed, there are no criteria ensuring that consumers understand messages in these reports. To assess clarity of message, trained raters evaluated a national sample of 30 CCRs using the Centers for Disease Control Clear Communication Index (Index) indices: (1) Main Message/Call to Action; (2) Language; (3) Information Design; (4) State of the Science; (5) Behavioral Recommendations; (6) Numbers; and (7) Risk. Communication materials are considered qualifying if they achieve a 90% Index score. Overall mean score across CCRs was 50 ± 14% and none scored 90% or higher. CCRs did not differ significantly by water system size. State of the Science (3 ± 15%) and Behavioral Recommendations (77 ± 36%) indices were the lowest and highest, respectively. Only 63% of CCRs explicitly stated if the water was safe to drink according to federal and state standards and regulations. None of the CCRs had passing Index scores, signaling that CWSs are not effectively communicating with their consumers; thus, the Index can serve as an evaluation tool for CCR effectiveness and a guide to improve water quality communications.