Options for adapting to changing water resources' availability due to climate change have been routinely assessed for ability to alleviate expected impacts. Methods for evaluating effectiveness of adaptation options are usually based on, fundamentally different but complementary, hazards and vulnerability assessment based approaches. In this paper, we describe a framework to integrate these approaches for improved climate change adaptation policy making. We further review assessment approaches and corresponding criteria used for evaluation of planned adaptation options in the water sector. A synthesis of methods for evaluation reveals four main categories: (i) natural sciences based, (ii) social sciences based, (iii) overlapping and (iv) trans-disciplinary. Criteria from reviewed studies are classified using a scale vis-à-vis theme based classification, and unified into a criteria set covering important adaptation requirements while allowing case-specific modification. We find that important criteria, such as temporal scope of adaptation and implementability, have not been explicitly considered in scientific literature. The type of adaptation has an important bearing on evaluation and should be effectively modelled to make an informed decision regarding performance. Moreover, we suggest that it is important to explicitly consider influence of future changes in non-climatic factors along with climatic changes for a more robust analysis.