Reliability of an instrument for direct observation of urban neighbourhoods

There is an increasing interest in evaluating physical and social characteristics of deprived and wealthy neighbourhoods to explain social health disparities. Direct observations enable obtaining a comprehensive and objective representation of such characteristics. The quality of this representation rests on the use of a reliable and valid observation tool. This study reports measurement properties of an instrument that was developed to directly observe structural and contextual environmental characteristics of urban neighbourhoods with respect to urban form (e.g., street safety elements, parking), negative attributes (e.g., rubbish, crumbling masonry, graffiti), and positive attributes (e.g., maintained dwellings, sitting benches). Two trained evaluators rated the condition of streets, public spaces, dwelling units and the presence/absence of specific characteristics of 250 street blocks within 30 socio-economically diverse census tracts of the Montreal Metropolitan Census Area in 2003. Inter- and intra-rater reliability was assessed using kappa and internal consistency was ascertained using Cronbach's alpha. Inter- and intra-rater reliability was excellent (>0.80) for items pertaining to urban form, and substantial (>0.60) for domains pertaining to positive and negative characteristics. Internal consistency was acceptable (>0.70) for urban form and negative characteristics, but not for positive characteristics. This study demonstrates the psychometric properties of an instrument designed to measure positive, negative, and urban form characteristics which has utility for examining various environmental effects on many aspects of health in a wide range of research contexts.