Business list vs ground observation for measuring a food environment: saving time or waste of time (or worse)?

In food-environment research, an alternative to resource-intensive direct observation on the ground has been the use of commercial business lists. We sought to determine how well a frequently used commercial business list measures a dense urban food environment like the Bronx, NY. On 155 Bronx street segments, investigators compared two different levels for matches between the business list and direct ground observation: lenient (by business type) and strict (by business name). For each level of matching, researchers calculated sensitivities and positive predictive values (PPVs) for the business list overall and by broad business categories: General Grocers (eg, supermarkets), Specialty Food Stores (eg, produce markets), Restaurants, and Businesses Not Primarily Selling Food (eg, newsstands). Even after cleaning the business list (eg, for cases of multiple listings at a single location), and allowing for inexactness in listed street addresses and spellings of business names, the overall performance of the business list was poor. For strict matches, the business list had an overall sensitivity of 39.3% and PPV of 45.5%. Sensitivities and PPVs by broad business categories were not meaningfully different from overall values, although sensitivity for General Grocers and PPV for Specialty Food Stores were particularly low: 26.2% and 32%, respectively. For lenient matches, sensitivities and PPVs were somewhat higher but still poor: 52.4% to 60% and 60% to 75%, respectively. The business list is inadequate to measure the actual food environment in the Bronx. If results represent performance in other settings, findings from prior studies linking food environments to diet and diet-related health outcomes using such business lists are in question, and future studies of this type should avoid relying solely on such business lists.