With regard to our measure of performance (i.e., EPRP data), there is reason to
question its accuracy in terms of documentation. Anecdotally, we were told that
EPRP data does not necessarily reflect the true state of affairs in reference to
guideline adherence. For example, performance may actually occur even though it
is not formally documented (e.g., conducting depression screening, though
neglecting to record this activity). Conversely, providers admitted to
documenting performance that may not have actually occurred (e.g., offering
smoking cessation counseling when none was actually given). Because our outcome
data may have suffered from documentation errors, the actual effect of knowledge
creation on guideline compliance may not have been realized. Also, statistical
power was not sufficient to detect potentially meaningful effects. Saal and
Knight (1996) recommend a 10:1 case to predictor ratio for estimating
statistical power for collection of predictor data. A 31-item instrument would
therefore require 310 cases or respondents to yield sufficient power, clearly
more than we were able to acquire. Finally, this research must be quantitatively
regarded as pilot work toward the development of a survey instrument. Indeed,
survey development is an iterative process in which this project was the first
iteration.
