We study the dispersion interaction of the van der Waals and Casimir-Polder (vdW-CP) type between a neutral atom and the surface of a metal by allowing for nonlocal electrodynamics, i.e. electron diffusion. We consider two models: (i) bulk diffusion, and (ii) diffusion in a surface charge layer. In both cases that the transition to a semiconductor as a function of the conductivity is continuous, unlike the case of a local model. The relevant parameter is the electric screening length and depends on the carrier diffusion constant. We find that for distances comparable to the screening length, vdW-CP data can distinguish between bulk and surface diffusion, hence it can be a sensitive probe for surface states.