We analyze the magnetic dipole contribution to atom-surface dispersion forces. Unlike its electrical counterpart, it involves small transition frequencies that are comparable to thermal energy scales. A significant temperature-dependence is found near surfaces with a nonzero DC conductivity, leading to a strong suppression of the dispersion force at T > 0. We use thermal response theory for the surface material and discuss both normal metals and superconductors. The asymptotes of the free energy of interaction and of the entropy are calculated analytically over a large range of distances. Near a superconductor, the onset of dissipation at the phase transition strongly changes the interaction, including a discontinuous entropy. We discuss the similarities with the Casimir interaction beween two surfaces and suggest that precision measurements of the atom-surface interaction may settle open questions around the temperature dependence of dispersion forces in dissipative media. One way to isolate the magnetic contribution could be shifts between isotopes with different spins.