Interactions of spectator partons in the Drell-Yan process