The mechanism by which P-glycoprotein (P-gp) interacts with a number of structurally unrelated substrates or inhibitors remains unknown. We have recently shown that a serine residue within the predicted transmembrane (TM) domain 11 of P-gps encoded by mouse mdr1 (Ser941) and mdr3 (Ser939) plays an important role in the substrate specificity of P-gp. We wished to determine if Ser939/941 is also important for efficient interaction of P-gp with structurally different modulating agents, a cyclic peptide (cyclosporin A, CsA), a diaminoquinazoline (CP100356), and a chiral, tricyclic structure (CP117227). For this, the capacity of these compounds to modulate the vinblastine (VBL) resistance phenotype of transfected cells expressing similar levels of P-gps bearing either the wild-type Ser or a mutant Phe at position 941 (mdr1) or 939 (mdr3) was initially tested. The Ser-->Phe substitution indeed affected the potency and P-gp isoform specificity of some of the modulators, in particular that of CP117227 (racemic mixture and enantiomers), which were active against wild-type but not mutant mdr3. The modulatory effect of the mutation on CP117227-mediated reversal of VBL resistance was parallelled by a comparable modulation of the steady-state levels of VBL accumulation in Ser939- and Phe939-expressing cells, but was not linked to differential cellular accumulation of the modulator, which was identical in both cell types. To further assess the role of this amino acid residue in P-gp interactions with modulators, the effect of additional mutations (Ala, Cys, Thr, Asp, Tyr, Trp) at that site on potencies of CsA, CP117227 enantiomers, and CP100356 was evaluated.(ABSTRACT TRUNCATED AT 250 WORDS)