Finite element-boundary integral formulation for electromagnetic scattering