Investigation of the electromagnetic structure of the η meson in the decay η → μ+μ−γ