We consider the role of N=4 conformal supergravity in the relation between N=4 SYM theory and D=5 gauged supergravity expanded near the Anti de Sitter background. We discuss the structure of the SYM effective action in conformal supergravity background, in particular, terms related to conformal anomaly. Solving the leading-order Dirichlet problem for the metric perturbation in AdS background we explicitly compute the bilinear graviton term in the D=5 Einstein action, demonstrating its equivalence to the linearized Weyl tensor squared part of the gravitational effective action induced by SYM theory. We also compute the graviton-dilaton-dilaton 3-point function which is found to have the form consistent with conformal invariance of the boundary theory.