XLF, also known as Cernunnos, is a newly identified core factor of the non-homologous end-joining (NHEJ) pathway for DNA double-strand breaks (DSBs) repair. XLF is known to stimulate DNA ligase IV in vitro through its interaction with XRCC4. Here, we outline the key findings on the dynamic behavior of XLF and XRCC4 at DSBs in living cells. XLF is quickly recruited to DSBs in the absence of XRCC4 or DNA-PKcs. The recruited XLF molecules constantly exchange at DSBs, and XRCC4 modulates the exchange rate of the recruited XLF. XRCC4 can be recruited to DSBs without DNA-PKcs, but DNA-PKcs stabilizes the recruited XRCC4. These observations are inconsistent with the prevailing concept that NHEJ proteins are sequentially recruited to DSBs, which is mainly supported by in vitro evidence. We propose a novel two-phase model for the assembly of NHEJ factors to DSBs in vivo. XLF, XRCC4, and DNA-PKcs are independently recruited to Ku-bound DSBs. The recruited factors are assembled into a large complex, in which the protein interactions observed in vitro define the stability of the recruited factors. This new view has broad implications for the mechanism of DSB sensing and functional protein assembly in the NHEJ pathway.