In the field of image coding, it has recently been shown that quad-tree schemes for representing geometric image features are unable to obtain the optimal exponentially decaying rate-distortion behaviour - a problem which can be rectified by following R-D optimal tree pruning with a leaf merging step. Inspired by such results, we note that quad-tree based motion representations for video compression suffer from the same fundamental shortcomings, which can again be overcome by leaf merging. Based on these observations, we describe a non-iterative low complexity extension to the existing motion model used by H.264, demonstrating significant improvements in overall compression performance. At low to moderate bit-rates, our experimental implementation yields reductions of 10% or more in overall bit-rate, relative to H.264. Perhaps more importantly, this paper reinforces the importance of leaf merging as a general tool for quad-tree based modeling schemes.