This paper proposes a new methodological pattern to assess the effectiveness of Order Review and Release (ORR) techniques in a job shop environment. The standpoint for this new method lies in the following remarks: (i) comparisons among ORR models should be performed in dynamic and uncertain environments; (ii) ORR techniques robustness toward the shop uncertainty and perturbations should be considered; and (iii) ORR models should be compared by changing their features one at a time, instead of comparing completely different ORR techniques. Consistently, we present a comparison among three ORR models, previously developed in literature, aimed at investigating: (i) the impact of a dynamic and uncertain environment on the performances achieved; (ii) the robustness of these ORR models when facing some environmental perturbations, like the system workload, the mix imbalance, the machine unavailability and the processing time variability, that usually take place in real life job shops; and (iii) the overall effectiveness of the way workload is accounted for over time, since the models differentiate only by this item, while any other feature of the release mechanism is the same. Simulation results highlight that the performances of the ORR techniques tested depend on how perturbed the environment where they are implemented is. Moreover, the ORR techniques tested greatly differ in their robustness against environment perturbations.