The use of commercially available software for analyzing reliability block diagrams (RBD) has become the rule for the vast majority of reliability analysts and engineers. After a model has been developed and checked, a software package is generally used to evaluate the model. For the evaluation of system maintenance, especially in complex redundancy schemes, simulation is required to resolve the availability performance of the model. The results produced by the software packages are usually presented by the analyst without significant questions about the algorithms, simulation methodology, etc used by the particular software package the analyst uses. This paper provides a comparison of the results of three competitive packages. It was hypothesized that there would be differences in results due to differences in algorithms and simulation methodologies, particularly for complex models. It was not the intent of this paper to judge the relative accuracy of the results produced. The purpose of this paper is to provide awareness to analysts that all results of reliability modeling, including those produced by computer simulation packages, need to be understood in the context of the modeling methodology and solution algorithms and methodologies. It is also necessary that the results are presented with the assumptions used by the particular software package. The three software packages that were compared are Reliasoft BlockSim Version 6.5.2, ARINC Raptor - Version 7.0.07, and Relex Software Reliability Block Diagram. This evaluation was performed with the cooperation of the software suppliers to the maximum extent possible. The results, particularly any differences, will be reviewed with the suppliers prior to this paper's presentation at RAMS 2007. The methodology used a one block diagram, a simple diagram, a complex diagram, and a project diagram. The project diagram was based on actual hardware. The other models were hypothetical. The only restriction placed on the models was that each model must be capable of being run on all of the software packages (after conversion to each software package's protocols). Special features that might be available in one or two software packages would not be evaluated. The primary effort was to assess the differences in results created by the differences in algorithms and simulation methodologies. The hypothesis was verified, even in the single block model