2.6 Using Multiple Processors
As of version 2.0, FEBio uses OpenMP to parallellize several of the finite element calculations, improving the performance considerably. Both the right-hand-side and the stiffness matrix evaluations for many types of problems have been parallelized. On a system with four processors, a speedup of 2-3 can be expected, depending on the size and type of model. Models with complex material behavior (such as EFD-type materials, biphasic, multiphasic materials, etc.) will benefit most from these parallelization efforts. In addition, FEBio implements the MKL version of the PARDISO linear solver, which is a parallel linear solver using OpenMP.
To use multiple processors set the environment variable OMP_NUM_THREADS to the number of desired threads. You should set the number of threads to be equal or less than the number of processors on your system (Setting it higher may actually decrease performance). For example, on a system with four processors you can set the environment as follows. On Linux using the Bash shell, execute:
> export OMP_NUM_THREADS=4
Using the c-shell, execute:
> setenv OMP_NUM_THREADS 4
Or at a Windows command prompt:
> set OMP_NUM_THREADS=4
On Windows, you can add this environment variable as well from the Control Panel. On Win7, open the Control Panel (Start Control Panel). Open the System panel and click Change Settings. The System Properties dialog box should open up. Select the Advanced Tab and click the Environment Variables button. In the next dialog box, click the New... button under the User variables. Enter OMP_NUM_THREADS for the variable name and 4. Click OK on all open dialog boxes.
Ideally when the same model is run repeatedly, either on the same machine or on different machines, it should produce the same convergence statistics and results. However, in practice this is not always the case and sometimes the convergence stats and even the results can differ. In most cases, the discrepancies should be small, however in some cases, and especially in models that are prone to ill-conditioning (e.g. contact), the discrepancies may be more significant. The underlying reason is that it can not always be guaranteed that all calculations are executed in the exact same order and, due to numerical round-off, the results of these calculations will not always be the same. On a single machine, this can happen when the model is run using multiple processors. (However, running the same model on a machine with a single processor should always produce the exact same results.) This behavior can also be observed when running the same model on different machines, especially if these machines have different OS. Similarly, although FEBio aims to be backward compatible, running the same model with different versions of FEBio, may also show discrepancies for similar reasons. In this case, the discrepancies are likely caused by algorithmic optimizations or changes in compiler settings.