Checkpoints and restarts

For very long simulations, it is good to set checkpoints. Checkpoints are files that contain all the simulation data at one given iteration, so that the simulation can be later restarted from this iteration.

Checkpoints are useful when there is a risk that the simulation crashes before the end (e.g. because of the finite walltime on HPC clusters). In this case, thanks to checkpoints, the simulation can be restarted without having to run it again from the beginning.

Setting checkpoints

fbpic.openpmd_diag.set_periodic_checkpoint(sim, period, checkpoint_dir='./checkpoints')[source]

Set up periodic checkpoints of the simulation

The checkpoints are saved in openPMD format, in the specified directory, with one subdirectory per process. The E and B fields and particle information of each processor is saved.

NB: Checkpoints are registered in the list checkpoints of the Simulation object sim, and written at the end of the PIC loop (whereas regular diagnostics are written at the beginning of the PIC loop).

Parameters:

  • sim (a Simulation object) – The simulation that is to be saved in checkpoints

  • period (integer) – The number of PIC iteration between each checkpoint.

  • checkpoint_dir (string, optional) – The path to the directory in which the checkpoints are stored (When running a simulation with several MPI ranks, use the same path for all ranks.)

Restarting a simulation

fbpic.openpmd_diag.restart_from_checkpoint(sim, iteration=None, checkpoint_dir='./checkpoints')[source]

Fills the Simulation object sim with data saved in a checkpoint.

More precisely, the following data from sim is overwritten:

  • Current time and iteration number of the simulation

  • Position of the boundaries of the simulation box

  • Values of the field arrays

  • Size and values of the particle arrays

Any other information (e.g. diagnostics of the simulation, presence of a moving window, presence of a laser antenna, etc.) need to be set by hand.

For this reason, a successful restart will often require to modify the original input script that produced the checkpoint, rather than to start a new input script from scratch.

NB: This function should always be called before the initialization of the moving window, since the moving window infers the position of particle injection from the existing particle data.

Parameters:

  • sim (a Simulation object) – The Simulation object into which the checkpoint should be loaded

  • iteration (integer, optional) – The iteration number of the checkpoint from which to restart If None, the latest checkpoint available will be used.

  • checkpoint_dir (string, optional) – The path to the directory that contains the checkpoints to be loaded. (When running a simulation with several MPI ranks, use the same path for all ranks.)