Persistent launchers
Persistent launchers allow you to keep your model running between operations, so that it is always ready when an operation is started. This can lead to significantly improved performance.
For a temporary launcher (that is, not using a persistent launcher), several steps need to be performed before the actual evaluation is started. Firstly, a virtual machine (launcher) is started. Then, the model is initialized. Depending on the model, the initialization can take some time. During this step, the files are loaded, the model code is executed and the model state is loaded. For a neural network, this means loading the code and frameworks such as TensorFlow or PyTorch, and then loading the weights and biases and setting up a neural network which uses those parameters.
This entire process can be skipped if we always keep the model initialized and running.
Pricing
Persistent launchers are always billed for, even if it has no running operations. The hourly cost is the same as for temporary launchers.
In many cases, running a persistent launcher is more expensive but improves performance. However, since the price for launchers is reduced after the first hour, for a persistent launcher that you keep alive for a long period of time you will therefore receive a substantial cost reduction. If you run many evaluations, this cost reduction might even be signficant enough to make it overall cheaper to use a persistent launcher than to create a new temporary launcher for every evaluation.
See pricing for more details.
