Add models
You can extend the functionality of the visual editor by adding another Decthings model inside your graph. Any Decthings model can be used within the visual editor - even if its running a different programming language. This can be a powerful tool for:
- Using advanced models created by others, and fine-tuning them to fit them to your dataset
- Creating a model that combines the functionality of multiple other models into one
- Adding functionality that you find missing in the visual editor
When a Decthings model is used as a layer in your graph, the model will be evaluated when it is needed in the computation. Evaluation means that some input data is given to the model, and it gives back some output data. The input data to provide to the model is specified as inputs to the node in the graph editor, and the model outputs can be provided to other nodes in the graph. You can combine multiple Decthings models and built-in nodes in a single graph, thus allowing you to create a larger, more complex model, from smaller building blocks.
Add a Decthings model to your visual editor graph
In the visual editor you can open up a model browser by right-clicking in the background of the editor, selecting "Models" and clicking "Browse models". There, search for the model to add and click it to proceed.
You are then prompted to select a snapshot to use. A snapshot is a fixed, non-changing version of a model. By using a snapshot, you can be sure that the behaviour of the model you create does not change. However, if the snapshot is removed from the original model, or if the original model itself is removed, the model that you create with the visual editor will no longer be able to be executed. If your model does not have a snapshot, you must go to the model page and create a snapshot before you can continue.
Fine-tuning (training) an added model
In machine learning, fine-tuning is the process of taking a pre-trained model and re-training it to fit your specific use case. This can significantly reduce training time, increase training accuracy, and reduce the number of training samples required.
Not all models that you add are trainable. The models that can be fine-tuned are marked with the labels "Trainable with TensorFlow", "Trainable with PyTorch" or "Trainable with TensorFlow and PyTorch" in the model browser inside the visual editor. In addition to being trainable, these layers are usually configurable. When you click on the model in your graph, you may see some options on the properties panel in the visual editor.
To train the added model, you need an optimizer. Learn more about how training works in the visual editor here.
Create a custom node for use in the visual editor
Adding another model to your graph can be used to add custom functionality to the visual editor. In many cases, the easiest way to get started with this is to use a built-in node as a starting point - you can then modify the behaviour of that node to fit your needs.
On the create model page, select "Visual editor", and then select "Custom node". Then, you will be asked to select a built-in node as a starting point. When you click create, a Python model will be created. This model will contain the code for the built-in node that you selected as a starting point. If you now create a snapshot of this model, and then go to the visual editor and add this model to your graph, the appearance and the behaviour will be identical to the built-in layer. In fact, they are both using the same code under the hood.
You can now go back to your Python model and edit the code. You can edit both the appearance of the Node (how it renders in the browser), the configuration options on the proprties panel, and the Python code which is executed when the model is compiled, trained and evaluated.
Make an existing Decthings model trainable
As described above, only some Decthings models can be fine-tuned (trained) in the visual editor. To make a model trainable in the visual editor, you need to add a few files to your Decthings model. These files will be read by the visual editor, thereby allowing the model to be trained.
The files that should be added are:
- /visual-editor.config.js - This JavaScript file is executed in the browser to determine properties and behvaiour of your node
- /visual-editor.display.jsx - This JavaScript (React) file is executed in the browser to determine the appearance of the node and the properties panel that shows when you click it. It uses React, which makes it easy to write HTML-like code that is interactive.
- /visual-editor-tensorflow.py - This Python file is executed when the model is compiled, trained or evaluated. It performs the actual computation.
