Datasets
When training our models we often need large amounts of data. In Decthings, datasets allow you to upload data, either in the browser or via the API, and then access that data within a model.
What is a dataset?
Machine learning (ML) likely the most common way to create artificial intelligence, in which a model is trained by looking at input data. When looking at the input data , the model will with the help of some clever mathematics automatically adjust its inner parameters so that the model output comes closer to the expected answer. Then, when the model is applied to real-world data that it has not been trained on before, it can use the information it learned while training to be able to output the correct value.
In Decthings, a dataset is a long list of data points. The data points can have different data types, such as number, text or image. A single data point can also contain multiple values. For example, in a dataset of images we may want to store the image as well as a corresponding label, such as "cat" or "dog" in a dataset of animals.
Datasets are billed by the used storage space. For more details, see pricing.
Quick start - create a dataset
To create a dataset, go to the datasets pageand click "Create". You will then be presented with a few options. First, give the dataset a name, such as "Animals".
Next, let's configure the dataset element data types. By default, the data type is set to "Image", but because we want to also have a label we will change this to "Dictionary". A dictionary is a key-value type which allows you to have multiple entries. Give the first entry a name such as "image" and set the data type to image. Then, click "Add entry" to add a second entry to the dictionary. Give this entry a name such as "label" and set the data type to text. Leave all the shapes empty.
You can leave the "Element shape" empty. The shape specifies the number of elements per data points. In this case we just use a single element, which is represented by an empty shape.
Now click "Create" and an empty dataset with the selected configuration will be created. To add some data, go to the "Add data" tab and upload an image file. Enter a value for the "label" field, and then click "Add". You now have one element in your dataset! You can use this dataset within a model when starting a training session, but only if the model expects an image and a label as input.
To learn more about datasets, check out the guides upload CSV data, automatically collect data and create a dataset using the API.
Datasets are collections of data entries which later can be used as input data to models. In machine learning, training makes use of large amounts of real world data to improve the accuracy and performance of the model.
Datasets can contain images, text, numbers, or any other data you can imagine.
There are a few datasets already available by us at Decthings. Using these is the quickest way to get started, but for more specific applications you can also collect your own data and upload it. You can upload data from your local computer, or write a program or script which automatiaclly uploads data. This is useful if you have a system which continuously records information, such as a camera connected to a computer. In this case, you can automatically upload these images using the API, and then create your AI which can learn to recognize patterns from these images.
Recommended reading: Guide - Create a dataset using the API
Pricing
Datasets are priced per gibibyte-month (one gibibyte = 1 GiB = 1024^3 bytes). If a dataset contains one gibibyte of data for half a month and three gigabytes for the next half of the month, you would be billed for two gibibyte-months (1 * 0.5 + 3 * 0.5).
See pricing for more details.
