Overview

This project aims to identify if self-supervised learning is a viable option when training on satellite imagery with limited access to labeled data. Our baseline models are trained on various neural network architectures and fractions of labeled data to quantify the performance impact as our fraction of labeled data changes. We will use the baseline models' results to assess the performance of our self-supervised models with the same labeling constraints.

Before diving into the models, it is essential to understand the methodology for applying the “irrigated” label. Using the Big Earth Net labels, we identified two different methods to determine irrigation state. We will refer to these methods as regular labels or expanded labels (described below) in our experiments.

• Regular Labels: Regular labels take the “permanently irrigated” label from Big Earth Net data and use it as the ground truth for classification. All images that do not include this label are part of the negative class.
• Expanded Labels: When analyzing the Big Earth Net labels, some of them corresponded to croplands that would imply irrigation. We used these additional labels to broaden what the definition of irrigated is to include the following: Fruit trees and berry plantations, rice fields, vineyards, olive groves, and permanently irrigated.

In addition to the different labeling methods, we also trained the models in two ways; full training or ImageNet pre-training. The full training experiments train a model from scratch using various fractions of labeled data. The ImageNet pre-training starts off the training with pre-trained weights from an ImageNet model. It is important to note that models trained with ImageNet weights are only trained on RGB channels, while training from scratch utilizes all available channels.

Baseline models were trained on the following architectures.

• InceptionV3
• ResNet101V2
• ResNet152
• ResNet50
• Xception

Full training

Models were trained from scratch on each architecture and labeling methodology.

Regular Labels

Supervised model with various splits of labeled data. For the first set of experiments we used only the “permanently irrigated” label for classification, which is only a small fraction of the dataset at roughly 2.3%. We explore models commonly trained with the ImageNet dataset, and several ResNet models of different sizes, on ever diminishing splits of our labeled dataset.

Interestingly, as shown more readily in Figure 1 there is a large variance in model performance for ResNet152 and other models. This variance was not consistent and is likely explained by randomness in models with very small data sets. Sometimes the models would perform much better than others which may just be chalked up to luck.

Expanded Labels

Supervised model with various splits of labeled data. In Addition to the “permanently irrigated” we also include Vineyards, Rice fields, fruit orchards and olive groves, which now represents approximately 6.24% of the dataset.

As stated above, it appears that randomness in the training has produced models with a large variance in accuracy for the 1 percent data split. Another interesting observation with using the expanded labels is that model performance improves quickly as opposed the more expected logarithmic curve exhibited when training with regular labels.

ImageNet Pretraining

Models were trained on each architecture and labeling methodology using weights from ImageNet as a starting point. When using ImageNet weights the models were only trained on RGB channels.

Regular Labels

Supervised model with various splits of labeled data. Used only the “permanently irrigated” label for classification but trained only on the RGB channels using ImageNet weights.

The ImageNet weights seem to produce more consistency in the models across all data splits but does not achieve the same accuracy as the full models as the fraction of labeled data increased. The Xception architecture was the best performing across all data splits.

Expanded Labels

Supervised model with various splits of labeled data. Used expanded labels (permanently irrigated, Vineyards, Rice fields, fruit orchards and olive groves) for classification but trained only on the RGB channels using ImageNet weights.

The performance with this experiment is very similar to the regular labels with the Xception model appearing to be the best.