Classification

Classification is the process of grouping images into predetermined categories. Classification can be used to discriminate images based on user preference, e.g. different cell types, phenotypes or tissue types etc.

Classification of different types of circulating tumor cells (CTCs) [1]

Labelling

To train a neural network for classification it is necessary to create images that are labelled accordingly.

Object detection tools

Different tools and functions in MIA exist to label images for classification.

To label an image select a tool and draw inside the image to label image objects.

Note

There is no save option as all changes during labelling are saved immediately. Use or ctrl + z to undo the last labelling action.

All labels are saved in a subfolder inside the currently active folder and have the same file name as the currently selected image, but with .npz as file extension.

Tools

As classification only needs a single label for each image, the tools are simpler compared to other applications and you do not have to explicitly click inside the image.

By pressing or F1 currently selected class is assigned to the image.

By pressing or F2 currently selected class is assigned to the image and the next image is selected.

Tip

The classification label is shown as a border around the image with the color of the corresponding class

Training

For details about neural network training see Training.

Neural Network architectures

For classification there is no extra architecture implemented, meaning that the architecture is identical to the backbone.

Tip

All architectures are implemented without fully connected layers, even if the original architecture had fully connected layers (like vgg-nets).

The following backbones are currently supported, untrained or with pretrained weights pre-trained on imagenet dataset [2]:

Model	Options	Ref.
DenseNet	densenet121, densenet169, densenet201	[3]
EfficientNet	efficientnetb0, efficientnetb1, efficientnetb2, efficientnetb3, efficientnetb4, efficientnetb5, efficientnetb6, efficientnetb7	[4]
Inception	inceptionv3, inceptionresnetv2	[5]
MobileNet	mobilenet, mobilenetv2	[6]
NasNet	nasnetlarge, nasnetmobile	[7]
ResNet	resnet18, resnet34, resnet50, resnet101, resnet152, resnet50v2, resnet101v2, resnet152v2	[8]
ResNeXt	resnext50, resnext101	[9]
SE-ResNet	seresnet18, seresnet34, seresnet50, seresnet101, seresnet152	[10]
SE-ResNeXt	seresnext50, seresnext101	[10]
SENet	senet154	[10]
VGG	vgg16, vgg19	[11]
Xception	xception	[12]

Tip

• Generally the numbers behind the backbone architecture gives either the number of convolutional layers (e.g. resnet18) or the model version (e.g. inceptionv3).

• When you have limited computing recources use a small network architecture or a network optimized for efficiency (e.g. mobilenetv2).

• From the supported network-backbones the nasnetlarge shows the highest performance on imagenet classification.

• From the supported network-backbones the mobilenet has the fastest processing time and fewest parameters.

Losses and Metrics

For classification several objective function have been tested for neural network optimization and directly impact the model training. Metrics are used to measure the performance of the trained model, but are independent of the optimization and the training process. The loss and metric functions can be set in Train Model → Settings.

Cross Entropy

The cross entropy loss is a widely used objective function used for classification. It is defined as:

\[L_{CE} = -\sum_{i=1}^{n}{p_i log(q_i)},\]

with \(p_i\) the true label and \(q_i\) the model prediction for the \(i_{th}\) class.

Focal Loss

The focal loss is an extension of the cross entropy, which improves performance for unbalanced datasets [13]. It is defined as follows:

\[L_{FL} = -\sum_{i=1}^{n}{(1-q_i)^\gamma p_i log(q_i)},\]

with \(\gamma\) as the focussing parameter. Default is set \(\gamma = 2\).

Kullback-Leibler Divergence

The Kullback-Leibler Divergence, sometimes referred as relative entropy, is defined as follows:

\[L_{KL} = -\sum_{i=1}^{n}{p_i (log(p_i)-log(q_i))}.\]

Accuracy

The pixel accuracy measures all images that are classified correctly:

\[L_{acc} = \frac{t_p + t_n}{t_p + t_n + f_p + f_n},\]

with \(t_p\) the true positives (\(p_i=1\) and \(q_i=1\)), \(t_n\) the true negatives (\(p_i=0\) and \(q_i=0\)), \(f_p\) the false positives (\(p_i=0\) and \(q_i=1\)) and \(f_n\) the false negatives (\(p_i=1\) and \(q_i=0\)). The accuracy is a misleading measure for imbalanced data.

Precision

The precision measures how many of all predicted positives, actually are positve:

\[L_{acc} = \frac{t_p}{t_p + f_p}.\]

Recall

The recall measures how many of the actual positives, are predicted as positve:

\[L_{acc} = \frac{t_p}{t_p + f_n}.\]

F1-Score

The F1-score is a balanced measure of precision and recall

\[L_{acc} = 2 \frac{precision * recall}{precision + recall}.\]

---

[1]

Dong, J., Zhu, D., Tang, X., Lu, D., Qiu, X., Li, B., Lin, D., Li, L., Liu, J. and Zhou, Q., 2019. Circulating tumor cells in pulmonary vein and peripheral arterial provide a metric for PD-L1 diagnosis and prognosis of patients with non-small cell lung cancer. Plos one, 14(7), p.e0220306.

[2]

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K. and Fei-Fei, L., 2009, June. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248-255). Ieee.

[3]

Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q., 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4700-4708).

[4]

Tan, M. and Le, Q., 2019, May. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning (pp. 6105-6114). PMLR.

[5]

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z., 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826).

[6]

Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. and Adam, H., 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

[7]

Zoph, B., Vasudevan, V., Shlens, J. and Le, Q.V., 2018. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8697-8710).

[8]

He, K., Zhang, X., Ren, S. and Sun, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

[9]

Xie, S., Girshick, R., Dollár, P., Tu, Z. and He, K., 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1492-1500).

[10] (1,2,3)

Hu, J., Shen, L. and Sun, G., 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141).

[11]

Simonyan, K. and Zisserman, A., 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

[12]

Chollet, F., 2017. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258).

[13]

Lin, T.Y., Goyal, P., Girshick, R., He, K. and Dollár, P., 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (pp. 2980-2988).