A Generalized Convolutional Neural Network for Small Dataset Classification

TL;DR — A generalized version of ConvNets.

Abstract

We propose a novel variant of neural networks, Generalized Convolutional Neural Networks, GConvNets, characterized by structured neurons. In contrast to conventional neural networks such as ConvNets, which predominantly employ 'scalar' neurons, GConvNets utilize structured 'tensor' neurons. In other words, we generalize ConvNets by substituting each scalar neuron in ConvNets with a tensor neuron in GConvNets, while preserving the weight-sharing mechanism. These structured neurons manifest as tensors with adaptable shapes and dimensions across different layers. To ensure their practical applicability, we have developed a mechanism that enables seamless handling of hybrid structured tensor neurons as they transition from one layer to the next. We conducted a comparative analysis between GConvNets and the currently popular ConvNets, which include ResNets, MobileNets, EfficientNets, RegNets, among others, using datasets such as CIFAR10, CIFAR100, and Tiny ImageNet. The experimental results demonstrate that GConvNets exhibit superior efficiency in terms of parameter usage.