FusionShot: Boosting Few Shot Learners with Focal-Diversity Optimized Ensemble Method

Abstract

Designing optimal few-shot learners is challenging. First, it is hard to train a few-shot model that can deliver the best generalization performance on all benchmarks compared to existing state-of-the-art (SOTA) methods. Second, unlike traditional deep neural networks (e.g., CNN, auto-encoder), few-shot learners utilize the metric space distance-based loss function to optimize the deep embedding learning on complex or multi-modal data. Both the choice of latent similarity computation methods and the choice of DNN embedding algorithms for latent feature extraction will impact the generalization performance of few-shot learners. This paper presents {\sc FusionShot}, a focal diversity optimized few-shot ensemble learning framework with three original contributions. First, we revisit the few-shot learning architectures to analyze why some few-shot learners perform well whereas other SOTA few-shot models fail miserably. Second, we explore and compare two alternative fusion channels to ensemble multiple few-shot learners: (i) the fusion of various latent distance methods, and (ii) the fusion of multiple DNN embedding algorithms that learn/extract latent features differently. Finally, we introduce a focal-diversity optimized few-shot ensemble learning framework for further boosting the performance of few-shot ensemble learning. Extensive experiments on representative few-shot benchmarks (mini-Imagenet and CUB) show that our {\sc FusionShot} can select the best performing ensembles from a pool of base few-shot models, which outperform the representative SOTA models, on novel tasks (unknown at training), even when a majority of the base models fails. For reproducibility purposes, trained models, results, and code are made available at \url{https://anonymous.4open.science/r/fusionshot-0A44/}.