AdaBoostNet: An Efficient Hierarchical Neural Network for Image Classification

Hierarchical Neural Networks [e.g., deep neural networks (DNNs)] have recently gained increasing attention for image classification task. Most of the previous works devote to stack multiple levels of feature learning modules with the hope that higher level modules can represent more abstract semantics of the image data. However, training of such hierarchical network is typically cast as a time-consuming non-convex optimization problem, and its effectiveness for image feature representation critically depends on expertise in parameter tuning with various ad hoc tricks. To address these issues, we advocate a biologically-inspired hierarchical neural network. One key philosophy is that the higher-level modules in the network should correct misclassification induced by the lower ones. Given this idea, a sequential stacking strategy of basic feature learning modules is presented. In practical, an efficient layer wise learning and decision aggregation method is applied to boost the hierarchical network, which is different from naive layers cascading and end-to-end finetuning, and we term it as AdaboostNet. Moreover, the basic feature learning module is set as extreme learning machine (ELM), an effective and cheaply-optimized model. Experiments are conducted on benchmark datasets to evaluate our claims. It shows that the proposed network achieves comparable or better image classification accuracy and training efficiency than traditional DNNs and hierarchical ELMs.