Fast Adaptive Hinging Hyperplanes

This paper proposes a fast algorithm for the training of adaptive hinging hyperplanes (AHH), which is a popular and effective continuous piecewise affine (CPWA) model consisting of a linear combination of basis functions. The original AHH incrementally generates new basis functions by simply traversing all the existing basis functions in each dimension with the pre-given knots. Meanwhile, it also incorporates a backward procedure to delete redundant basis functions, which avoids over-fitting. In this paper, we accelerate the procedure of AHH in generating new basis functions, and the backward deletion is replaced with Lasso regularization, which is robust, requires less computation, and manages to prevent over-fitting. Besides, the selection of the splitting knots based on training data is also discussed. Numerical experiments show that the proposed algorithm significantly improves the efficiency of the existing AHH algorithm even with higher accuracy and it also enhances robustness in the given benchmark problems.