Representation of neural networks through their multi-linearization

The studies on interpretability of neural networks have been playing an important role in understanding the knowledge developed through their learning and promoting the use of neurocomputing in practical problems. The rule-based setting in which neural networks are interpreted provides a convenient way of expressing knowledge in a transparent and modular manner and at a desired level of granularity (specificity). In this study, we formulate a certain engineering-based style of interpretation in which a given neural network is represented as a collection of local linear models where such models are developed around a collection of linearization nodes. The notion of multi-linearization of neural networks captures the essence of the proposed interpretation. We formulate the problem as an optimization of (i) a collection of linearization nodes around which individual linear models are formed and (ii) aggregation of the individual linearizations, where the linearization fields are subject to optimization. Given the non-differentiable character of the problem, we consider the use of population-based optimization of Particle Swarm Optimization (PSO). Numeric experiments are provided to illustrate the main aspects of the multi-linearization of neural networks.