Learning to Generate Preferences for Multiobjective Deep Learning

Multiobjective optimization (MOO) is important for deep learning applications with multiple conflicting objectives. Pareto front learning (PFL) methods learn a single model conditioned on the preference of objectives and can be applied to any preference at inference time. However, existing PFL methods use predefined strategies (e.g., uniform sampling) to generate preferences, which could result in unevenly spaced solutions since the shape of Pareto front is largely ignored. In this article, we propose a lightweight and model-agnostic method to train a preference generator for a given PFL model, which learns to generate proper preferences from uniformly sampled ones, such that the resulting solutions are evenly spaced on the Pareto front. Compared to previous works, our method enables a more rational allocation of preferences, which can either be utilized to enhance a pretrained PFL model or be seamlessly integrated into the PFL training process to improve efficiency. We apply our method to state-of-the-art PFL methods with various backbones (e.g., multilayer perceptron, convolutional neural network, transformer) and validate the significance of preference generation across various tasks, from multitask supervised learning to multiobjective reinforcement learning-based neural combinatorial optimization. Experimental results show that our method improves the backbone algorithm in most settings, showing its effectiveness and general applicability.