A-DDPG: Attention Mechanism-based Deep Reinforcement Learning for NFV

Nan He, Song Yang, Fan Li, Stojan Trajanovski, Fernando A. Kuipers, Xiaoming Fu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

17 Citations (Scopus)

Abstract

The efficacy of Network Function Virtualization (NFV) depends critically on (1) where the virtual network functions (VNFs) are placed and (2) how the traffic is routed. Unfortunately, these aspects are not easily optimized, especially under time-varying network states with different quality of service (QoS) requirements. Given the importance of NFV, many approaches have been proposed to solve the VNF placement and traffic routing problem. However, those prior approaches mainly assume that the state of the network is static and known, disregarding real-time network variations. To bridge that gap, in this paper, we formulate the VNF placement and traffic routing problem as a Markov Decision Process model to capture the dynamic network state transitions. In order to jointly minimize the delay and cost of NFV providers and maximize the revenue, we devise a customized Deep Reinforcement Learning (DRL) algorithm, called A-DDPG, for VNF placement and traffic routing in a real-time network. A-DDPG uses the attention mechanism to ascertain smooth network behavior within the general framework of network utility maximization (NUM). The simulation results show that A-DDPG outperforms the state-of-the-art in terms of network utility, delay, and cost.

Original languageEnglish
Title of host publication2021 IEEE/ACM 29th International Symposium on Quality of Service, IWQOS 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665414944
DOIs
Publication statusPublished - 25 Jun 2021
Event29th IEEE/ACM International Symposium on Quality of Service, IWQOS 2021 - Virtual, Tokyo, Japan
Duration: 25 Jun 202128 Jun 2021

Publication series

Name2021 IEEE/ACM 29th International Symposium on Quality of Service, IWQOS 2021

Conference

Conference29th IEEE/ACM International Symposium on Quality of Service, IWQOS 2021
Country/TerritoryJapan
CityVirtual, Tokyo
Period25/06/2128/06/21

Keywords

  • Network function virtualization
  • deep reinforcement learning
  • placement
  • routing

Fingerprint

Dive into the research topics of 'A-DDPG: Attention Mechanism-based Deep Reinforcement Learning for NFV'. Together they form a unique fingerprint.

Cite this