Structure-Aware Stochastic Control for Transmission Scheduling

In this report, we consider the problem of real-time transmission scheduling over time-varying channels. We first formulate the transmission scheduling problem as a Markov decision process (MDP) and systematically unravel the structural properties (e.g. concavity in the state-value function and monotonicity in the optimal scheduling policy) exhibited by the optimal solutions. We then propose an online learning algorithm which preserves these structural properties and achieves -optimal solutions for an arbitrarily small . The advantages of the proposed online method are that: (i) it does not require a priori knowledge of the traffic arrival and channel statistics and (ii) it adaptively approximates the state-value functions using piece-wise linear functions and has low storage and computation complexity. We also extend the proposed low-complexity online learning solution to the prioritized data transmission. The simulation results demonstrate that the proposed method achieves signifi...

• Type: Journal
• Year: 2010
• Where: CORR
• Authors: Fangwen Fu, Mihaela van der Schaar
• Keywords: CORR 2010, Education, Optimal Scheduling Policy, State-value Functions, Structural Properties