Near-optimal reinforcement learning framework for energy-aware sensor communications
- 4 April 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 23 (4), 788-797
- https://doi.org/10.1109/jsac.2005.843547
Abstract
We consider the problem of average throughput maximization per total consumed energy in packetized sensor communications. Our study results in a near-optimal transmission strategy that chooses the optimal modulation level and transmit power while adapting to the incoming traffic rate, buffer condition, and the channel condition. We investigate the point-to-point and multinode communication scenarios. Many solutions of the previous works require the state transition probability, which may be hard to obtain in a practical situation. Therefore, we are motivated to propose and utilize a class of learning algorithms [called reinforcement learning (RL)] to obtain the near-optimal policy in point-to-point communication and a good transmission strategy in multinode scenario. For comparison purpose, we develop the stochastic models to obtain the optimal strategy in the point-to-point communication. We show that the learned policy is close to the optimal policy. We further extend the algorithm to solve the optimization problem in a multinode scenario by independent learning. We compare the learned policy to a simple policy, where the agent chooses the highest possible modulation and selects the transmit power that achieves a predefined signal-to-interference ratio (SIR) given one particular modulation. The proposed learning algorithm achieves more than twice the throughput per energy compared with the simple policy, particularly, in high packet arrival regime. Beside the good performance, the RL algorithm results in a simple, systematic, self-organized, and distributed way to decide the transmission strategy.Keywords
This publication has 10 references indexed in Scilit:
- Power controlled multiple access (PCMA) in wireless communication networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Energy concerns in wireless networksIEEE Wireless Communications, 2002
- Design challenges for energy-constrained ad hoc wireless networksIEEE Wireless Communications, 2002
- Low-energy wireless communication network designIEEE Wireless Communications, 2002
- Jointly optimized bit-rate/delay control policy for wireless packet networks with fading channelsIEEE Transactions on Communications, 2002
- Efficient power control via pricing in wireless data networksIEEE Transactions on Communications, 2002
- Power control for wireless dataIEEE Wireless Communications, 2000
- Finite-state Markov model for Rayleigh fading channelsIEEE Transactions on Communications, 1999
- Introduction to Discrete Event SystemsThe Kluwer International Series on Discrete Event Dynamic Systems, 1999
- Finite-state Markov channel-a useful model for radio communication channelsIEEE Transactions on Vehicular Technology, 1995