Hidden state and reinforcement learning with instance-based state identification
- 1 June 1996
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- Vol. 26 (3), 464-473
- https://doi.org/10.1109/3477.499796
Abstract
Real robots with real sensors are not omniscient, When a robot's next course of action depends on information that is hidden from the sensors because of problems such as occlusion, restricted range, bounded field of view and limited attention, we say the robot suffers from the hidden state problem, State identification techniques use history information to uncover hidden state, Some previous approaches to encoding history include: finite state machines [12], [28], recurrent neural networks [25] and genetic programming with indexed memory [49]. A chief disadvantage of all these techniques is their long training time, This paper presents instance-based state identification, a new approach to reinforcement learning with state identification that learns with much fewer training steps. Noting that learning with history and learning in continuous spaces both share the property that they begin without knowing the granularity of the state space, the approach applies instance-based (or ''memory-based'') learning to history sequences-instead of recording instances in a continuous geometrical space, we record instances in action-percept-reward sequence space. The first implementation of this approach, called Nearest Sequence Memory, learns with an order of magnitude fewer steps than several previous approaches.Keywords
This publication has 13 references indexed in Scilit:
- On the Characteristics of Sequential Decision Problems and Their Impact on Evolutionary Computation and Reinforcement LearningLecture Notes in Computer Science, 2010
- Alecsys and the AutonoMouse: Learning to control a real robot by distributed classifier systemsMachine Learning, 1995
- Robot shaping: developing autonomous agents through learningArtificial Intelligence, 1994
- Training Agents to Perform Sequential BehaviorAdaptive Behavior, 1994
- Overcoming Incomplete Perception with Utile Distinction MemoryPublished by Elsevier ,1993
- Principles of animate visionCVGIP: Image Understanding, 1992
- Using Transitional Proximity for Faster Reinforcement LearningPublished by Elsevier ,1992
- Self-improvement Based On Reinforcement Learning, Planning and TeachingPublished by Elsevier ,1991
- Outline for a theory of intelligenceIEEE Transactions on Systems, Man, and Cybernetics, 1991
- Classifier systems and genetic algorithmsArtificial Intelligence, 1989