The week's pick
Reseach Article
A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 58 - Number 9 |
| Year of Publication: 2012 |
| Authors: G. Kumaravelan, R. Sivakumar |
10.5120/9310-3541
|
G. Kumaravelan, R. Sivakumar . A Learning Automata based Solution for Optimizing Dialogue Strategy in Spoken Dialogue System. International Journal of Computer Applications. 58, 9 ( November 2012), 20-27. DOI=10.5120/9310-3541
Abstract
Application of reinforcement learning methods in the development of dialogue strategies that support robust and efficient human–computer interaction using spoken language is a growing research area. In spoken dialogue system, Markov Decision Processes (MDPs) provide a formal framework for making dialogue management decisions for planning. This framework enables the system to learn the value of initiating an action from each possible state which in turn facilitates the maximization of the total reward. However, these MDP systems with large state-action spaces lead to intractable solution. The goal of this paper is, thus, to present a novel approximation method with sampling practice to compute an optimal solution to control dialogue strategy based on learning automata. Compared to other baseline reinforcement learning methods the proposed approach exhibits a better performance with regard to the learning speed, good exploration/exploitation in its update and robustness in the presence of uncertainty in the states obtained.
References
- McTear, M. 2004. Spoken Dialog Technology: Toward the Conversational User Interface, Springer-Verlag.
- Sutton, R. S. , and Barto, A. G. 1998. Reinforcement Learning an Introduction, MIT press, Cambridge, MA.
- Singh, S. , Litman, D. , and Walker, M. 2002. Optimizing dialogue management with reinforcement leaning: Experiments with the NJFun system. Journal of Artificial Intelligence, vol. 16, 105– 133. .
- Paek, T. , and Pieraccini, R. 2008. Automating spoken dialogue management design using machine learning: an industry perspective. Speech Communication, vol. 50, no. 8-9, 716–729.
- Chang, H. S. , Fu, M. , Hu, J. , and Marcus, S. I. 2007. Recursive learning automata approach to markov decision processes. IEEE Trans Automat Contr, Vol. 52, no. 7, 1349-1355.
- Young, S. , Gasic, M. , Keizer, S. , Mairesse, F. , Schatzmann, J. , Thomson, B. , and Yu, K. 2010. The hidden information state model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language, vol. 25, 150-174.
- McTear, M. 1998. Modelling spoken dialogues with state transition diagrams: experience with the CSLU toolkit. In Proc. ICSLP, 1223-1226.
- Goddeau, D. , Meng, H. , Polifroni, J. , Seneff, S. , and Busayapongchai, S. 1996. A form-based dialogue manager for spoken language applications. In Proc. ICSLP, 701–704.
- Rich, C. , and Sidner, C. 1998. Collagen: A collaboration manager for software interface agents. User Modeling and User-Adapted Interaction, vol. 8, no. 3/4, 315–350, 1998.
- Pietquin, O. , and Dutoit, T. 2006. A probabilistic framework for dialog simulation and optimal strategy learning. IEEE Trans Audio Speech Lang Process, vol. 14(2), 589–599.
- Frampton, M. , and Lemon, O. Recent research advances in Reinforcement Learning in Spoken Dialogue Systems. 2009. The Knowledge Engineering Review, vol. 24, no. 04, 375–408.
- Henderson, J. , Lemon, O. , and Georgila, K. 2008. Hybrid Reinforcement/Supervised Learning of Dialogue Policies from Fixed Data Sets. Computational Linguistics, vol. 34 (4), 487–512.
- Cuayáhuitl, H. , Renals, S. , Lemon, O. , and Shimodaira, H. 2010. Evaluation of a hierarchical reinforcement learning spoken dialogue system. Computer Speech & Language, vol. 24, no. 2, 395–429.
- Toney, D. , Moore, J. , and Lemon, O. 2006. Evolving optimal inspectable strategies for spoken dialogue systems. In Proc HLT, 173–176.
- Rieser, V. 2008. Bootstrapping Reinforcement Learning-based Dialogue Strategies fromWizard-of-Oz data. PhD dissertation, Saarbruecken Dissertations in Computational Linguistics and Language Technology, Vol. 28.
- Thathachar, M. A. L. , and Sastry, P. S. 2004. Networks of Learning Automata: Techniques for Online Stochastic Optimization, Kluwer.
- Oommen B. J. , and Misra, S. 2009. Cybernetics and learning automata. Springer Handbook of Automation, pp. 221-235.
- Chang, H. S. , Fu, M. , Hu, J. , and Marcus, S. I. 2007. An adaptive sampling algorithm for solving Markov decision processes. Operations Research, vol. 53(1), 126-139.
- Schatzmann, J. , Weilhammer, K. , Stuttle, M. N. , and Young, S. 2006. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The Knowledge Engineering Review, vol. 21, no. 02, 97–126.
- Walker, M. , Litman, D. , Kamm, C. , and Abella, A. 1998. PARADISE: a framework for evaluating spoken dialogue agents. In Proc. , Assoc. Comput. Linguist. (ACL), 271–280.
- Young, S. ATK: an application toolkit for HTK. Available:http://mi. eng. cam. ac. uk/research/dialogue/atk_home
- Yamagishi, J. , Zen, H. , Toda, T. , and Tokuda, K. 2007. Speaker-independent HMM-based speech synthesis system – HTS-2007 system for the Blizzard challenge 2007. In Proc The Blizzard Challenge, http://www. cstr. ed. ac. uk/projects/festival/
- Dethlefs, N. , Cuayahuitl, H. , Richter, K. , Andonova, E. , and Bateman, J. Evaluating task success in a dialogue system for indoor navigation. In Proc. 14th Workshop on the Semantics and Pragmatics of Dialogue, 143-146.
Index Terms
Keywords