Tadahiro Taniguchi, Associate professor College of information tech.&sci., Ritsumeikan univ. Shonan meeting 2013/11/11 ‐ 14 Shonan village
Tadahiro Taniguchi @tanichu Associate professor, Emergent system laboratory College of information science and technology, Ritsumeikan University, Japan 2006, PhD Eng. , Kyoto university 2008‐ Assistant professor 2010‐ Associate professor Machine Cognitive Decentralized Human learning robotics autonomous communication system Symbol Emergence in Robotics
Computational understandings of mental development ‐ From behavioral learning to language acquisition ‐ A human child acquires many physical skills, concepts and knowledge including language through physical and social interaction with his/her environment. How can we become able to communicate using symbolic system? We’d like to obtain computational under standings of mental development and symbol emergence. Constructive approach towards human intelligence enabling semiotic communication 3
Symbol grounding problem How can a robot “ground” his/her symbol to the real physical world? [Harnad ‘90] The robot has to give some meaning to a symbol in its symbolic system designed by a human designer through sensor‐motor interaction with its environment. Arbitrary nature of symbol (in semiotics) SGP is Arbitrariness of labeling/naming missing Arbitrariness of categorization/segmentation SGP implicitly assumes that human “arbitrarily evolved” symbolic system is a “true” symbolic system. Should a symbolic system for a robot be same as human’s ?
Understanding that symbolic system is an emergent property of human cognitive and social system “Worlds of robots” “Worlds of animals” (Uexküll 1934) Umwelt (self‐centered world) Animals can receive information only from their sensor‐motor system. A human has to obtain various behaviors, concepts and language on the basis of experiences in his/her Ernst Mach’s famous picture umwelt (closed cognitive system). Human Symbolic system should be understand on the basis of human embodiment (sensor‐motor system). Concepts have to be formed on the basis of sensor‐motor information "Early Scheme for a circular in a bottom‐up way Feedback Circle by Uxkull
Social constraints in semiotic communication Concept formation is not “enough” for semiotic communication. We have to share a symbolic system involving syntax, semantics and pragmatics lexicon and mutual belief bad interpretation Shared symbolic system bad naming Sheep the coming, aren’t they!!! bad syntax (true) A thief is coming!! It’s impossible to help her
Social constraints in semiotic communication Concept formation is not “enough” for semiotic communication. We have to share a symbolic system involving syntax, semantics and lexicon pragmatics and mutual belief interpretation situation recognition constraint constraint Shared symbolic system constraint ! (;O;) decision making A thief is coming!! constraint speech generation
Before introducing “symbol emergent system”, please remind.... Emergent System Emergent property Through local interaction between elements of a system, global (or macro‐level) order or pattern emerges. The macro‐level order becomes constraints of micro‐level dynamics, and governs the micro‐level interaction. Such bidirectional process makes the system have novel function, morphology and/or behavior. micro‐macro loop emergence constraint
Symbol emergent system Symbolic System Emergence Shared lexicon, syntax, pragmatics, semantics, and mutual belief Concept formation through interaction with environment Physical interaction Semiotic / Social interaction 谷口忠大. 「コミュニケーションするロボットは創れるか‐記号創発システムへの構成論的アプローチ」(2010). Tadahiro Taniguchi “Constructive approach toward symbol emergence system” (in Japanese)
Symbol emergence in robotics (SER) Constructive approach toward symbol emergent systems “Constructive” viewpoint to an intelligent system which uses symbolic system. 1. Constructive approach Understanding human intelligence by constructing robots obtaining symbolic system 2. Constructivism Understanding human intelligence which construct his/her subjective world (Umwelt). Understanding by developing “models” (c) Nagai lab.
Topics in SER multimodal communication learning interaction strategy Prof. Nagai learning motor skills emergence of concept language acquisition and segmentation of communication formation and mental development time‐series infomation
Segmentation of sensor‐motor time‐series information Dynamics in sensor‐motor time‐ series information changes depending on its environmental situations. An autonomous system should differentiate such situations and obtain representations. How do humans and how can the robots segment the time‐series data? What is the criteria of segmentation? What is an adequate computational algorithm for the segmentation?
Modular learning architecture for segmenting sensor‐motor time series MOSAIC [Kawato et al.] Mixture of experts [Jacobs et al.] HAMMER [Yiannis et al.] Dual schemata model [Taniguchi et al.] “Segment” corresponds to a linear system local information a short‐term event How can we grasp long‐term context? http://www.cns.atr.jp/cnb/HarunoG/ harunoG.ja.html T. Taniguchi, T. Sawaragi, "Incremental acquisition of multiple nonlinear forward models based on differentiation process of schema model“, Neural Networks, Vol.21 (1), pp.13‐27 .(2008)
Hierarchical mixture of RNN experts [Tani and Norfi 1999] 階層的な分節構造の Representation of a room and a part of 自己組織的な獲得 a room and was encoded in RNN in a hierarchical manner. Dynamical S ys Sy te stem m A ppr A oach ppr Symbols are implicitly obtained. It is difficult to understand how the system formed concepts.
Double articulation structure in semiotic data Semiotic time‐series data often has double articulation Speech signal is a continuous and high‐dimensional time‐series. Spoken sentence is considered as a sequence of phoneme People don’t give a meaning to a phoneme, but give a meaning to a word which is a sequence of phoneme. semantic How much is this? (meaningful) [h a u ] [m ʌ́ tʃ] [ i z ] [ð í s] h a u m ʌ́ tʃ I z ð í s meaningless unsegmented
Segments and chunks Conditions iHMM Unknown number of [Fox ‘07] words and letters Unknown emission distributions Inference Approximate Inference Procedure of Double Articulation Analyzer [Taniguchi ‘11] Nonparametric Bayesian approach Tadahiro Taniguchi, Shogo Nagasaka, Double Articulation Analyzer for Unsegmented Human Motion using Pitman‐ Yor Language model and Infinite Hidden Markov Model, 2011 IEEE/SICE SII.(2011)
iHMM [Fox ‘07] sticky HDP‐HMM [Fox ‘07] An infinite HMM is an HMM which can estimate the number of hidden state flexibly (potentially infinite)[Beal ‘02] [Teh ‘06] Sticky HDP‐HMM is a generative γ β model for iHMM with a stickiness parameter [Fox ‘07]. κ Fox et al. developed fast inference α πk algorithm with weak‐limit z1 z2 z3 zT approximation. λ θk ∞ Gaussian emission distribution y1 y2 y3 yT E.B. Fox, E.B. Sudderth, M.I. Jordan, A.S. Willsky, "A Sticky HDP‐HMM with Application to Speaker Diarization," Annals of Applied Statistics, June 2011. (First appeared as MIT LIDS Technical Report P‐2777, November 2007.)
NPYLM [Moachihashi ‘09] Unsupervised morphological analysis Thisisanapple ‐> This is an apple Morphological analysis わたしはたなかです． ‐> わたし は たなか です To segment sentences into words(morpheme). This usually requires dictionary (preexisting knowledge of language model). Unsupervised morphological analysis It does not assume preexisting dictionary. Mochihashi proposed an unsupervised morphological analysis method based on Nested Pitman‐Yor language model (NPYLM)[Mochihashi ‘09]. http://chasen.org/~daiti‐m/paper/nl190segment‐slides.pdf
NPYLM [Mochihashi ‘09] (Nested Pitman‐Yor Language Model) Mochihashi developed NPYLM for unsupervised morphological analysis. NPYLM has word n‐gram model and letter ngram model, hierarchically. Each adopts hierarchical Pitman‐Yor language model as a language model (smoothing method). Bayesian nonparametric model Efficient blocked Gibbs sampler Daichi Mochihashi, Takeshi Yamada, Naonori Ueda."Bayesian Unsupervised Word Segmentation with Nested Pitman‐Yor Language Modeling". ACL‐IJCNLP 2009, pp.100‐108, 2009.
Application of Double Articulation Analyzer Human motion data • Imitation learning [Taniguchi ‘11] • Motion segmentation [Taniguchi’11] Driving behavior • Extracting driving chunk [Nagasaka ‘12] • Detecting intentional changing points [Takenaka ‘12] • Prediction [Taniguchi ‘12] • Video summarization [Takenaka ‘12] • For topic modeling  collaborative work DAA with DENSO co. Auditory data* • Language acquisition [Araki ‘12] bottle !! collaborative work *Only NPYLM was used. with Nagai lab.
Contextual Scene Segmentation of Driving Behavior based on Double Articulation Analyzer [Takenaka ‘12] DAA could extract changing points of driving context recognized by human Kazuhito Takenaka, Takashi Bando, Shogo Nagasaka, Tadahiro Taniguchi, Kentarou Hitomi, Contextual Scene Segmentation of Driving Behavior based on Double Articulation Analyzer, IEEE/RSJ International Conference on Intelligent Robots and Systems 2012 (IROS 2012), 4847‐4852 .(2012)
Semiotic Prediction of Driving Behavior using Unsupervised Double Articulation Analyzer [Taniguchi ‘12] Histogram Averaged number of correctly predicted hidden states Tadahiro Taniguchi, Shogo Nagasaka, Kentarou Hitomi, Naiwala P. Chandrasiri, and Takashi Bando Semiotic Prediction of Driving Behavior using Unsupervised Double Articulation Analyzer 2012 IEEE Intelligent Vehicles Symposium, 849 ‐ 854 .(2012)
Drive Video Summarization based on Double Articulation Structure of Driving Behavior [Takenaka ‘12] http://www.youtube.com/watch?v=knwiO6dVbnY Kazuhito Takenaka, Takashi Bando, Shogo Nagasaka, Tadahiro Taniguchi, "Drive Video Summarization based on Double Articulation Structure of Driving Behavior", ACM multim media 2012,
Online Learning of Concepts and Words Using Multimodal LDA and Hierarchical Pitman‐Yor Language Model [Araki ‘12] Connecting multimodal categorization and word segmentation to achieve language acquisition through daily interaction. Takaya Araki, Tomoaki Nakamura, Takayuki Nagai, Shogo Nagasaka, Tadahiro Taniguchi, Naoto Iwahashi Online Learning of Concepts and Words Using Multimodal LDA and Hierarchical Pitman‐Yor Language Model IEEE/RSJ International Conference on Intelligent Robots and Systems 2012 (IROS 2012), 1623‐1630 .(2012)
Conclusion Summary Defining the symbol emergence system Introducing symbol emergence in robotics Introducing double articulation analyzer Current challenge Unsupervised lexicon acquisition using double articulation analyzer and multi‐modal categorization Discussion topic What is the important feature of language which we have to model to obtain computational understanding of human language.