Natural Language Understanding and Cognitive Robotics: 1st Edition (Hardback) book cover

Natural Language Understanding and Cognitive Robotics

1st Edition

By Masao Yokota

CRC Press

250 pages | 8 Color Illus. | 60 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9780367360313
pub: 2020-01-15
SAVE ~$37.99
Available for pre-order

FREE Standard Shipping!


In the not so distant future, there is likely to be a symbiotic world of people and robots with friendly interactions between them. But for this symbiosis to occur, we need to understand the action of people (e.g., seeing, hearing, thinking, speaking, …) and imbibe the human-like traits in robots. The most essential feature necessary for robots to achieve is that of IMU (integrative multimedia understanding) which is a natural function in humans. It allows us to assimilate pieces of information expressed through different modes such as speech, pictures, gestures, etc. The book describes the methodology to provide robots with human-like capability of natural language understanding (NLU) as the central part of IMU. The core of the methodology is mental image directed semantic theory (MIDST) which is based on the hypothesis that NLU in humans is essentially processing of mental image associated with natural language expressions, namely, mental-image based understanding (MBU). MIDST is intended to model omnisensory mental image in human and to afford a knowledge representation system in order for integrative management of knowledge subjective to cognitive mechanisms of intelligent entities such as humans and robots based on a mental image model visualized as ‘Loci in Attribute Spaces’ and its description language Lmd (mental image description language) to be employed for predicate logic with a systematic scheme for symbol-grounding. This language works as an interlingua among various kinds of information media, and has been applied to several versions of the intelligent system IMAGES (interlingual understanding model aiming at general system). Its latest version, i.e. conversation management system (CMS) simulates human MBU and comprehends the user’s intention through dialogue to find and solve problems, and finally, returns its response in text or animation.

The book is aimed at researchers and students interested in artificial intelligence, robotics, or cognitive science. Based on philosophical considerations, the methodology will also have an appeal in linguistics, psychology, ontology, geography, and cartography.

Key Features:

  • Describes the methodology to provide robots with human-like capability of natural language understanding (NLU) as the central part of IMU
  • Uses methodology that also relates to linguistics, psychology, ontology, geography, and cartography
  • Examines current trends in machine translation

Table of Contents

Table of Contents:


Anna - an ideal home robot

Intuitive human-robot interaction

Integrative multimedia understanding and natural language understanding

Knowledge and cognition

Natural Language Processing Viewed from Semantics

Trends in machine translation

Case study of current MT systems (as of October, 2018)

Fundamentals for Robotic NLU

NLU in accordance with semiotics

Syntactic analysis

Semantic analysis and pragmatic analysis

Robust NLU

Response synthesis

Syntax and semantics of discourse

Cognitive Essentials for Midst

Functional model of human mind

Human knowledge and cognitive propensities

Semantics and mental image

QSIs (quasi-symbolic images) and human concept system

Primitive quasi-symbolic images

Perception of causality

Semantic articulation and QSI connectors

Negation of mental image

Imaginary space region

Computational Model of Mental Image

Atomic locus as primitive QSI

Temporal conjunctions as QSI connectors

Empty event

Attributes and Standards

Formal System

Semantic principle of Lmd

Syntax of Lmd

Tempo-logical connectives

Formulation of event concepts

Formulation of laws of the world

Fundamental Postulates and Inference Rules for Deductive System

Properties of Loci

Inference rules for deduction

Tempo-logical deduction with TLCs






Human-Specific Semantics of 4d Language as Mental Images

Conventional approaches to 4D language understanding

4D language semantics as mental images

Formulation of concepts of spatial prepositions

Properties of static 4D concepts as human intuitive mental images

Reversal operation on spatial change event concepts as mental images

Problem Finding and Solving in Formal System

Definition of problem and task

Creation problem finding and solving

Maintenance problem finding and solving

Human Language Understanding by Robots

Two-staged robotic NLU

Robotic concept system for iHRI

Compound concept system for robots

Robot manipulation as cross-media operation via Lmd

Aware computing in robots

Homogeneous/Inhomogeneous Communication

4d Language Understanding for Cognitive Robotics

Requirements for robotic NLU

Logical Adequacy of Lmd

Translation between NL and Lmd

Reasoning in Lmd

Anchoring via Lmd

Behavioralization via Lmd

Systematic interpretation of Lmd

Multilingual Operation Via Lmd

Meaning definition

Optimization of grammatical description for word meaning definition

Language operation via Lmd

Question answering through Lmd

Computational Model of Japanese for NLU

Brief description of basic Japanese

Phrase structure grammar for Japanese

Dependency grammar for Japanese

Sentence and discourse of Japanese

Sentence types of Japanese and phrasing

Implementation of Mental-Image Based Understanding

Configuration of CMS

MBU versus conventional NLU

Stimulus sentences to CMS and human subjects

Mental image based understanding by CMS

Problem finding and solving in CMS

Awareness control of CMS




About the Author


Masao Yokota was born in Miyazaki Prefecture, Japan, in 1949. He received Bachelor degree from Kyushu Institute of Technology, and Master and Doctor degrees from Kyushu University. He is now a professor of informatics at Fukuoka Institute of Technology. His research focus is on AI, especially in integrative multimedia understanding by robots in being as 'natural' as humans where natural language understanding (NLU) plays the central role. He has proposed 'Mental Image Directed Semantic Theory (MIDST)' based on a hypothesis that NLU in humans is mental image processing. MIDST provides an omnisensory mental image model and a formal language called 'Lmd (Language for Mental Image Description)'. This formal language has been already implemented on several versions of the intelligent system IMAGES including integrative multimedia understanding system IMAGES-M and conversation management system CMS. Dr. Yokota authored many papers on AI and was the leader of many projects funded by Ministry of Education, Culture, Sports, Science and Technology, Japan (MEXT), and Fukuoka Institute of Technology (FIT).


Knowledge representation as mental image description

Integrative multimedia understanding

Spatiotemporal knowledge formalization as natural semantics

Intuitive human-system interaction

Human mind model for robotics

Research Projects:

MEXT project: Representation and computation of human intuition of space and time

MEXT project: Automatic linguistic understanding of human action data for nursing aid

MEXT project: Integrative multimedia understanding of clinical reports

MEXT project: Conceptual understanding and generation of conversation

MEXT project: Complimentary acquisition of knowledge through language and picture

MEXT project: Meaning description of natural language based on human cognition model

MEXT project: Natural language understanding in correspondence with human mental imagery

MEXT project: Automatic understanding of clinical records described in natural language

FIT project: Spatiotemporal Inference

FIT project: Distributed Intelligent Robot Networking (DIRN)

FIT project: Cross-media Operations

FIT project: Natural Language Understanding in Humans


Information Media Science for undergraduates

Multimedia Information Processing for undergraduates

Advanced Multimedia Engineering for graduates

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Intelligence (AI) & Semantics
LANGUAGE ARTS & DISCIPLINES / Linguistics / Semantics
SCIENCE / Life Sciences / General