Glossary
Key terms and concepts in HARMONIC
AMR (Action Meaning Representation)
Semantic representations specifying the content of actions to be executed, generated by the decision-making service before being rendered into executable commands.
Action Data Communication APIs
The strategic and tactical layers are connected through a bidirectional interface that enables efficient inter-layer communication and data transfer. High-level action commands sent by the OntoAgent through the Interface APIs are unpacked and used to update corresponding variables in the blackboard.
Action Outputs
Physical and communicative outputs generated by the system. Represents the system's ability to interact with and affect the environment.
Actuation and Motor Control Policies
Specific rules and algorithms for controlling physical outputs. Implements safety constraints and control policies for physical actions.
Attention Service
Focuses cognitive resources on relevant information for strategic decision-making and directs sensory focus for immediate task relevance. Manages information filtering and prioritization at both cognitive and tactical levels.
Actionability Assessment
The process of determining whether an agent has sufficient understanding of a situation to proceed with action despite potential incomplete information.
Adjacency Pairs
Discourse-level action-response patterns governing turn-taking behaviors in dialog (e.g., request-compliance, question-answer, proposal-acceptance sequences).
Behavior Trees (BTs)
Hierarchical structures in the tactical layer that execute operations with priority ordering, enabling reactive responses while maintaining planned behavior.
Bidirectional Interface
The communication channel between strategic and tactical layers enabling data transfer and command execution across the dual-control architecture.
Collaborative-Activity Script
A meta-script that helps agents operating in teams organize themselves to accomplish shared goals, with different versions for team leaders and subordinates.
Common Ground
Shared understanding between team members about goals, plans, and situational awareness, essential for effective collaboration.
Communicative Acts
The intended function or purpose of an utterance (broader than speech acts as it includes non-linguistic communication), such as requests, assertions, or questions.
Discourse Relations
Semantic relationships between propositions in dialog turns that may be explicit or inferred, important for assessing actionability.
Episodic Memory
Long-term storage of remembered instances of world objects, events, and past processing experiences, enabling agents to leverage past experiences for future decision-making.
Explainability
The system's ability to provide transparent, human-understandable explanations of its reasoning, decisions, and actions through traces of cognitive processing.
GMR (Generation Meaning Representation)
Intermediate semantic specifications for language generation, encoding communicative content grounded in ontological concepts prior to surface realization through the natural language generator.
Grounding
The process of connecting symbolic representations to physical world entities, perceptual data, or episodic memory instances (indicated by # indices in representations).
HARMONIC (Human-AI Robotic Team Member Operating with Natural Intelligence and Communication)
A dual-control cognitive-robotic architecture that integrates strategic (cognitive) level decision-making with tactical (robotic) level control, enabling robots to function as trusted teammates in human-robot teams through transparent reasoning and natural language communication.
LEIA (Language-Endowed Intelligent Agent)
The cognitive architecture incorporated in the strategic layer (aka OntoAgent). Neurosymbolic, multimodal cognitive-robotic systems implemented in the HARMONIC architecture that can interpret experiences, reason, and learn using ontologically-grounded knowledge. This is used interchangeably with OntoAgent.
Metacognitive Reasoning
Self-monitoring capabilities enabling introspection of internal states, team member modeling (mindreading), and dynamic strategy adjustment based on situational assessment.
Mindreading
The metacognitive capability of modeling teammates' mental states, beliefs, capabilities, and intentions to enable effective collaboration.
OntoAgent
The cognitive architecture incorporated in the strategic layer (aka LEIA), responsible for semantic interpretation, attention management, goal-setting, sophisticated planning, and addressing unexpected challenges in interpretable ways. This is used interchangeably with LEIA.
OntoGraph
A knowledge base API providing unified format for representing and accessing knowledge across the system, supporting inheritance, flexible organization into "spaces," and efficient querying.
Ontology
A hierarchical knowledge repository containing formalized representations of entities (concepts), relationships, properties, and procedural schemas (scripts) that serve as the semantic foundation for agent reasoning.
Perception Interpretation
The process of converting multimodal sensory inputs (speech, vision, haptic) into ontologically-grounded meaning representations for unified reasoning.
Perception Data Communication APIs
The tactical layer provides the strategic layer with preprocessed multimodal (speech, vision, etc.) perception data and relays robot state information, employing a suite of perception models within the perception services.
Perception Inputs
Various sensory data streams feeding into the system. Provides the raw information needed for environmental understanding and decision-making.
Perception Services (Tactical)
Handles raw sensory input and provides real-time environmental awareness. Processes sensor data for immediate action and safety monitoring.
Plans
Instances of scripts with parameter values set for specific situations, executed through the goal and plan agenda.
Preconditions
Requirements that must be satisfied before a plan can be executed, checked and resolved during plan instantiation.
Reference Resolution
True resolution of referring expressions to specific instances in episodic memory, distinguished from textual coreference resolution.
Scripts
Complex events or procedural knowledge recorded as sequences of events with coreferenced participants and props, representing how typical actions unfold.
Short Horizon Plans
Tactical plans for immediate future actions, often derived from strategic goals. Bridges the gap between high-level planning and immediate execution.
Strategic Reasoning & Decision-Making
Generates high-level plans and makes decisions based on long-term goals and knowledge. Handles complex reasoning about goals, constraints, and trade-offs.
Semantic Memory
Long-term storage of generic knowledge about types of events and objects, distinct from specific instances stored in episodic memory.
Situation Model
Working memory containing currently active concept instances and representations of entities and events that are part of the current task context.
Strategic Layer
The cognitive component responsible for high-level decision-making, planning, perception interpretation, attention management, and goal selection. Implements System 2 (slow, deliberative reasoning) processing.
Tactical Layer
The robotic control component responsible for real-time execution, reactive planning, and physical safety. Implements System 1 (fast, reactive) processing through behavior trees.
Tactical Reasoning
Real-time planning and decision-making for immediate actions, executed through behavior trees with priority ordering.
TMR (Text Meaning Representation)
Ontologically-grounded semantic representations of natural language input, capturing meaning in a normalized format independent of surface linguistic form.
VMR (Vision Meaning Representation)
Ontologically-grounded semantic representations of visual perception, converting visual input into structured meaning representations suitable for cognitive reasoning.