Multi-Robot Search & Retrieve

What's happening: Danny initiates the search task by sending a message to the robot team. This triggers the team leader (UGV-U) to place a COLLABORATIVE-ACTIVITY on its agenda.

HARMONIC's response: The UGV-U recognizes this as a search task and begins the collaborative planning process:

@COLLABORATIVE-ACTIVITY (leader)
[INIT]
  *identify-team-members
[SELECT-PLAN]
  // Select a plan from available options.
  RUN *identify-candidate-plans
  RUN *select-plan
  // Selected: SEARCH-FOR-LOST-OBJECT

What's happening: The UGV-U verifies preconditions for the SEARCH-FOR-LOST-OBJECT plan by gathering information about the object's appearance.

Strategic layer processing: The UGV-U updates its plan with the new information:

[PRECONDITIONS]
  // Preconditions from the selected plan
  RUN NEW @REQUEST-OBJECT-TYPE ✓
  RUN NEW @REQUEST-OBJECT-FEATURES ✓
  // Features: red keychain, small flashlight
  RUN NEW @REQUEST-LAST-SEEN-AT
  RUN NEW @REQUEST-LOCATION-CONSTRAINED

What's happening: The UGV-U gathers location information to prioritize search areas. The front door area becomes a priority location for the search sequence.

Plan instantiation: The UGV-U now has enough information to instantiate the complete search plan:

@SEARCH-FOR-LOST-OBJECT
  [SEARCH-ZONES]
    // Search in each ZONE the agent knows.
    // Priority: FRONT DOOR (last used location)
    FOR #ZONE-1 IN #LOCATION-1.SEARCHABLE-ZONE
      RUN ASYNC AWAIT *search
      INTERRUPT WHEN #OBJECT-1.LOCATION KNOWN
      RUN *consider-reporting

What's happening: With preconditions met, the UGV-U moves to SUGGEST-PLAN and shares the domain plan with the drone through natural language dialog.

Subordinate response: The DRONE-D receives this information and adopts its assignment within the suggested plan:

@COLLABORATIVE-ACTIVITY (subordinate)
[INIT]
  *identify-team-members
[WAIT-FOR-PLAN]
  // Wait for a plan from the leader
  AWAIT $.HAS-COLLABORATIVE-PLAN ISA @EVENT ✓
[RUN-PLAN]
  // Execute assigned search area: APARTMENT
  RUN NEW @SEARCH-FOR-LOST-OBJECT

What's happening: Both robots begin exploring their assigned areas using a waypoint strategy controlled by their individual tactical modules.

Strategic vs Tactical: The strategic (cognitive) module maintains awareness of area existence without directly guiding robot navigation, allowing for efficient local path planning while preserving high-level planning.

What's happening: Throughout the search, robots report their findings to each other. When a robot fails to locate the keys in a searched area, it communicates this to its partner.

Coordination protocol: This continuous communication ensures efficient coverage and prevents redundant searching. The UGV-U reports that the doorway area has been searched without success.

What's happening: The UGV-U's Vision Meaning Representations (VMRs) widget displays object detection results that the strategic layer processes from sensing frames communicated by the tactical layer.

Strategic analysis: The strategic module continuously analyzes sensor data frames, grounding these VMRs against the instance of the KEY object stored in its episodic memory. The keys are located north of the couch.

What's happening: When the keys are found, the UGV-U reports the successful completion to Danny. Notice the difference in location description: "north of the couch" (M8) vs "behind the couch" (M9).

Context-appropriate communication: This demonstrates the cognitive agent's ability to generate context-appropriate language for different communication partners. The UGV-U uses more natural language when reporting to the human operator.

Mission Summary: The multi-robot search and retrieve mission demonstrates HARMONIC's distributed planning capabilities:

• Collaborative Planning: Leader-subordinate coordination through meta-scripts
• Distributed Execution: Strategic plans divided into tactical commands
• Real-time Coordination: Continuous communication and status updates
• Context-aware Communication: Different language styles for different audiences
• Efficient Search: Coordinated coverage without redundancy

Key Success Factors: The mission succeeded due to effective plan distribution, clear role definitions, and robust communication protocols between the strategic and tactical layers.

Complete HARMONIC Multi-Robot Demonstration

Watch the full multi-robot demonstration showing HARMONIC's distributed planning and collaborative execution process with UGV and drone coordination.