← All projects
In progress

Agentic Vision-Language-Navigation

AstraNav · Language-guided navigation

Teaser figure for Agentic Vision-Language-Navigation

Language-guided navigation demo

From a black wagon to a bicycle, then back to a plant: follow the agent’s visual observations and navigation decisions as it works through a multi-stage instruction.

Inside the navigation

Follow the task through the agent’s eyes, one decision at a time.

Instruction

Go straight to black wagon, turn right 90 degrees and go straight to the bicycle, and return to the plant

Step 01

User · observationUser camera observation at step 1
Agent
  1. Turn left
  2. Move forward
  3. Move forward
  4. Move forward

The black wagon is ahead, slightly left; align with it and approach across the clear floor.

Step 02

User · observationUser camera observation at step 2
Agent
  1. Move forward
  2. Move forward
  3. Move forward
  4. Move forward

The black wagon is directly ahead with clear floor before it; approach it before making the right turn.

Step 03

User · observationUser camera observation at step 3
Agent
  1. Turn right
  2. Turn right
  3. Turn right
  4. Turn right

The black wagon is now close ahead. Turn right 90 degrees as instructed to face the bicycle.

Step 04

User · observationUser camera observation at step 4
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is visible in the dark doorway ahead-right; align with it and advance across the clear floor.

Step 05

User · observationUser camera observation at step 5
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle remains ahead-right in the dark doorway; adjust toward it and continue over the clear floor.

Step 06

User · observationUser camera observation at step 6
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is still ahead-right by the dark doorway. Aim toward it and continue across the unobstructed floor.

Step 07

User · observationUser camera observation at step 7
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is visible ahead-right against the dark doorway, with clear floor leading to it. Continue approaching.

Step 08

User · observationUser camera observation at step 8
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is ahead-right beside the dark door, still some distance away. Steer toward it and continue along the clear floor.

Step 09

User · observationUser camera observation at step 9
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is ahead-right beside the dark door, with open floor leading to it. Align and continue approaching before returning to the plant.

Step 10

User · observationUser camera observation at step 10
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is ahead beside the glass door. Continue toward it over the clear floor, leaving room to turn back toward the plant.

Step 11

User · observationUser camera observation at step 11
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Move forward

The bicycle is closer but still ahead-right, with clear floor before it. Approach it before turning back toward the plant.

Step 12

User · observationUser camera observation at step 12
Agent
  1. Turn left
  2. Turn left
  3. Turn left
  4. Turn left

The bicycle is now close ahead. Turn around to begin returning to the plant seen at the start.

Step 13

User · observationUser camera observation at step 13
Agent
  1. Turn left
  2. Move forward
  3. Move forward
  4. Move forward

The return passage is open ahead-left, while the wall is close on the right. Align with the passage and head back toward the starting area.

Step 14

User · observationUser camera observation at step 14
Agent
  1. Turn left
  2. Move forward
  3. Move forward
  4. Move forward

The return corridor is open ahead-left, with the wall close on the right. Move toward the open center and continue back toward the starting plant.

Step 15

User · observationUser camera observation at step 15
Agent
  1. Move forward
  2. Move forward
  3. Move forward
  4. Move forward

The corridor opens toward the familiar red suitcase and work area. Continue over the clear floor toward the starting area to locate the plant.

Step 16

User · observationUser camera observation at step 16
Agent
  1. Move forward
  2. Move forward
  3. Turn left
  4. Turn left

The return route opens beside the red suitcase. Advance over the clear floor, then look left for the starting plant.

Step 17

User · observationUser camera observation at step 17
Agent
  1. Turn right
  2. Turn right
  3. Move forward
  4. Move forward

The plant is not visible in the yellow seating alcove. Turn back toward the open floor beside the glass room and continue returning.

Step 18

User · observationUser camera observation at step 18
Agent
  1. Move forward
  2. Move forward
  3. Move forward
  4. Move forward

The plant is not yet visible. Continue toward the end of the yellow wall beside the familiar work area, where the floor opens to the left.

Step 19

User · observationUser camera observation at step 19
Agent
  1. Turn left
  2. Turn left
  3. Turn left
  4. Turn left

Plant foliage is visible at the far left beside the glass room. Turn left to bring it into view before approaching.

Step 20

User · observationUser camera observation at step 20
Agent
  1. Turn right
  2. Move forward
  3. Move forward
  4. Turn left

The plant is visible beside the glass wall, partly behind the tall white table. Approach through the clear floor to the right of the table and black post.

Step 21

User · observationUser camera observation at step 21
Agent
  1. Stop

The leafy plant in its white planter is now close beside the table and glass partition. Returned to the plant after visiting the wagon and bicycle.

Overview

Exploring how embodied agents can understand everyday instructions, interpret their surroundings, and decide where to go next. AstraNav connects visual perception with language-guided decision-making, with a focus on multi-stage tasks that require recognizing landmarks and returning to previously visited places.

Resources