Enviroment Types (COMP3411)

Environment Types

Accessible (observable) vs Inaccessible

If an environment is completely observable by the agent then it is accessible. This means the agent's sensors can detect all data relevant to any choice. It's convenient because no internal data need be kept.

Deterministic vs Non-Deterministic/Stochastic

If the next state of the environment can be determined solely from the current state + selected actions then it is deterministic.

Essentially there is no uncertainty or luck. I.e. a coin toss is non-deterministic.

However it's often better to think of an environment as being either deterministic or not from the point of view of the agent (so if it's inaccessible it may appear non-deterministic to them).

Episodic vs Non-Episodic/Sequential

If each move is self-contained, and the game is divided into 'episodes' of perceiving followed by acting, then the environment is episodic.

This means that actions do not depend upon previous actions to determine their quality. No thinking ahead is needed and so the environment is simpler than a sequential environment.

Static vs Dynamic

In a static environment, nothing changes while the agent thinks. I.e. it's a turn-based game, and all action freezes whilst the agent perceives and deliberates.

Static environments are easy as no information need be updated whilst deciding on a move.

If passage of time affects an agent's score, then the environment is semi-dynamic.

Discrete vs Continuous

If there are a clear number of distinct possible actions then the environment is discrete (e.g. a chess board has defined squares - you are on one or not, never half in one half in another).

However a car is continuous, as it is constantly switching between speeds and locations.

Known vs Unknown

When the rules of the game or the physics of the environment are known to the agent we consider it a Known environment.

Simulated vs Situated/Embodied

If a separate program is used to simulate the environment or feed percepts to agents then it is simulated.

Situated implies that the agent is actively within the environment and interacting with it specifically. If embodied the agent must physically interact with the environment (e.g. with a human body).

A situated agent would be an airline reservation system - you're changing actual real world things, dealing with actual money and seats, but not physically touching said seats.

An embodied agent would be a spray-painting robot that just runs a pre-programmed series of actions. So it is physically interacting with the environment. As it makes no intelligent decisions based on percepts, it is not necessarily situated.

Multi-Agent vs Single Agent

The number of players in the environment is either limited to the agent or multiple.

Real World

The real world is

  • Partially Observable
  • Stochastic
  • Sequential
  • Dynamic
  • Continuous
  • Unknown
  • Situated
  • Embodied
  • Multi-Agent