Multiagent system architecture

Here is a step-by-step description of the Multiagent architecture of PyMASEP.

First step : Agent and Environment

The core concepts are the following : an Agent observe the Environment, choose its action and the environment give him another observation and so on. The main loop is controlled by the Engine.

State, Agent inside Environment

One step ahead, the Agent is part of the Environment and observes the State, which is also part of the environment. The Environment applies the action of the Agent to modify the State. Note that in this kind of model, the State must contains only information needed for the environment to evolve. Usually, all other information must be transfered into the interface subsystem.

Introducing Game

Ultimately the rules which control the evolution of the state of the environment is defined outside the environment. The Game class defines all static data and rules of the evolution.

More “realistic”: Agent inside State

We can now include the Agent into the State.

At this step, it is time to begin the understanding of the creation/init workflow since it will become more complex in future.

Introducing objects

In order to be able to describe the state, we introduce a more complex structure adding Object, ObjectState, Characteristics and BaseObject. Note that we refer to parent in the documentation, we refer to the ‘is composed of` link (with diamond, in the following figure) not to the object-oriented parent. With this concept, a ObjectState is a parent of a Characteristic, the Object is the parent of the ObjectState and the State is the parent of all Objects and Agents.

It is important to understand the initialization process as seen in the following schema. The following schema shows only the creation of Agent but the process is the same for the Object. All BaseObject are created calling Environment.create_object() method.

Agents can act with Action

Agents can now choose Action to act.

Here is the call sequence of the main loop.

Agents use Controller to choose an action

In order to be more generic in the “intelligence” of agents, a Controller is used to choose the Action only from the observation of the state. Controller is like Game, it does not contain any dynamic data but only code, to be used by many agents.

Template to create BaseObjects

Templates are defined to be able to generate the object hierarchy from xml without being entirely explicit.

Intention of the Agent

Agent can plan Intention as a list of Actions and validate them before the Environment/Game execute them.

Container

A Container, as the name suggests, contains other Objects. Note that the Container does not refer to other Objects but the instances are put in the Container.

A Container is not an Object, it is more a property of an Object. For example, a chest is not a container, it’s an object with a container. It can even have more container (for example, a secret compartment).

Belief

A belief is a knowledge of an agent get from the observation. This knowledge could impact the observation (and could, in the future, the state transition).

For those who knows the DEC-POMDP model [OA16], a belief is inspired from these kind of models with some differences. Here, the belief is controlled by the environment and can be updated by the Transition model or the Observation model.

In this model, the observation function (Omega) is divided in two as shown in the next figure

In the model, the Belief is outside the state to represent the fact that the transition part (next_state() in Pymasep) should not modify the Belief since it is an observation concept. However, in the code, the Belief is inside the State to be able to use all functionalities of the state (template, initializer, serialization, etc.). Note that, at the moment, Belief and Intention concepts are experimental and could be moved (inside or outside the state, if necessary) in the future.

More details will be added into a more scientific paper in the (hope near) future.