VideoDB Documentation

Society of Machines

Autonomy - Do we have the choice?

Ashutosh Trivedi

Why it is hard to take some decisions for humans? Whenever we have to take a complex decisions we have to deal with rationality, emotions and our beliefs. It’s a cognitive load to take decisions sometimes on certain issues. We find the situation complex, baffling. Sometimes we don’t even take difficult decisions and leave the situations as it is for years.

⁠

Interestingly, does this complexity lie in the situation or in our mind ?

Some decisions don’t take much time for example when we have to survive on the road while walking or jogging or when we have to decide it’s time to eat. These decision making processes are part of our motor skills and our body works in accordance. For example - You get hungry when you need food, thirsty when you need water, you don’t have to think and decide - “should I eat food?” and weigh its pros and cons all the time.

In some cases we don’t have to choose whether to decide or not, we act like a trained machine. We decide instantaneously or unconsciously. These are called instincts, they are imbibed in our brains.

Using machine learning we can train a machine or agent to take a decision on certain task using useful data. If you trained a machine to take intelligent decisions on ten tasks and leave that machine freely in the environment does it have a choice to not to take a decision? - No!

Choice of taking decision or not taking a decision requires a free will. Machines do not have free will. They do what they do, some machines do intelligent things but not with choice.

Interesting question to think is - what is choice? or what is autonomy?

Let’s understand the autonomy and choice framework in detail, Although we can’t really answer what choice or freewill is, we can learn other useful things in this process of finding an answer to this intriguing question.

Again, we will learn from real life and try to create a principle or framework, which we can apply computationally to create autonomous agents or machines.

Human Framework of Choice

Let me walk you through a situation. Imagine you are at war with your opponent. The objective of the war is obvious - to win! But there is a catch - you have no idea about warfare, absolutely nothing! You don’t even know which ammunition is used for which purpose.

You and your opponent can take certain actions. The choice of action depends upon situation and you can decide the action based on your intelligence. When this war happens for the first time. Suppose you had the first move, you choose an action and wait for the result, which we call payoff.

⁠

Now obviously, your opponent strikes, and you have to respond again. Which action will you take this time?

You have choice of these 5 actions - [ ‘fire a gun’, ‘send a missile’, ‘send a tank’ , ‘throw a grenade ‘ ]

If you have no knowledge, you will try something randomly and wait for the result.

⁠

This can keep happening for sometime. There is no fun in this. This is really dumb. Isn’t it? So, how would you become better at this warfare? By gaining more knowledge about it.

This knowledge can be acquired by -

Repeating it multiple times and finding the best moves.

Gaining more information about it from some other sources, such as books, articles, videos, etc.

It can either be in form of some thumb rules, strategies and tactics. You also need to remember this knowledge to use it later.

In order to win- you need an evolution in your thinking. This evolution will create strategies in your mind and also beliefs about your opponent. A belief is nothing but your guess about your opponents action. Strategies, are set of actions which can take you closer to a win. You can learn these strategies by looking at other wars and finding the best action possible in similar situations

⁠

You might also start discovering strategies. These strategies can be opponent specific or can be independent of it. For example, if your opponent strikes by a tank, then firing a missile at tank would take you closer to win. You are now capable of making complex choices.

Having these beliefs and strategies can also help you run this whole game in your mind and update your beliefs, create new strategies which can increase your chances of winning this game.

Wait… Did we say game?

Life is a Game

Hope you didn’t get too serious about the war. War is just an intense situation similar to any other life situation where we have to make rational choices. Choices from set of actions we can take.

You can describe most of the social situations as a game. A game is a situation which requires you to choose. By game, I don’t mean only sports. Sports are one set of game.

Any situation which requires you to choose is a game.

In most of the games, you have few more things -

A set of actions to choose,

Payoff after taking that action. payoff can be positive ( gain ) and negative ( loss ).

Other agents who are playing the same game.

Can you remember your life choices - Did you play games?

Yes! A lot of games everyday.

Went to a grad school? Yes/No

Which school ? …………

Which major ? ………..

You made so many choices. Now let me break it down for you. Games which you play only once are called - One shot game. Usually you choose an action from set of available actions which can be limited and you get a payoff and you move on.

⁠

Well, you can play this game again but most of the people don’t have time and money to play this game again.

Iterative Game

There is another type of game which is called iterative game which we play again and again and constantly get payoffs. For example how to reach office.

⁠

You have multiple actions to choose from [ “take a cab and pay $5” , “take the subway and pay $1”, “take the bus and pay $1.2” ]

Another example from

previous blog post⁠

- Remember Game of Life? It is an iterative game. Where agents keep on playing the game by taking the action from choices - [“live”, “die”]. The choice of agents are not intelligent, It is just according to a rule.

Evolutionary Game

Imagine you moved to a city with a new job and you’re trying to figure out the best way to reach the office from your place. It gets interesting from here - Now, each day is different, and there are other agents in this game - other people who are also going to office, the cab drivers, the environmental agents such as rain, snow, your mood, your budget and many many more. You may not even be aware of your environment completely. These are called external factors.

Each agent affecting and changing the situation or environment and you need to make the best choice to reach office. What would you do?

⁠

You need strategies to play it better. This is called an evolutionary game. You may become better/worse each time by playing it more. You may form beliefs about the cab drivers, the weather, other people’s choices which can govern traffic on your route, and strategy to deal with different situations. This valuable information will help you become better at it, and you need to remember/update this multiple times when you play this game.

Our framework of choice has agents, a situation where agents have to make choices, these choices are about actions they can take. There is an objective to be fulfilled by each agent, which we can measure by payoff. Since there are multiple agents, there can be conflict of interest of agents, there can be co-operation and also limitations or constraints while choosing an action.

This is formally called a Multi Agent System ( MAS ). I just introduced you to a very interesting kind of science. Study of complex systems. MAS solves design problems such as Supply Chain, Logistics, Organization Design, Policy Designs etc. It has huge value in Artificial Intelligence due to autonomy and rational choice being integral part of it.

There is also a very special property of MAS, the global objective of any MAS - The Emergence.

Emergence

Emergence is a property of the whole system. It is a consequence of actions taken by each agent in this system out of their own objectives - self interests. There can be interesting design problems for achieving a certain emergence.

Emergence can be designed as the global objective, and whole system is trying to achieve this objective.

We also saw emergence property in Game of Life in

my last post⁠

. The agents played an iterative game. The action they could take at each iteration was to live or die. Their choice of action was fixed and governed by constraints or rules of the game. Here the emergence design problem can be of achieving certain special final state. In this interactive game we could only decide the initial placement of the agents and the game will unfold itself according to its rule.

Let me connect you with a real life example. How would you find a solution to this problem -

Your city is having chronic traffic problem. How would you adjust fares of multiple transports such that you reduced traffic on certain routes at certain times.

You may have to consider all agents, their objectives, constraints and limitations. For instance certain limitation could be - to cover all economic classes of people, capacity of transports, office and non office hours, location of prime places etc. I won’t go in the details, but this framework can help in dealing with these kind of problems. You can simulate these complex systems of agents by using softwares suitable for designing MAS.

In future posts we will look at emergence in detail and also with a practical famous example. Later posts will be focused on human decision making, choice and game theory. If we have to solve complex system problem for humans - understanding human decision making and rational choice plays a very important role. Similarly it applies to complex machine systems.

There is also a great question for you - Can you identify how many iterative games are you playing in life? Do you wish to convert them to evolutionary games and become better at playing at it. Start with simple negotiations :)

Human Framework of Choice

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.