CSCE 475/896

Topic Summary Assignment 2: Intelligent Agents

Questions and Answers

September 16, 2002

An important note: Is communication an absolutely necessary component within a multiagent system? What if the agents cannot communicate but can only observe each other’s actions? Is it possible for the agents to coordinate their actions without direct communication? Is it possible for the agents to exhibit coherent behavior as a whole by simply observing and learning from each other’s actions?

Q1: Since each cooperation protocol has its advantages and disadvantages, can they be combined to make hybrid systems that are superior? What are some examples of combinations that work well? What are some real-world examples of systems that use protocols like these? Which is used most commonly? Are there many others?

A1: There are many parts to this question. Yes, these different protocols can be combined to make hybrid systems that are superior. But then the issue now becomes how to choose which protocols to use for which situations. To do so, one will need to profile each situation and map these situations to their appropriate protocols.

There are examples of combinations that work well in the literature. But so far I have not encountered real-world applications using hybrid cooperation protocols.

There are, however, real-world examples of systems that use protocols like these. HEARSAY, was a very famous system, that used the blackboard architecture. This system was used in real-world applications such as a hospital information board.

A leading researcher in MAS, Tuomas Sandholm, has a company that provides solutions in negotiations, control net protocols, and other methodologies. With these solutions, the company has saved hundreds of million of dollars for many other companies in just the past 2-3 years.

Contract net protocols, blackboard systems, and negotiations are almost equally popular. These days, contract net protocols and blackboard systems are more commonly used in the real world. Negotiations are still a hot research topic—researchers are still investigating this.

Yes, there are many other protocols—hierarchical ones, opportunistic ones, hybrid ones, adaptive ones, and ones with machine learning capabilities.

Q2: How dynamic are these systems able to become? In many situations it seems like in order for the protocol to actually work, like in a Blackboard system for instance, there must be a predefined set of “problems” that can be put on the board in order for agents to be able to respond. How is “flexible representation of blackboard information” achieved?

A2: Actually, these systems may become very dynamic. If we look at the Blackboard systems, it is a very flexible design. We do not necessary need to have a predefined set of “problems”. Why? Let us consider this. I want to buy a novel written in Malay. I put a note (in Malay) on the bulletin board. Students pass by the bulletin board. Since most the students do not understand Malay, they will pay no attention to the note. But, may be, there is a student out there who knows Malay and happens to notice the note, and hopefully he or she will respond to my request.

We may see the same flexibility on a Blackboard system. The agents actually may use their own languages to post their tasks and solutions. Agents that understand the postings will process those postings. Other agents that do not understand the postings will just ignore them. This is why the blackboard system is so flexible.

In addition, suppose that the system starts with 5 agents handling a set of problems. Now, a new problem appears. Nobody knows how to solve it. The problem was posed by one of the 5 agents and after a long period of no solution, the blackboard controller removes it from the blackboard. However, after a while, a new agent is added to the system. The addition of this agent is very simple—all it needs to know to join the group is the location of the blackboard and the way to post things on the blackboard. And now, if the new problem appears again, this new agent may know how to solve it. And this new agent will through this introduce itself to the other agents. Unlike the negotiations and contract net protocol which requires more direct interaction among agents, blackboard is thus more flexible and more dynamic.

Q3: The author clearly described the definition of cooperation and negotiation among agents. However, it is not well discussed about how an agent would decide whether it should cooperate or compete with another agent during its first encounter. Is there any special initial message passing procedure to exchange some information (e.g., goals and resources) between agents before they decide?

A3: Usually, whether the agents are competitive or cooperative is determined when one designs the system. Theoretically, this can be determined at real-time. However, it is difficult to make sure that coherence will be achieved if we allow that—too difficult to predict the system behavior. However, that does not mean it is impossible to have such a system. Yes, there would be special initial message passing or processing so that the agents would be able to find out about each other’s mental state. See also Q6 and Q11.

Q4: In communication, what will happen when a query results in another query instead of a reply as expected?

A4: If the query is expected as one of the possible responses, then the agent should be able to continue. However, if the response is not expected and throws the agent off its interaction protocol, then the agent either aborts the interaction process, or tries the same query again.

Q5: In order to cooperate successfully, each agent must maintain a model of the other agents, and also develop a model of future interactions. Does this mean that a particular agent must store the complete information of the other agents or only the considered important information? How would an agent decide what is the important information to use as models of other agents?

A5: No, this does not mean that a particular agent must store the complete information of the other agents or only the considered important information.

An agent may store all information but some may be outdated. An agent may also store only important information. Also, through interaction, an agent may be able to update the information. We call this modeling of other agents or profiling of other agents.

There are several ways that an agent uses to identify the important information to use as models of other agents. First, it is pre-designed. We as the designer of the system can build this. Second, the important information is obtained from other agents through interaction. Sometimes, other agents may tell the agent what is important to know. Third, the important information may be learned by the agent from looking at its experience. This can be achieved in this manner. The agent initially uses a lot of information to profile another agent, for example. It uses the information to make decisions. Information that leads to good outcomes will be given more weights or points. Gradually, the agent will have some weighted information. Some of the information will be weighted so lowly that the agent finally drops them from further profiling.

Q6: Is it possible that an initially self-interested agent changes itself to be more cooperative? If yes, under what circumstances?

A6: See Q3 and Q11 also for this question. In general, we do not want agents to change from being self-interested to cooperative, and vice versa during run-time. However, it is possible to have such agents. For example, when agent fails to perform any task while being cooperative, it may switch its strategy to being self-interested. That means the agent would have to have the knowledge to behave cooperatively and self-interested-ly. It will determine which behavior to follow based on its observation of the environment and of its own performance.

For example, if an agent first starts out being very competitive and realizes that it cannot win any bid in the contract net protocol system, then it may start cooperate with other agents to form coalitions or teams to make better bids.

Q7: Some problems with the blackboard systems seem to be that someone who is well qualified but busy would not see a job and would not be able to post anything to the effect of “can you wait 5 minutes, then I can do it.” Whereas with contract net, they can reply with that kind of message. How does the blackboard system address this problem?

A7: This observation is valid. But theoretically speaking, there is nothing to prevent a busy but qualified agent to post “I am busy right now and will be able to deal with your request in 5 minutes.” So, now the issue lies with the agent that initiated the request in the first place. How will it handle such reply? It is then similar to the contract net protocol.

Q8: If some agents are self-interested, and the overall system goal is not explicitly defined to all the agents, is not it possible that the agent will perform actions it believes are in the best interest of the system, but turn out to be detrimental?

A8: This is a good philosophical question. First, if we design the agents such that each believes it is making the correct decisions for the good of the system and it turns out that the decisions are detrimental to the system, then that means we have failed to design a coherent multiagent system.

Remember that one of the key goals of designing a good MAS is to have only explicit local controls and yet the system exhibits coherent behavior as a whole.

So, yes, it is possible that if we design the MAS incorrectly, the agents will perform something that is detrimental to the system. We have to be careful.

Q9: … It is not immediately clear what factors affect efficiency and how to design a system with efficiency in mind. For instance, how does one know the number of agents that will yield the most efficient system, and if one adds more agents, how does this affect the performance? It is also unclear on how to select interaction and communication protocols based on the number of agents and desired performance of the system.

A9: First, many factors affect efficiency. When we measure the efficiency of a system, we look at the costs. How much time is expended? What is the disk space usage? How many messages are transmitted? How many floating point operations are performed? How many agents are involved?

Then, we have to look at the factors affecting efficiency. There are factors such as communication bandwidth and delay, sensor data accuracy (noise and uncertainty), how dynamic the environment is, how distributed the solutions are, and so on.

Because of these factors, sometimes it is impossible for one to know the number of agents that will yield the most efficient system. In general, if the problems that the system is to encounter require an average of N agents, then the system should have N agents. This is our baseline. Then, given the different factors, we adjust this number N. For example, if the communication channel is noisy, meaning that some agents may not be able to receive messages properly, then the system may need more agents to increase its robustness.

If one adds more agents, this may affect the performance acutely. An additional agent may bring more expertise and resources into the system. (Think about an extra person to help you move.) However, an additional agent also means another agent to consider and this leads to additional processing. (Think about an extra person that you need to obtain a signature from to approve a task.)

Selecting interaction and communication protocols should be mainly based on the characteristics of the problem instead of the number of agents. However, if those characteristics are of no relevance, then here are some rules of thumb. If the number of agents is large, then the blackboard systems are better than the contract net protocol and the negotiations. If the number of agents is small, then the negotiations are better than the contract net protocol and the blackboard systems.

The desired performance has to be based on the problem characteristics and thus not easy to single out to discuss.

Q10: I don’t understand how the control component works in a blackboard system.

A10: Actually, when we build a blackboard system, we simply use a process. When a message is received, the process will store this message in a shared memory where every agent has access. The control component, as another thread monitoring the main process of the blackboard, may have many different functions. One of them is to keep track of the duration of each problem posted on the blackboard. If the duration is over t minutes, for example, the controller may decide to remove it from the blackboard—from the shared memory. Another one is to clean up the blackboard—the shared memory. For example, if a problem is solved, then all the messages posted as a result of that problem must be removed from the blackboard. The controller can do this.

Q11: Can an agent be cooperative to some agents in a multiagent system and be competitive or self-interested to some other agents in the same system?

A11: See also Q3 and Q6. Theoretically, yes, an agent can be designed to be cooperative to some agents and be competitive to others in the same system. Practically, this can be achieved through task-based reasoning. For problem A, for example, the agents may want to cooperate since they rely one each other to solve the problem. For another problem B, the agents may want to compete with each other since every agent knows how to solve the problem on its own.

Q12: What is the order of multiagent system design and task decomposition? In other words, do we design a multiagent system and try to use it solve many problems? Or do we decompose a special task and then design multiple agents to do subtasks and group them together as a multiagent system?

A12: Either way is fine. It depends on the characteristics of the problems that we want to solve. First, if we know clearly how to decompose a task into many subtasks, and we know which agent should perform which subtask, then we may use the second approach. Second, if we only know that a distributed solution is good but we do not know for sure how to decompose the tasks or how to assign the tasks, then we may want to build the multiagent system first and apply it to different problems and fix the system accordingly to solve each problem.

Q13: What if more than one agent wants to be a leader/manager in a Contract Net Protocol?

A13: First of all, in a Contract Net Protocol, more than one agent can be a leader/manager at the same time if they are handling different problems. If the same problem is observed by two agents at the same time and both initiates their own contracting process, then this could make things really cumbersome. The two agents then will need to realize that they are both handling the same problem and need to resolve this dual leadership. So there are two issues here. First, the agents need to figure out they are dealing with the same problem. This can be easily detected when they receive each other’s announcement of task. Second, the agents need to decide who will continue to be the leader and who will drop out and assume a contractor role. This can be done by looking at the current status, resources, and past experience of the agents. The agent with the better quality wins the leadership role.