CSCE 475/875

Handout 20: Cooperation Day Analysis

November 30, 2007

 

1.  Tables of Results

Day 1

Team

Items Retrieved

Critical Item

Sequence

Utility

Rocky & Bullwinkle

1, 2, 3, 4, 6, 7, 8, 9 (8)

5(+$2500)

1, 2, 3, 4 (4)

$500

Team Win

10, 9, 7, 6, 5, 4, 3, 2 (8)

8 (+$2800)

10, 9 (2)

$200

The Flood

1, 3, 7, 9, 2, 4, 6, 8, 10 (9)

5 (+$3800)

1, 3 (2)

$200

Boole’s Fools

2, 4, 8, 10, 1, 5, 7, 9 (8)

6 (+$800)

2, 4 (2)

$200

Nick

1, 2, 3, 4, 10, 8, 7 (7)

5 (+$1500)

1, 2, 3, 4 (4)

$500

Mongii Agent Systems

5, 4, 3, 2, 1, 6, 8, 9 (8)

7 (+$2000)

5, 4, 3, 2, 1, 6 (6)

$1000

Table 1: Items retrieved, critical item needed to make a longer sequence, the actual sequence formed, and the actual utility obtained.

From the above table, we see that some critical items could have brought large utility gains to some teams.  However, these critical items were not obtained.  Based on Day 1, Mongii Agent Systems gained the largest utility with $1000 from their sequencing task.  Three teams tied with $200 each, and two teams tied with $500 each. 

Provider

Consumer

Price

Information

Rocky & Bullwinkle

Nick

$40

Item 2

Boole’s Fools

Nick

$100

Item 4

Rocky & Bullwinkle

Mongii Agent Systems

$40

Item 1

Boole’s Fools

Rocky & Bullwinkle

$60

Item 2

Boole’s Fools

Nick

$60

Item 1

Mongii Agent Systems

Team Win

$200

Item 10

 

TOTAL

$500

6 items

 

Average

$83.33

 

Table 2: Information providers, consumers, prices, and information for the transactions.  See Table 4 below as well.

Team

#Items Sold

#Items Bought

Total

Rocky & Bullwinkle

2

1

3

Team Win

0

1

1

The Flood

0

0

0

Boole’s Fools

3

0

3

Nick

0

3

3

Mongii Agent Systems

1

1

2

Total

6

6

 

Table 3: Number of items sold and bought.  See Table 5 below as well.

From Table 3, we see that most teams were quite active in terms of buying and selling information, except for The Flood with no transactions recorded.

Day 2

No critical items.  Every team retrieved all their items.

Provider

Consumer

Price

Information

Rocky & Bullwinkle

Mongii Agent Systems

$100

Item 1

Rocky & Bullwinkle

Team Win

$100

Item 10

Team Win

The Flood

$100

Item 7

Nick

Team Win

$300

Item 3

Nick

Boole’s Fools

$300

Item 4

Nick

Rocky & Bullwinkle

$300

Item 2

Team Win

Rocky & Bullwinkle

$100

Item 10

Team Win

Nick

$100

Item 4

Rocky & Bullwinkle

Nick

$200

Item 6

Team Win

Mongii Agent Systems

$200

Item 6

Boole’s Fools

Mongii Agent Systems

$500

Item 2

Boole’s Fools

The Flood

$800

Item 3

 

TOTAL

$3100

12 items

 

Average

$258.33

 

Table 4: Information providers, consumers, prices, and information for the transactions.  See Table 2 above as well.

Team

#Items Sold

#Items Bought

Total

Rocky & Bullwinkle

3

2

5

Team Win

3

3

6

The Flood

0

2

2

Boole’s Fools

2

1

3

Nick

3

2

5

Mongii Agent Systems

0

3

3

Total

12

12

 

Table 5: Numbers of items sold and bought.  See Table 3 above as well.

From the above table, most teams were active, and certainly more active than on Day 1. 

 

Mongii

Flood

BsFs

R&B

TeamWin

Nick

Day1

Start

$1000

$1000

$1000

$1000

$1000

$1000

Sale

$200

$0

$220

$80

$0

$0

Purchase

-$40

-$0

-$0

-$60

-$200

-$200

Utility

$1000

$200

$200

$500

$200

$500

SubTotal

$2160

$1200

$1420

$1520

$1000

$1300

Day 2

Start

$1000

$1000

$1000

$1000

$1000

$1000

Sale

$0

$0

$1300

$400

$500

$900

Purchase

-$800

-$900

-$300

-$400

-$400

-$300

Utility

$4000

$4000

$4000

$4000

$4000

$4000

SubTotal

$4200

$4100

$6000

$5000

$5100

$5600

TOTAL

$6360

$5300

$7420

$6520

$6100

$6900

Table 6:  Subtotals and totals in terms of utility ($) for each team for the game day.

From Table 6, we see that Boole’s Fools had the most utility ($7420).  Thus, Boole’s Fools is the winner of Game Day 3.  They distanced themselves from the other teams on Day 2 by net-gaining $1000 in their transactions. Nick, Rocky & Bullwinkle, Mongii Agent Systems, and Team Win placed 2nd-5th with total utilities between $6000 and $7000.  The Flood finished 6th with $5300.  Two teams failed to make a sale on Day 2: Mongii Agent Systems and The Flood. That hurt their final standings. 

2.  General Observations

Here are some general observations:

1.      Similar Strategies:  Most teams used the same underlying strategies: utility-maximizing.  Boole’s Fools used game playing as well but probably that did not factor into the results. 

2.      Day 1 vs. Day 2: 

·         More transactions took place on Day 2 than on Day 1 (12 items vs. 6 items).

·         The average price for each item sold or bought on Day 2 was significantly higher than that on Day 1 ($258.33 vs. $83.33).

·         More teams sold at least an item on Day 2 than on Day 1 (4 vs. 3).

·         More teams bought at least an item on Day 2 than on Day 1 (6 vs. 4).

·         All teams solved the search-and-retrieve task on Day 2.  None solved the task on Day 1 (the closest was 60% done).

·         Overall, Day 2 was much less hectic as Day 1.  On Day 1, the teams were less willing to purchase information as each thought they would be able to find all the items they needed.  On Day 2, the teams were more willing to purchase information and also willing to purchase information at a much higher price.

·         Further, on Day 2, there were more “NEED” postings as the teams realized that they could gain significantly much more utility from finding their sequence of items than selling information to others.  That actually caused the multiagent system to cooperate.  And that was the objective of this design!!  The Game Day was designed to motivate the agents to cooperate.  On Day 1, the agents did not cooperate as much since each believed that it could solve its tasks within the time constraint and resource constraint.  I was actually puzzled that there was so much more postings for selling information than postings for buying information.  I intentionally designed the system such that obtaining long sequences of items was very profitable; however, this was not exploited on Day 1.  On Day 2, the agents, having learned from Day 1, were “motivated” to cooperate out of necessity, and that was based on the utility gains and the constraints.

·         Further, on Day 2, searcher threads of agents were more observant and methodical.  This is key.  Covering a search area quickly allows one to obtain an overall picture of the area at the expense of high confidence.  In other words, that leads to uncertainty in sensing.  When there is uncertainty in sensing, it makes real-time decision making difficult.  On Day 1, most agents attempted to cover as much area as possible, because of the perceived time constraint.  On Day 2, most agents attempted to search as thoroughly as possible. 

·         Further, on Day 2, less inter-thread communication was incurred but more blackboard communication (in terms of postings) was incurred.  This is a very good transfer of computational resources.  On Day 1, each agent spent too much time communicate between threads, relaying and recording information that might or might not be useful, and that actually caused loss of information and ineffective blackboard postings.  On Day 2, more teams had more time posting and monitoring the blackboard.

3.  Team-Specific Observations

·         Nick:  This team did a poor job of tracking and recording their activities.  Their pre-game and mid-game strategies were concise.  Their strategies were mainly utility-maximizing.  Their mid-game strategies were very similar to their pre-game strategies.  The changes included cutting down the inter-thread communication overload and the blackboard communication overhead. 

·         Team Win:  This team did a so-so job of tracking and recording their activities.  Their pre-game and mid-game strategies were brief.  Perhaps, they planned to be reactive.  Reactive strategies are usually only appropriate for situations that are less uncertain.  The Cooperation Day had high uncertainty.  As a result, simply reacting to the happenings—without an overall big picture—might lead towards chaos.  They did not change their strategies mid-game.  They did cut down the communication delay overhead by keeping the cell phone on at all times.

·         The Flood:  This team did a rather good job of tracking and recording their activities.  Their pre-game and mid-game strategies were quite detailed.  On Day 1, their strategies were more cautiously utility-maximizing.  That led to no information purchase/sale.  After Day 1, they changed their strategies quite a bit.  They did a thorough search, similar to what Rocky & Bullwinkle and Boole’s Fools did.  They were also more risk taking: willing to purchase information no matter the cost.  That certainly gained them much higher utility.  They realized that “much greater utility to be gained from consuming information and giving less”.  And they reasoned that information selling was time intensive and yield less utility to them.  This is correct.  However, there is another reason: by purchasing, a team adds to its utility more than it adds to the seller’s utility; by selling, a team adds to the buyer’s utility more than it adds to its own utility.

·         Rocky & Bullwinkle:  This team did a rather good job of tracking and recording their activities.  Their pre-game strategies were detailed, very tactical as well.  However, their utility-maximizing designs did not factor in the “resource constraint”: in order to gradually, adaptively come up with the best information offers requires constant monitoring and posting on the blackboard.  That was not feasible in a time-pressured situation like Game Day 3.  This team had good lessons learned from Day 1.  For Day 2, they were more ready to purchase needed information, offer information at a higher price.  They adopted a similar “thorough one-over search” approach as did Boole’s Fools: search through once and then start buying information.

·         Boole’s Fools:  This team did an excellent job of tracking and recording their activities.  Their pre-game and mid-game strategies were mostly utilizing-maximizing with quite a heavy dose of game-playing.  Their pre-game strategies were very detailed and multi-faceted, taking into account a host of factors.  However, their pre-game strategies were not flexible enough to consider the impact of large utility gains for obtaining long sequences of items.  Further, they did not consider the time constraints.  However, they learned from their Day 1 mistakes and came up with a very goal-directed set of changes for Day 2.  Their mid-game strategies were precise and concise.  That allowed them to manage their communication better.  Their “searcher” thread was more observant on Day 2 and that helped them identify the “not-so-obvious” items and they had plans ready for exploiting those to increase their gains.

·         Mongii Agent Systems:  This team did an excellent job of tracking and recording their activities.  Their pre-game and mid-game strategies were utility-maximizing, with a touch of aggressiveness.  They did very well on Day 1 but realized that it might be due to luck.  Thus, they learned from other teams and planned to buy information to avoid losing out on obtaining a long sequence of items because of missing one item.  In their post-game analysis, they pointed out correctly that Day 2’s difference was in the sales and purchases.  They were not able to make any sale.

 

4.  Lessons Learned

·         On Day 1, several teams made poor decisions: emphasizing selling information too much as opposed to focusing on purchasing information.  From the viewpoint of utility-maximizing, that means they failed.  However, looking at the pre-game strategies, all teams planned to utility-maximizing.  So, what went wrong?

o   Most teams did not consider the resource constraints.  When an agent posts on the blackboard, it is obligated to entertain responses to its postings.  That is a resource-draining activity.

o   Most teams also did not consider the time constraints.  This led to hectic attempts of purchasing information at the end of Day 1.

This shows that it is important to consider both resource and time constraints when designing a MAS, especially in such a dynamic, uncertain, time-constrained environment.

·         Inter-thread communication plays a role.  Too much information relayed between threads cause the threads to slow down—not able to conduct other tasks. 

·         Accurate sensing is important.  It reduces uncertainty about the environment, and in turn allowing more confident decision making.  When designing a MAS, one must consider how the agents sense their environments and how certain the sensing results can be.  This allows the MAS designer to decide how to shape the agents’ reasoning process accordingly.

·         How to design a cooperative MAS?  Is the MAS in Game Day 3 a cooperative MAS?  Fundamentally, it is a competitive MAS since each agent competes with the other when making their local decisions.  However, the emergent, coherent behavior (on Day 2) was actually cooperative.  Without cooperation, most teams would not be able to find all items for the time allocated.  Because of information exchange, all teams solved their search-and-retrieval task.  The “cooperative spirit” was motivated through utility: each self-interested agent was willing to “cooperate” as long as it gained from the “cooperation”.  This is quite commonly done in MASs.  (Note:  on December 3rd, the first seminar of the class will address the notion of cooperative multi-agent learning.  The paper has an interesting and rather similar look at the “cooperative” notion.)  By designing agents this way, one still gives them their autonomy, but “implicitly” leads the system to achieve coherence.

·         When and how to post an information offer or an information need is important.  On both days, I see weaknesses in the postings.  On Day 1, there were too few “NEED” postings.   And thus, most of the pricings on the “OFFER” postings were “unguided” (no “ceiling” on how much one was willing to pay for a piece of information).  As a result, it was a bit of guesswork.  On Day 2, there were more “NEED” postings, but the “OFFER” postings were still the majority.  Because of the “NEED” postings, more “OFFER” postings were priced better.  That also led to more transactions.  When designing a MAS with an open communication protocol like Blackboard (i.e., every agent can see all the postings), the designer must consider how to reduce “ineffective” message traffic.  What are “ineffective messages”?  These are postings with pricings outside of a common zone between a seller and a buyer.  Key is to motivate the agents to post what they need and what they can offer in a timely fashion.  The Game Day actually had that design; though not apparent on Day 1, it was in effect on Day 2. 

5.  Game Days League

 

Teams

Auction Day (10/29-10/31)

Contract Day (11/14-11/16)

Cooperation Day (11/28-11/30)

Total

Team Win

2

1

5

8

Rocky & Bullwinkle

1

3

3

7

Boole’s Fools

4

2

1

7

The Flood

3

4

6

13

Mongii Agent Systems

6

5

4

15

Nick

5

6

2

13

 

There are two winners of the Game Days League: Rocky & Bullwinkle and Boole’s Fools.  Both finished with 7 points each.  Team Win finished third with 8 points.  The Flood and Nick finished tied at 4th with 13 points each.  Mongii Agent Systems finished 6th with 15 points; however, they improved their ranking two Game Days in a row.

 

Figure 1.  Ranking of each team for the three game days.  Team Win made the biggest drop from Contract Day to Cooperation Day.  Nick made the biggest jump from Contract Day to Cooperation Day.  Both Boole’s Fools and Mongii Agent Systems improved from the first game day to the third game day.