• Thomas
• Hao
• Wei Wei

1. Image annotation, region segmentation, CAPCHTA
2. Tagging of multimedia contents, e.g., music, video, images, etc.
3. Social gaming

* Why Human Computation is needed?

• Human is much better than computers when solving some problems. (Image labeling, segmentation, natural language understanding, etc.)
• Other reasons?

• What (types of) tasks is suitable for Human Computation to solve?
  • Tasks that:
    • Computers are poor at, and
    • Human excels
  • Others?

• What characteristics make a task suitable for Human Computation?

• What are the major Components in Human Computation?
  • Human
  • Problem to solve
  • etc.

• How human can contribute/involve in the Human Computation process/algorithm?

• How to motivate human to contribute?

• What / How does a Human Computation algorithm looks like?

• Types of Human Computation?
  • Human Side: Single-player, 2-players, multi-players or Zero-players?
  • Relationship among players:
    • Do they know each other?
      • They know exactly who the others are? or
      • They just know that the opponents are someone they know
      • They know nothing about the others?
      • Combination of the above?
    • Trust the others? (Friends or enemies in social network?)
    • How does the relationship affect the Human Computation process?
  • Types of human involvement?
    • Game
    • Test/Quiz
    • Paid Work
  • The purposes / ultimate goals of the human computation algorithm

• How do local goals (human objectives) and meta goals (global objectives) relate to each other?

• Should players aware of the meta goals?
  • How this will affect the Human Computation process?
    • Cheating?
    • Abusing?

• How to prevent players from being misbehaved (Anti-Cheating)?

• How can we verify player supplied data?
  • Symmetric Verification Game
  • Asymmetric Verification Game
  • Formulate the probabilities that a collected result is bad?

• Distributed Computing Issues
  • Scaling? Can this problem be scaled up to allow more people to work together?
  • Incremental improvement? Some problems involves data that change in time, can we cope with that?

• Related Terms
  • Social Gaming
  • Social Algorithm
  • Human(-based) Computation on Wikipedia
  • Wisdom of Crowds on Wikipedia
  • Crowdsourcing on Wikipedia
  • Collaborative Intelligence on Wikipedia
  • Volunteer Computing on Wikipedia
  • Symmetric Verification Games
    • Constraint is number of outputs per input
  • Asymmetric Verification Games
    • Constraint is number of inputs that yield the same output
    • Luis describes peekaboom and verbosity as “asymmetric verification games”: Input is given to player 1, whose output is sent to player 2, who has to guess the input given only player 1's output. Once he guesses, this verifies the connection between the output and the input. This is in contrast to the ESP Game, which is a symmetric verification game. But symmetric games work only when there is a constrained list of possible outputs.

* Existing Games / Applications

• ESP Game ESP Game on Wikipedia
• Google Image Labeler
• Peekaboom
• Matchin
• Squigl
• Tag a Tune
• reCAPTCHA reCAPTCHA on Wikipedia
• Verbosity
• games with a purpose

• Luis von Ahn
  • http://www.cs.cmu.edu/~biglou/
  • http://www.cs.cmu.edu/~biglou/CV.pdf

• Papers
  • Designing games with a purpose
  • Games with a Purpose
  • Invited Talk: Human Computation
  • Labeling images with a computer game
  • Peekaboom: a game for locating objects in images
  • Volunteer Computing Using Casual Games
  • Improving accessibility of the web with a computer game
  • Verbosity: a game for collecting common-sense facts
  • Internet-scale collection of human-reviewed data
  • Common Consensus: a webbased game for collecting commonsense goals
  • Why do tagging systems work?
  • Productive Play: Participation and Learning in Digital Game Environments
  • TAGATUNE: A GAME FOR MUSIC AND SOUND ANNOTATION
  • Improving Image Search with PHETCH
  • WebInSight:: making web images accessible
  • A fuzzy constraint based model for bilateral, multi-issue negotiations in semi-competitive environments

• Articles
  • Terra Nova: Games with a purpose
  • The rise of crowdsourcing
  • Crowdsourcing as a Model for Problem Solving
  • Google Image Labeler, the ESP Game, and Human-Computer Symbiosis

• Videos
  • Human Computation - Luis' Talk in Google
  • WebInsight Making Web Images Accessible
  • Computers versus Common Sense * selected as a Google video of the year
  • Google Talk on Mechanism Design (Game Theory)

• Organizations / Projects
  • Open Mind Initiative Publications
  • The Open Mind Common Sense Project
  • Distributed Computing Projects Distributed Human Computing Projects

• Conferences
  • Conference on Human Factors in Computing Systems

• Books
  • Fundementals of Multiagent Systems