Performance of distributed problem-solving networks

Addressing the uncertainties that surround the coordination and performance of 'Distributed Problem Solving Networks' (DPSN), as well as the areas in which these new Internet-based forms offer advantages over more familiar modes of problem-solving.

Contact:

Professor William Dutton

Tel: +44 (0)1865 287210

Email: william.dutton@oii.ox.ac.uk

Overview

There has been a growing 'buzz' in the business press about exploiting 'the wisdom of crowds', and executives in the public and private sectors are beginning seriously to consider the future potential of these developments to transform the organisation of information-intensive work in a network society. But many uncertainties surround the performance of 'Distributed Problem Solving Networks' (DPSN) and the coordination mechanisms they employ, as well as the areas in which the new forms that have emerged on the Internet offer distinct advantages vis-a-vis more familiar sources of expertise and modes of problem-solving.

Goals and Approach

We have taken an exploratory approach to understanding the workings of these networks by preparing a set of case-studies within the areas of computer software, e-science, and media production (encyclopedias and open content films), news and opinion aggregators - including prediction markets. We have also examined new business models that involve creating Web 'platforms' that gather information that forms a basis for analysis and brokerage services that will be valuable to third parties, and the internal application of games that could enable a firm to modify the communication behaviors of its employees.

A major goal of this project has been the articulation of a framework and a synthesis of concepts and approaches that can be applied more broadly to new DPSN forms that are continuing to emerge on the Internet. Such a framework and the illustrations of the ways in which it can be implemented is an important step toward systematic measurement of the performance of these 'network organisations'. It has produced the analytical basis for guiding the decisions of managers in the private and public sectors about allocating resources to DPSN mechanisms, and equally, the leaders of volunteer-communities that are considering the formation of commons-based peer production of new cultural products and information goods.

The framework and the concepts have been discussed and developed during a series of workshops with McKinsey and the academics involved in the project. The workshop series ended with a public lecture 'The Power of Diversity: The Performance of Distributed Problem Solving Networks' at the Said Business school (31 Jan 2008), which brought in an outside perspective from the distinguished speakers Scott Page, John Wilbanks, and Karim Lakhani. A webcast of the event is available.

This project contributes to the OII's efforts to provide an informed basis for a public discussion of the changing nature of work in a networked society, as well as the longer-term societal implications for people's lives and livelihoods in societies where a multitude of experiments in knowledge-sharing for co-creation and problem-solving are being facilitated by the availability of high-speed Internet access to digital content.

I. Project Framework, Synthesis and Commentary

The lecture 'The Power of Diversity: The Performance of Distributed Problem Solving Networks' along with invited responses was designed to inform the debate about the potentialities and limitations of these new organizational forms. View the webcast of the lecture and discussion.

II. The Case Studies

1. Sermo: A Community-Based Knowledge Ecosystem

Exemplary of Web 2.0 developments, some problem-solving networks are anchored in user-generated content, such as Sermo: a community-based knowledge ecosystem for licensed physicians in the USA. Physicians can ask and answer questions and surveys posed by other doctors or pharmaceutical firms or other paying problem-holders. The Sermo community, of over 50,000 doctors, sorts through conversations and identifies interesting health trends, cases, and other novel health insights for the benefit of multiple stakeholders.

2. Seriosity: Addressing the Challenges of Limited Attention Spans

Human beings like to earn points, as demonstrated by Seriosity, a creative use of games as way to create an incentive for individuals to pay closer attention to their use of email and solve their problems with information overload. The system enables individuals to use a virtual currency attached to email, to simulate the redistribution of resources in ways that will lead them to be more strategic about the priority messages they send, read, open from co-works. The Seriosity approach has interesting side-effects, to include individuals exchanging the virtual currency for real-world tasks and favors in the workplace.

3. The Performance of Distributed News Aggregators

Originating as a study of the online news aggregator Digg, which relies on user ratings to determine which articles to put on the frontpage, this study evolved into a comprehensive survey of the online news aggregator space. This case study explores the conditions under which crowds are smart, analyzes the bias of several modes of information aggregation, and shows the risk of mob behaviour.

4. Information Markets: Feasibility and Performance

The performance of prediction markets has been one driving force behind the renewed attention on distributed problem solving. This case reviewed the feasibility and performance of prediction or information markets, discussed some apparently successful applications, considered likely limitations of information markets, and identified important areas for future research.

5. The ATLAS Collaboration: A Distributed Problem-Solving Network in Big Science

The nature of the problem tackled by the ATLAS collaboration – the creation of a radically innovative particle detector experiment - makes ATLAS an exceptional case for studying DPSNs. The problem solving is distributed across multiple groups of problem solvers comprising 2000 scientists in 165 working groups across the globe. Similarly, the engineering, construction and installation of the many components is distributed across this collaborative network. The initially surprising finding of the case study is that this joint innovation effort succeeded despite breaking with most rules of traditional project management. Philipp Tuertscher analyzes what it took to make such a loosely structured organization work, and raises the question if such structure was even required to develop a complex technological system like the ATLAS detector?

6. Bug-Patching for Mozilla's Firefox

The case study of the Mozilla project is focused on the organization of quality control and quality assurance in a distributed innovation environment, and focuses on the coordination of the detection and correction of operating defects ('bugs') in Mozilla's Firefox web-browser. Analyzing two samples of bugs drawn from the 40,000 or so that have resulted in a change to the Firefox code base, the case study finds among others that bug treatment behavior in the project was not temporal stable, that bug reports from 'outsiders' took longer to find a successful resolution and were more likely to remain 'un-fixed and that factors such as the objective technical complexity of the bug-patching problem, and the level of effort devoted to the contextualisation the reported defect played significant roles determining the speed with which a bug was typically fixed.

7. Wikipedia as a Distributed Problem Solving Network

Wikipedia, the free online encyclopaedia put together by volunteers, is a prime example of a distributed problem-solving network with a global array of contributors creating a resource that has been compared to leading encyclopedia. The study focused on efforts to maintain the quality of Wikipedia entries and in particular of the use of tagging to signal the need for improvement in entries of Simple Wikipedia.

8. Distributed film production: Artistic experimentation or feasible alternative? The case of 'A Swarm of Angels'

A Swarm of Angels was selected as a case study of open content film production. The project based in Brighton, England, but it extended the open source model to movie making in ways that could bring distributed collaborators into the film project. The case study highlighted the existence of a core group of contributors and a periphery of silent supporters, which both play an important role in the project's performance.

III. Cross-Cutting Thematic Papers

Support

This Project was supported by the McKinsey Technology Initiative. We are particularly grateful to James Manyika, Brad Johnson and Michael Chui, and other McKinsey colleagues.

People

Project Leads

Researchers

  • Dr Robert Ackland, Australian Demographic and Social Research Institute, Australian National University

  • David A. Bray, Goizueta Business School, Emory University

  • Irene Cassarino, Politecnico di Torino

  • Karen Croxson, Department of Economics, University of Oxford

  • Professor Jean-Michel Dalle, Université Paris VI and IMRI-Université Paris Dauphine

  • Dr Matthijs den Besten, Oxford e-Research Centre, University of Oxford

  • Tobias Escher, Oxford Internet Institute

  • Dr Aldo Geuna, Science and Technology Policy Research (SPRU), University of Sussex, and Department of Economics, University of Torino

  • Max Loubser, Oxford Internet Institute

  • Dr Jukka-Pekka Onnela, Department of Physics, and Saïd Business School, University of Oxford

  • Felix Reed-Tsochas, James Martin Institute and Saïd Business School, University of Oxford

  • Wolf Richter, Oxford Internet Institute (Project Coordinator)

  • Philipp Tuertscher, Institute for Innovation and Entrepreneurship, Vienna University of Economics and Business Administration

Webcasts

  • Performance of Distributed Problem Solving Networks

    Performance of Distributed Problem Solving Networks

    Recorded on: 31 January 2008 Duration: 01:38:53

    How do we exploit 'the wisdom of crowds'? This event explored the potentialities, limitations, performance and coordination mechanisms of the 'Distributed Problem Solving Networks' which are transforming the organisation of information-intensive work.