Collaborating on Research Projects

     So far, we've looked at collaborative museum projects in which the outcome was an exhibition or educational program. But what about museum research? Visitors and students can collaborate with institutions on research and data analysis projects, from working with collection materials to performing knowledge creation tasks. Let's look at two collaborative projects in which museum users partnered with institutions to perform research: WIkipedia Loves Art and the Children of the Lodz Ghetto. Both of these projects used web-based tools for the collaboration, but they involved a range of onsite and online relationships and activities.

     Wikipedia Loves Art was a month-long event in February, 2009, in which fifteen museums (mostly art institutions) partnered with the Wikipedia community to invite people to take photographs of art pieces that could be used to illustrate Wikipedia articles. While to its many consumers, Wikipedia is a resource, it also is home to a thriving community of participants who are passionate about making the world's knowledge freely available to all internet users. While the user activity in Wikipedia Loves Art was a simple contributory action (taking photographs of objects and uploading them to Flickr), the process was collaborative because the outcome images were for use by the Wikipedia community, not the institutions.

This was in many ways an uncomfortable collaboration. The Wikipedia community (or Wikimedians, as they often call themselves) is passionate about "liberating" cultural content to be digitized and published online using as open a licensing structure as possible. Museums, on the other hand, are often concerned about losing control of images of their collections, even those that are in the public domain, for fear that the images will reflect poorly on the objects themselves or be taken out of context. To make the Wikipedia Loves Art collaboration work, the museums developed careful rules about what could and couldn't be shot, and how participants could upload their images for use by the project.

     To avoid a conflict of interest in which museums would "pump" Wikipedia with content of the museums' choosing, the museums asked representatives of the Wikipedia community to provide the institutions with lists of thematic topics that required illustration. Museums used these thematic lists to develop scavenger hunt lists to distribute to participants so that they might find art objects to illustrate Wikipedia topics like "Roman architecture" or "mask." ( Great example of providing a channel or framework for participation. SB) The institutional project leaders encouraged Wikimedians to form teams and created an online game platform to support competition among teams participating at different institutions around the world. In this case, unlike in the case of The Tech Virtual, the competition was a positive motivator that encouraged individual teams to see themselves as part of a worldwide project. (Any evidence that more people participated because there was an opportunity to work in teams? SB)

     Wikipedia Loves Art was an incredibly decentralized effort, and that led to some confusion about how to participate. From the institutional perspective, the best way to deliver good participant experiences was to constrain contributions to very specific collaborative platforms and structures (the scavenger hunts and Flickr). But there were many participants who were confused or frustrated by what they perceived as arbitrary institutional constraints. Some people found their own rogue ways to upload museum images outside of the project framework, much to the consternation of museum representatives, who saw these actions as causing more confusion, not more opportunities to participate. Worse, as in The Tech Virtual, the institutions managing Wikipedia Loves Art had to make changes throughout the process to the scavenger hunt lists and scoring mechanisms, which caused additional confusion and prolonged discussion among participants and institutional representatives on Flickr. In a blog post reflecting on the experience, Shelley Bernstein of the Brooklyn Museum commented that they should have "frozen" all the lists, noting that "changes are enough to drive participants off the deep end." (Demonstration of amazing investment on the part of the participants that they could be 'driven off the deep end."Coudl be viewed as sign of success. SB)

     The participatory experience was confusing and frustrating for some Wikimedians, but that wasn't the end of the project. The staff work required once contributions were received was massive. Over 13,000 photographs were submitted by 74 participating teams at the fifteen different institutions, documenting about 6,200 pieces of art. While these 102 photographers had done the hard work of capturing the images, it was up to the institutions to validate, tag, caption, and prepare them for Wikipedia's use. This was a herculean effort, and some institutions found themselves unable to deal with the data received in a timeframe that also accommodated the participants' desire to see the fruits of their labor. As Brooklyn Museum data processor Erin Sweeney explained, she used a ten-step process for determining whether an image was a valid contribution. After determining validity, Sweeney added tags to the images to identify the objects with which they were associated, the number of points the team received for the images, and more. Eventually, all of the work was completed, but when the dust settled, the overall effort for institutions involved in Wikipedia Loves Art was so great that many saw it as an unsustainable collaboration.

     Wikipedia Loves Art is a good example of a collaboration that has all the right elements: a scalable, distributed data collection model, a goal that is appealing to institutions and participants alike, interesting work for participants to do, and an institutionally-managed platform to keep the project on track. But ultimately, the project was overambitious, and the institutions involved found themselves overwhelmed by the amount of work required to manage what had seemed at the outset like a simple partnership. The short time frame helped institutions see Wikipedia Loves Art as an experiment and quickly learn from the challenges of the collaboration. The one-month campaign model allowed this experiment to succeed without causing too many headaches and allowed institutions and participants alike to reflect on what worked and didn't. (Did the institutions really not anticipate the number of potential contributions and the level of effort required to sort them out? Sounds like the real need is for a real project manager. An experiment does not mean you just throw out some bait and see what happens. SB)

(Another question: after the experiment ended, did the institutions determine how to make it work without the giant demand at the end? Is there a way to avoid the extensive post-curation? To narrow the contributions with more specific criteria? To change the scale, change the control point? SB)

     In contrast, around the same time, the US Holocaust Memorial Museum launched a small pilot of a collaborative research project that was tightly controlled and was being rolled out slowly on an institutionally-driven schedule. The Children of the Lodz Ghetto, like Wikipedia Loves Art, is an artifact-based project, but the similarity ends there. For many years, the Holocaust Museum has provided research tools and services for researchers and survivor’s families who would like to access survivors' registries or find out more information about various incidents related to the Holocaust. Staff have used these portals to make specific requests for survivors to offer information or stories related to their experiences, but the majority of the Holocaust's audience are not survivors, and these contributory efforts are small. The Children of the Lodz Ghetto research project was intended to reach a wider audience, and was developed in concert with the museum’s dedication to the use of educational technologies to connect students to the lessons of the Holocaust. The project is an educational program in which students investigate the paths taken by over 13,000 children who signed a school album (an artifact in the Museum’s collection) in the Lodz Ghetto in 1941. Using a subset of the online research databases used by professional Holocaust researchers, the students try to find out what happened to individuals in the album by running a variety of searches on different spellings of names of children across many geographic locations and concentration camps.

     To participate, users must register, and for the first several months of the project, registration was primarily advertised to small groups of university students and teachers. By April of 2009, there were 145 active user accounts. Once registered, users could select a name from the album to research or continue on research started by other users. The database queries are sorted into timeframes (ghetto, labor camps, concentration camps, liberation) so that users can progressively add information about individuals' location and status throughout the 1940s. Eventually, the goal is to have a record of each child's story, starting from those 13,000 signatures from 1941.

     For the institution, the Children of the Lodz Ghetto research project provides valuable information about the children in the album. As the project website says, "Now the museum needs your help." However, this help comes at an incredible (but acceptable) cost. Staff vet every entry in the research project, and in the first few months of 2009, only 26% of user-contributed submissions were validated as accurate. The rest were incorrect or inconclusive. However, despite the fact that staff researchers could have done this research more accurately (and more quickly) on their own, the value of researchers engaging in discussion with participants and helping them learn how to be researchers was deemed high enough to make this project worth the low quality of data submitted.

     The educational experience for participants in the Children of the Lodz Ghetto research project in terms of research skill-building and content learning has been very high.  Additionally, performing research increased participants' emotional engagement and perspective on the Holocaust, as many commented that they now had tangible, specific people and incidents to connect to the horror of the time.

     Participants noted in particular how much they enjoyed and learned from commenting on each other's research and receiving feedback from staff and other participants alike. In an evaluation, one participant commented that, "Having their help made this project less stressful and made it feel like we were working as a team. Much of the time, our peers allowed our research to continue without any dead ends. When we were stuck, it was comforting to know that the United States Holocaust Museum and our peers had our backs."

     Museum staff are continuing to tweak the project as time goes on, in particular, to encourage a community of self-motivated, more skilled researchers to sustain the project on their own.(Is this a change in direction after seeing what happened, or was it part of the original intention? SB) David Klevan, director of learning technology, believes that the research can improve in quality and the community can effectively self-police entries if they can grow to a sustainable level (which he estimated as about 1000 registered users). Because the project is built to support and integrate peer review and active collaboration on individual research efforts, it is certainly possible that the Children of the Lodz Ghetto research project will grow into one that is self-sustaining and can provide real value to institutional and research audiences as well as to the participants themselves. If not, it will remain an educational project, one that provides quality engagement but doesn’t necessarily promote users as true partners to the institution.

(What kind of access do visitors to the Holocaust Museum have to this project? What is their response as observers, rather than active participants? SB)

Continue to the next section, or return to the outline.