A Messy Collaboration: The Tech Virtual Test Zone

     These lessons may sound obvious, but I learned them through a series of projects that were poorly structured with unclear goals and shifting criteria. I want to share with you the story of The Tech Virtual Test Zone, the collaborative project that taught me about the essential role of clear structure and criteria to co-created projects.

     The Tech Virtual Test Zone was a project of The Tech Museum of Innovation in San Jose, CA. In the fall of 2007, I joined the staff of the Tech Museum to help lead an initiative called The Tech Virtual, of which the Test Zone was the pilot project. The goals of The Tech Virtual were clear. As a hands-on science museum focusing on technology, the director deemed it inappropriate for us to use traditional multi-year processes to develop new exhibits. If it takes you two years to develop a new exhibit about technology, the thinking goes, the exhibit is already obsolete by the time it hits the floor. The Tech Virtual was a project designed to solve this problem in an innovative way: by crowd-sourcing exhibit development to participants all over the world via web-based and virtual world-based collaborative platforms. 

     We used a virtual world platform called Second Life to invite participants not only to share their ideas in words but to prototype three-dimensional, digital versions of their proposed exhibits. The idea was for the community to develop many exhibits in parallel virtually, and then for the Tech Museum to select the best of these for fabrication in real life. In addition to speeding up the process, we hoped we would attract participants who were content experts or at least passionate amateurs about the content, whose experience and ideas would broaden the limited exhibit development abilities of the staff (the exhibition development and education departments had recently been cut). On October 31, 2007, I joined the staff as curator/exhibit developer/community manager. We opened the collaborative digital platforms in December, offering would-be participants training in exhibit development as well as a cash prize for the best exhibit ideas. Over the next six months, I worked closely with about 200 participants worldwide to develop their ideas, and on June 4, 2008, we opened a real-world 2,000 square foot gallery presenting seven interactive exhibits on the topic of "art, film, music, and technology," all of which originated in collaboration with twelve external partners, ranging from a Toronto bartender to a Manchester accountant to a local music teacher. While the museum did not do any formal evaluation of the exhibition, the exhibits have been generally well-received by visitors. Because The Test Zone was a pilot of a collaborative process, we focused less on the audience outcomes of the products than on the participatory process. (This seems to be a mistake, as you have earlier noted that collaborative process is a tool. This tool was to be used to develop exhibits that were more relevant and more cost-effective in light of decreased staff. An exhibition was always the intent. SB)

     From an institutional perspective, the desired outcome was clear, but the the collaborative process with participants that took us from December to June was anything but [clear?CR].  The Test Zone was an experimental pilot, and rather than setting fixed goals, participant criteria, and outcomes at the outset, they evolved over the experiment. In some cases, we changed course based on community feedback, but in many instances, museum leadership dictated changes in schedule, budget, and desired outcomes, and I scrambled to adjust the project accordingly, not always to the benefit of participants. While it's easy to say, "this is an experiment," it's hard to build trusting relationships with people who might be affected adversely by the various changes that every experimental project undergoes. When we changed something, we were changing it on people. Fortunately, by maintaining honesty and open communication with participants, the majority were willing to weather the changes and stick with the project. Much as the 826 tutoring staff use an imaginary authoritative publisher to establish rapport between staff and students, I shared my own challenges and frustrations with the Test Zone participants, honestly sharing the ways that I, like them, felt tugged around by the chaotic process.

     The Tech Virtual Test Zone was an experiment in crowd-sourced exhibit development. From the outset, I tried to balance offering participants an open slate for their creativity in terms of exhibit ideas with a highly structured and staff-supported process for turning those ideas into viable exhibits. We offered participants many resources to support their exhibit development. On the web, we set up templates for participants to flesh out the big idea, content, look and feel, interaction, and key objects/technology associated with each exhibit concept. In Second Life, I produced three-dimensional interactive walk-through tutorials on how to design interactive exhibits that are engaging, accessible, easy to use, etc.--a virtual exhibit on exhibit-making. I worked with contractors and volunteers to design a few simple open source Second Life exhibits on the theme of technology in art, film, and music, which served as working models for participants, many of whom had not spent much time in interactive science centers. And most importantly, we provided participants with a sandbox, where anyone could build anything at any time, and dedicated workshop space, where individuals could house their projects-in-progress. The virtual workshop became the home base for exhibit development and at its high point, was a thriving, active room filled with people working on diverse projects.

     Because Second Life is a social environment, users can talk to and work with each other in real time, and we quickly discovered that interpersonal interaction, not tutorials or templates, was the key to motivating participants and encouraging them to develop their skills. We offered Second Life-based exhibit design classes twice a week, which blended basic Second Life building skills with exhibit thinking. These classes were a kind of marketing outreach. They helped promote the project to new people who may have been enticed by the chance to learn how to create a color-changing sculpture and then found themselves interested in designing whole exhibits. For invested participants, we hosted a weekly exhibit designers' meeting in Second Life in which community members sat at a large virtual table to discuss the challenges in their projects, new developments in the Test Zone project overall, and whatever else was on their minds. These meetings only attracted a small percentage of the community (about 10-15 people per week, compared to about 100 in the workshop at any time), but these participants tended to be the most motivated folks who often informally volunteered their time to greet new workshop denizens and help out wherever they could.

     Because things kept changing both in their own projects and at The Tech Museum, these meetings became an essential way to keep lines of communication open and address any problems in the system. Many participants who attended the meetings were highly networked within the larger Tech Virtual community, and they were able both to bring in problems from the community and share information back to less-engaged participants. There were several times participants solved their own problems at these meetings; for example, when we needed a better way to delineate the "frame" of each exhibit in the workshop, participants rapid-prototyped sample virtual frames and voted on the best one to distribute to everybody for use. (

(Agreed that this gray section could be deleted. SB)

     In their research on public participation in science research, Rick Bonney and others discovered that this model of "core" community members who work closely with staff, combined with secondary level community members who contribute at a more basic level, is successful for collaborative and co-creative projects. As you’ll see in the next section on co-creation, this model is also employed at the Wing Luke Asian Museum for their comprehensive community exhibit process. Could delete this section. SB) In the case of The Tech Virtual the core group was self-defined, which made them more effective than a group pre-selected by staff may have been.

     While some Tech Virtual participants showed extraordinary community spirit, going so far as to offer their own classes and pitch in with "office hours" in the workshop, the overall collaborative goals were greatly compromised by the inclusion of a contest with cash prizes. We awarded $5000 to each exhibit design that was translated to real life. Doing so meant we could raise awareness very quickly, which was useful given the short time frame. It also focused the experience. People weren't coming to The Tech Virtual to muse about exhibits; they came to build exhibits on a deadline for submission to the contest. However, the contest also prevented us from legitimately fostering collaboration and community. People were unsure whether they should go it alone (and try to win the whole prize) or team up with others. We had several community discussions about the competition disincentivizing collaboration, and I fielded bizarre but understandable questions about whether participants should try to get involved with as many exhibits as possible to optimize chances of winning, or do only solo projects to maximize potential reward. The money sent a contradictory signal to all our talk about community.

     This problem was most acutely felt in our experimental and limited work with teens in the Teen Grid of Second Life. We focused our efforts in the "adult grid" of Second Life and primarily worked with participants who ranged from their 20s to their 50s (with the majority trending older). I also created a scaled-down version of the walk-through tutorial and workshop in the Teen Grid of Second Life, but we spent less time working with teens than with adults. This was partly due to the restricted role of adults in Teen Second Life (most programs need to be teen-led to be successful) and to a late entry into the space. But we also found that teens were less self- and community-motivated and desired more staff attention than we were able to give. Unlike the adults, who appreciated connecting with staff on a regular basis but were happy to work on long-term projects on their own, the teens wanted short-term goals, lots of staff feedback, and were generally more competitive (and less collaborative) with each other. They were obsessed with the contest and the cash. They also expressed several times that it was not fair that their work was being evaluated in the same pool with adult work; we were surprised by the extent to which they felt themselves inferior to the (often bumbling) participants in the adult grid. In response to these concerns, we added a special "teen prize" to ensure that at least one teen entry would be honored in the final exhibit design contest.

     The contest not only caused problems for collaboration among participants in the Test Zone--it also complicated the collaboration between staff and participants. Staff and participants could not be true partners when one was in a position to judge the other. As the community manager, I had to maintain positive, encouraging relationships with all of our virtual exhibit designers. To do so, I had to mask the reality that I was also the primary arbitrator of which exhibits would be selected for inclusion in the physical exhibition. I couldn't be both the person who cheered participants on and helped them consider how their ideas might become appealing interactive exhibits and the person who told them their exhibit wasn't good enough to win. So I hid behind an imaginary panel of judges, invoking them to tell participants that "the judges didn't understand this part of your project," or, "the judges don't believe this would be feasible in the real world to fabricate." Using this device, I was able to keep working with the participants and encouraging them throughout the process as their partner, not their evaluator. But obviously this was far from ideal from a trust or honesty perspective.

(This section is important to keep. It clarifies the need for a 3-part system noted in last section. Can't play the two roles credibly or effectively. SB)     The contest caused one final problem that relates to honesty: we could not easily align a clear, fair contest structure with the goal of developing seven interactive exhibits in five months. As museum professionals know, not every exhibit that sounds like a great idea can actually be built successfully, and conversely, some great exhibits emerge from hazy and unpromising beginnings. In some cases, such as an exhibit called "musical chairs," our internal team of engineers were able to quickly identify the concept as a winner from a simple one-paragraph description of the concept. While Leanne Garvie, the participant who contributed that concept, did build a working (and quite fun) virtual prototype in Second Life, it bore little similarity to the real-world version we designed in parallel at The Tech.

     Garvie’s idea was definitely a winner, but the criteria that made it a winner were different from those that governed other virtual exhibits which only emerged as winners based on progressive work on the virtual prototypes that eventually led us to see how they could be successful real-life installations. In the end, we gave the $5,000 award to each exhibit that was built in real life, but we also gave lesser prizes ($500 and $1000) for outstanding virtual-only projects to address the issue that many participants contributed a lot of time and good ideas without winning. Our criteria for awarding the $5,000 prize remained consistent throughout the project--top prizes would be given to those people whose exhibits were developed in real life--but once we really started working with participants, it became clear that the underlying criteria used to determine winners was not easily quantifiable to contestants' satisfaction.

(Keep this section. Powerful demo of the need for clarity in criteria for evaluation. SB)

     In the beginning of the project, the museum director would speak about "copying" exhibits from Second Life to real life. The theory was that we would hold a contest with staged judging, and at each judging point, we would select fully completed virtual exhibits to "copy" to the real museum. Our fabrication team quickly realized that this was unrealistic, both technically and conceptually. Once we realized that virtual exhibits would not translate directly to the real world, we transitioned to a model where the real exhibits were "inspired by" the virtual. In all cases we chose superlative virtual exhibits in which the core idea was powerful enough to transcend platforms. We maintained that core idea in the real version of the virtual design, and tried as much as possible to retain other aspects of the virtual designers' goals in recreation.

     We didn't leave our virtual designers out of the ensuing real-world process, though at that point the Tech Museum staff asserted the upper hand in the collaborative relationship. (This sounds parallel to the shift in responsibility noted in the Yup'ik exhibition. Typical for the institution to hold the fabrication and installation process. It was even stated at the start. SB) In many ways, the collaboration became easier for museum staff when we moved to fabrication, because we knew how the process would work and where we could and couldn't integrate input from the participants themselves. In cases where participants were local, they often came in to check on our progress and even helped put their exhibits together. For those who were hundreds or thousands of miles away, I shared our real-world progress in virtual meetings, photos, calls, and emails. (This kind of contact across geography was also present for Yup'ik advisors. SB)

     Wherever possible, we asked participants to provide their own or preferred content for exhibit artwork, audio, and video. All final exhibits featured a didactic label about the core art/film/music/technology content as well as a second label about the virtual designer and the collaborative process. Three exhibits featured original art and music by the virtual designers, and (overlapping) three relied heavily on the technical expertise of the virtual designers. They enabled our engineering and fabrication team to go beyond our in-house capabilities to tackle some exhibit components and or content elements that we could not have produced in this timeframe.

     We also branched out of the virtual-to-real process to solicit amateur content. One of the exhibits features video of original paintings being created. To produce that content, I put an ad on the online forum Craigslist and invited artists down to The Tech Museum to be videotaped while creating art. One of these artists, a graffiti artist named Dan, had such a good time that he came back to the shop with friends several times to do more graffiti with us on his own dime. [CA - 'his own dime' I get the sense but that's unfamiliar language for a Brit reader]. Dan had lived in the San Jose area all his life but hadn't visited the museum since he was a child. He formed a new relationship with staff and arrived at the exhibit opening overwhelmed with excitement to see his piece in the museum.

(Agreed that this gray section could be cut. SB)

     Overall, the Test Zone experience was an exciting and frustrating one for staff and participants alike. In some ways, this made us great collaborators because everyone was dependent on each other to complete the project in such a short time frame. But the openendedness and chaotic nature of the project did not make for the foundation for a sustaining community of amateur exhibit developers. There was no way for participants to rely on each other; they had to rely on me as the conveyor of changing information and criteria for success. We formed an unhealthy community that revolved around me as the community manager, exhibit design mentor, and institutional representative. When the Test Zone opened in real life, I ended my time with The Tech Museum and The Tech Virtual. The community did not survive after I left. (Was that really one of the criteria of success?  All experiments can not and do not meet all criteria. Is it possible to send a note out to all participants and see how this collaboration affected the course of their work and lives? Would be interesting collection of stories, I suspect. SB) We'll explore this issue further in the next chapter on managing and sustaining participatory projects, but clearly, communities that are dependent on one central figure are not sustainable. (Good clear and well-demonstrated conclusion! SB)

Finally, The Tech Virtual Test Zone demonstrated some of the barriers that "regular" people feel to collaborating with institutions. Despite all of our efforts to be as friendly and open as possible to would-be collaborators (and our desperate desire for their participation), many found the idea of working with a big museum overwhelming. When the Smithsonian American Art Museum ran a game called Ghosts of a Chance that relied on participant-contributed art, staff member Georgina Goodlander was comparably surprised that some people reported being unsure of their ability to "live up" to the standards of the Smithsonian.

     In the case of The Tech Virtual, the use of Second Life as an exhibit development platform helped ameliorate some of this threshold fear. This may seem paradoxical, since Second Life is itself a very high-barrier to entry, hard-to-use software platform. But many of The Tech Virtual participants were much more proficient in the Second Life environment than I was, and that virtual space was a place where they felt confident in their skills as designers and builders. Not only was Second Life a comfortable, familiar place for them to engage, it was a place where my authority as the museum exhibit designer came down a notch and we became individuals bringing different skills to the table. As participant Richard Milewski, commented, "Second Life is an abstract enough environment that the somewhat intimidating prospect of attempting to collaborate with an institution such as The Tech was made to appear possible. ‘After all, it's not real! It's just a cartoon on my computer screen and I could always just turn it off.’ (Not really... but I told myself that more than once). " Later, when several of the virtual participants came to the opening of the real world exhibition, we offered them a tour of the fabrication shop where their exhibits were made. While a few people were enthused, several were strikingly overwhelmed or uncomfortable in the shop space. It became immediately apparent to me that these were not people who would have ever engaged with us as exhibit developers had it required them coming to the actual museum or the staff design area. By meeting them on "their own territory" in Second Life, we tipped the scales in favor of a positive collaboration. (My vote is to keep this last gray section. You employed a collaborative tool, Second Life, that was an effective way to cross the barrier to entry. This is worth noting. A shorter book is not necessarily a better book. SB)

(I would like to see a summary of lessons learned as final paragraph. There is much to be gleaned. And I really hope you send an email to all participants to hear their observations of what has changed since they participated in the Tech's experiment. After all, isn't this book also one of the outcomes speaking to the necessary and sufficient conditions of successful visitor-participation? SB)

Continue to the next section, or return to the outline.