Reserve your tickets online in advance and join us for our brand-new exhibits, Arctic Adventure: Exploring with Technology, Engineering Design Workshop, Powered by Mathworks, and more!
View Today's Schedule
This guide is designed to help staff at informal science education organizations, and others who are interested, to develop, implement, and evaluate activities and events that incorporate the multi-directional dialogue and mutual learning at the heart of public engagement with science.
When the Museum of Science (MOS) decided that technology and engineering education should be an equal partner to science education in its programs and exhibits, it raised questions about how museum activities would address social values along with the relevant science. The American Association for the Advancement of Science (AAAS) publication Science for All Americans, published in 1989, noted that “Engineering decisions, whether in designing an airplane bolt or an irrigation system, inevitably involve social and personal values as well as scientific judgments.” We found an answer to those questions in 2002 when a group of researchers from North Carolina State University presented their research about “citizen consensus conferences” at the AAAS Annual Meeting that year. In these citizen consultations, patterned a er work done by the Danish Board of Technology to get citizen input into technology policy questions, the organizers provide the science content and the citizens bring their values. The program is not a lecture or series of lectures, but a set of conversations in which both the scientific and technological experts and the public participants bring their different expertise to the table, along with their values, personal experience, and perspectives.
In 2008, the Center for the Advancement of Informal Science Education, under the leadership of Ellen McCallie, convened an inquiry group on PES in ISE (public engagement with science in informal science education). Participants included Larry Bell, Tiffany Lohwater, John Falk, Jane Lehr, Bruce Lewenstein, Cynthia Needham, Ben Wiehe, and Ellen herself. The group produced the report Many Experts, Many Audiences: Public Engagement with Science and Informal Science Education.
The report described PES in this way:
Public Engagement with Science (PES) is usually presented as a “dialogue” or “participation” model in which publics and scientists both benefit from listening to and learning from one another—referred to as mutual learning. The model is premised on the assumption that both publics and scientists have expertise, valuable perspectives, and knowledge to contribute to the development of science and its application in society.
The MOS has experimented with public dialogue programs it calls “Forums” since 2003 and has conducted over 100 such programs since then. In 2005, MOS staff joined forces with staff from four other science museums in an activity of the Nanoscale Informal Science Education Network (NISE Net) to co-develop and test Forums on societal issues related to nanotechnology. Partners included the Science Museum of Minnesota, the Exploratorium, the Museum of Life and Science, and the Oregon Museum of Science and Industry.
In 2010, the MOS launched an NSF-funded project (Dimension of Public Engagement with Science) to collect, analyze, and publish a collection of PES case summaries about then-current PES activities in the ISE field. The project also included a workshop involving 50 of the catalog contributors to explore future directions in PES. The group developed nine “Strategic Priorities for Advancing Public Engagement with Science within the Informal Science Education Community.”
In 2014, the MOS was successful at winning NSF support for a project called Multi-Site Public Engagement with Science–Synthetic Biology (MSPES). This project was aimed at
The project focused on the rapidly advancing field of synthetic biology, a topic well-suited to its PES goals because new genome editing technologies were in the news and raising questions about ethical applications. Also, the field is populated with young researchers who recognize the need for and are interested in public engagement. The principal investigators on the MSPES project were Larry Bell, Elizabeth Kunz Kollmann, and David Sittenfeld, all of the MOS; Ti any Lohwater of AAAS; and Natalie Kuldell of MIT and the Biobuilder Foundation. Other key personnel included Kayla Berry, Katie Todd, and Caroline Lowenthal of the MOS; Jeanne Braha and Emily Cloyd of AAAS; Megan Palmer and Kevin Costa of SynBERC; Catherine McCarthy of the Science Museum of Minnesota; Ali Jackson of the Sciencenter; Camellia Sanford and Claire Quimby at Rockman et al; and Gretchen Gano at UC Berkeley. Many others were involved as advisors, project consultants, educational activity developers and their scientist partners, host site leaders and their scientist partners, reviewers of educational materials and this guide, and others who contributed in a variety of ways.
This guide, which is intended to exist as both a designed booklet and a website, is an outgrowth of all the past work described here and most specifically of the MSPES Building with Biology project. It is possible thanks to the commitment and enthusiastic participation of hundreds of professionals who contributed to the findings of the project.
Public engagement with science (PES) is about dialogue between scientific and technological experts and public audiences about societal questions that science can inform but not answer. In making decisions about these kinds of societal questions, social values and personal experience play roles equal to or greater than the one played by science. Rather than focusing exclusively on science itself, PES focuses on discussing problems that communities view as worth solving; the information society needs and wants from scientists; the potential risks, benefits, and consequences of new technologies; and building trust among stakeholders.
The 2017 National Academies report Communicating Science Effectively: A Research Agenda1 notes that “the most widely held, and simplest, model of what audiences need from science communication...is wrong. A common assumption is that a lack of information and understanding of science fully explains why more people do not appear to accept scientific claims or engage in behaviors or support policies that are consistent with scientific evidence....And although people may need to have more information or to have information presented more clearly, a focus on knowledge alone o en is insufficient....What is known now, though, is that public engagement o en is essential for acceptable decisions about science-related controversies.”
This guide is designed to help staff at informal science education organizations and others who are interested to develop, implement, and evaluate activities and events that incorporate the multi- directional dialogue and mutual learning at the heart of public engagement with science.
Chapter 1 (What is Public Engagement with Science?) digs into the fundamental characteristics of PES and provides some examples of programs that incorporate one or more of them.
Chapter 2 (Strategic Public Engagement) focuses on kinds of public engagement and the benefits and outcomes of each for those involved and for the broader society as a whole.
Chapter 3 (Planning and Designing a Public Engagement Event) provides a framework for putting on a PES event for some type of public audience.
Chapter 4 (Planning and Designing a Public Engagement Activity) focuses on creating hands-on activities and forum programs that stimulate dialogue and incorporate the characteristics of PES.
Chapter 5 (Evaluating Public Engagement Outcomes) shares resources for evaluating PES events and activities. It focuses on defining appropriate outcomes for PES and selecting the right methods for measuring those outcomes.
Chapter 6 (Disseminating Public Engagement Outcomes) covers methods of disseminating work of PES projects—both methods for doing PES and publics’ and scientists’ shared views on the particular topics or questions of PES events.
Chapter 7 (Future Directions for Public Engagement with Science) explores opportunities for further development of PES in the informal science education community.
+ - References
By Larry Bell
Public Engagement with Science (PES) is usually presented as a “dialogue” or “participation” model in which publics and scientists both benefit from listening to and learning from one another...premised on the assumption that both publics and scientists have expertise, valuable perspectives, and knowledge to contribute to the development of science and its application in society.1
The National Academies of Science, Engineering, and Medicine define public engagement in the context of science communication as “seeking and facilitating the sharing and exchange of knowledge, perspectives, and preferences between or among groups who often have differences in expertise, power, and values.”2 The Center for the Advancement of Informal Science Education (CAISE) describes public engagement in the context of informal science education (ISE) as focusing on conversations in which scientific experts listen as well as talk to publics who provide input, deliberate, and make recommendations on topics for which publics can contribute useful knowledge from their life experience. Through PES, ISE institutions can help facilitate dialogue to address important science communication goals.
Sometimes confusion arises because the term “public engagement with science” is used to mean a variety of things that are different from the National Academies or CAISE definition, such as overall educational engagement or public participation in scientific research.3 These different uses of the term overlap in a variety of ways and sometimes people in the same conversation are using the term to mean different things. This guide uses the CAISE definition and the following three dimensions of PES, which were also defined in the 2009 report.1
Alan Leshner, former CEO of the American Association for the Advancement of Science4
Sheila Jasano, Professor of Science and Technology Studies at Harvard Kennedy School6
Daniel Yankelovich, public opinion analyst and social scientist5
Science Centre World Congress leaders displaying the “Toronto Declaration”
Photo courtesy of the 5th Science Centre World Congress
In a 2011 Dimensions of PES project survey, ISE organizations provided descriptions of 201 projects that they felt stepped beyond traditional education goals toward PES.8 But analysis of the data concluded that there was a lack of PES elements in the programs as defined in the 2009 CAISE report, especially in terms of how the experts participated,9 and a 2013 review of studies on how scientists view the goals of communicating with publics found that most “tend to favor one-way communication...viewing engagement as chiefly about dissemination rather than dialogue.”10
These examples of informal educational activities move toward PES in one or more of the three dimensions shown in the chart on the previous page.
WHAT THE FOCUS IS:
Choosing a focus that includes societal impacts, values, and priorities is one way to implement PES.
Science theater is one way to incorporate societal impact content. Short plays were used by NISE Net to stimulate conversation about the impact nanoscale science may have on our lives and to get the audience thinking about how we should respond— both individually and collectively—to those potential impacts.11
Science theater is one way to incorporate societal impact content. Short plays were used by NISE Net to stimulate conversation about the impact nanoscale science may have on our lives and to get the audience thinking about how we should respond— both individually and collectively—to those potential impacts.11
Nano and Society posters are another example of a way to add PES content to science festival events like NanoDays that are otherwise mostly about science phenomena and processes.1
WHAT THE PUBLIC DOES: Incorporating participant views, deliberation, sharing, or problem-solving in connection with topics that include societal impacts, values, and priorities is another way to implement PES.
“Exploring Nano & Society—You Decide!” is a hands-on activity in which visitors sort and prioritize cards with new nanotechnologies according to their own values and the values of others. Visitors explore how technologies and society influence each other and how people’s values shape how nanotechnologies are developed and adopted.13
Risks, Benefits, and Who Decides. NISE Net’s Forum programs encourage audience consideration of the societal and ethical implications of science and technology topics. Older youth and adults participate in one- to two-hour facilitated discussions that promote exploration and foster dialogue and deliberation.14
WHAT THE EXPERTS DO: Having scientists listening to publics and welcoming their views, as well as talking to them, in activities sharing of views, deliberation, and problem-solving takes place is another way to implement PES.
Forest Science Dialogues at the Hubbard Brook Research Foundation is a project to develop and test mechanisms for dialogue-based engagement between ecosystem scientists and local citizens in the rural Northeast.15
The Building with Biology project16 was designed to create conversations about synthetic biology between scientists and the public through public events that include hands-on table-top activities and forum programs. The hands-on activities are intended to be facilitated by scientists or educators to promote conversations with the public about the societal implications of synthetic biology.
The Building with Biology forums “Should We Engineer the Mosquito?” and “Should We Edit the Genome? When, Why, and How Much?” give public participants an active role in deliberating together with scientists and science students.17
While the CAISE definition of PES focuses on mutual learning between science experts and public audiences, policymakers may also be interested in the outcomes of informed, thoughtful deliberations by diverse participants.
World Wide Views18 is a global citizen consultation initiative that provides decision-makers with a unique insight into the global public opinion on complex governance issues that are debated and negotiated at global venues such as the UN.
Thousands of citizens in countries around the world have participated in daylong discussion of issues, coordinated internationally by the Danish Board of Technology Foundation and in the U.S. by the ECAST Network.
Members of the ECAST Network 19 collaborated to conduct in-person and online citizen forums to get public views for NASA about their upcoming asteroid redirect mission.
Network members worked with NASA scientists to determine what they would like to learn from public audiences, created engaging ways to inform demographically representative groups of 100 participants, and collected and analyzed participant recommendations to support NASA’s decision- making processes.20
Youth citizen researchers measure air quality near North Station in Boston with a mentor from the Cambridge Public Health Department
Citizen science or public participation in scientific research offers additional possibilities for PES. Some citizen science projects primarily engage publics as data collectors. Scientists use the input but publics are not deliberating on societal impact, values, and priorities, or participating in problem-solving. Co-created projects, however, engage publics and scientists together in responding to specific individual or community needs. By jointly identifying relevant questions and finding meaning in results, co-created projects have more of the characteristics of PES while conducting scientific research.21
+ - References
This chapter introduces public engagement with science in the context of informal science education. It includes a brief summary of the broader field of PES, including mutual learning and the value to each participant and stakeholder.
By Larry Bell
To be most effective, public engagement activities should be designed with a clear understanding of the purpose and goals for the engagement.
These may include specific benefits for the science museum or other engagement practitioner organization, for the scientists involved, or for the larger community or society as a whole. Depending upon the purpose and goals, the design of engagement activities may be different. This chapter focuses on different purposes for PES and how differences may effect the design.
The American Association for the Advancement of Science (AAAS) identifies four kinds of engagement on its website,1 each of which provides a unique set of opportunities, strengths, constraints, and benefits to the organizers and participants.
Because of the specific work and expertise of the authors, this guide focuses on public dialogue and policy deliberation.
EXAMPLE OF PUBLIC DIALOGUE:
The Building with Biology 2016 physical kit contains six hands-on activities that are designed to be facilitated by scientists or educators to promote conversations with public audiences about the societal implications of synthetic biology.
EXAMPLE OF POLICY DELIBERATION:
This forum encourages in-depth discussion about the potential ways CRISPR, a new technology for genetic engineering, could be used across a variety of applications. Participants learn about genetic engineering and the related ethical and societal dimensions, along with several ways scientists are considering using it. In small groups, they consider and make a plan about whether and how to bring this technology to the world.
The following table outlines a vision for public engagement with science as presented by the AAAS Center for Public Engagement with Science & Technology on their website.4
Host institution benefits
PES conveners at science museums and similar organizations may want to conduct public engagement activities because of benefits to their own organization, such as:
Scientists themselves may want to participate in public engagement activities because of benefits to themselves and to their work, such as:
Public Participant Benefits
Publics may want to participate in PES because of personal benefits, such as:
Broad societal benefits
There are also benefits to society as a whole around which all participants might rally, such as:
Achieving each of the four broad societal benefits identified on the previous page may require different approaches to the PES activity you implement.
Central to the way CAISE defined PES in 2009 is mutual learning by both scientists and publics from listening to one another. This model of PES is “premised on the assumption that both publics and scientists have expertise, valuable perspectives, and knowledge to contribute to the development of science and its application to society.”5 For this kind of PES, scientists are encouraged to interact directly with public participants in one-on-one or small-group conversations. Not only do public participants get to learn about research directly from scientists, but scientist participants also get to learn from the public participants.
The summative evaluation for the Building with Biology project found that 90% of the activity facilitators, who were mostly science experts, agreed or strongly agreed that participating in the project increased their skills in engaging the public in science.
“Members of the public were very eager to learn and discuss implications of synthetic biology research and think critically about how these emergent technologies would impact society more broadly.”
People make decisions that affect the development of technologies in all areas of their lives.
David Guston at Arizona State University talks about anticipatory governance as “a broad-based capacity extended through society that can act on a variety of inputs to manage emerging knowledge- based technologies while such management is still possible.”7 PES designed for this purpose is all about enhancing publics’ civic engagement skills in connection with science and society. The focus is on public empowerment, which includes:
The design of this kind of PES activity is focused on giving the public participants an active role in discussing topics they can contribute to through their life experience and their understanding of community needs and values. Sometimes in this kind of program science experts may advise organizers and make presentations to public audiences but take a back seat in the discussions and negotiations so that they do not inadvertently hinder public participants from taking fully active roles in the discussion because they defer to the experts.
This kind of PES, which could be called “citizen consultation,” may require greater rigor in various aspects of the program design, particularly if outcomes are going to be reported to policymakers who are not themselves participants.
Added rigor for PES aimed at providing policy advice
The complementary skills of social science researchers and informal educators or science communicators may be helpful in meeting these more rigorous conditions for PES for public policy consultation. The ECAST (Expert and Citizen Assessment of Science and Technology) Network8 was established in 2010 to bring together academic research, informal science education, citizen science, and non-partisan policy analysis to engage citizens in peer-to-peer deliberations to inform citizens about and solicit their input on science and technology policy issues.
This kind of PES builds on earlier work carried out by the Danish Board of Technology Foundation which continues to organize World Wide Views global citizen consultations.9 Science museums in Europe have been involved in recent years in a number of citizen consultation projects funded by the European Commission:
Science museums and many other ISE organizations mostly take a public understanding of science approach focused primarily on young audience members, but in recent years some science museums have explored aspects of PES. In 2011, the Museum of Science collected a sample of 201 activities submitted by 125 organizations as representative of their efforts at the time to step beyond public understanding of science into topics and processes that explore public engagement with science. The case summaries fell into 10 categories: art and theater, citizen science, exhibits, festivals, forums and science cafés, inquiry, media, meet the scientist, on-site research, reference, and take action. The catalog has not been updated since 2011 but is available to download on the Dimensions of PES Wikispace.12 Chapter 7 explores opportunities for further development of PES in the informal science education community.
Priority areas for advancing PES in Informal Science Education
You can find a further exploration of these priorities by navigating the links in the Dimensions of PES Wikispace.14
+ - References
This chapter covers the goals of public engagement with science for various stakeholders, as well as broader societal goals. It also addresses how those goals and their corresponding outcomes relate to the structure or design of the activity or program.
By Caroline Lowenthal
This chapter provides a framework for an individual or organization interested in hosting a PES event for some type of public audience. The framework begins with how to define the goals and needs that a public engagement with science event will cover, with guidance on setting goals and outcomes. This chapter also addresses identifying target participants and available resources. There is also information on choosing the type of event to hold, preparing for the event, recruiting participants, and assessing and following up a er the event. Examples are interspersed to illustrate the principles described. Designing specific activities to use in public engagement events is the subject of Chapter 4.
To be most effective, public engagement events should be designed with a clear understanding of the overall purpose and goals for the engagement, and who the audience will be. Informal science educators should begin by identifying the particular community or societal need the public engagement event hopes to address. The need may come from the educator’s own institutional priorities, or from those of project partners, community stakeholders, or clients funding the work. A public engagement event should include some combination of public and scientist participants.
Finding a good topic for a public engagement event can be challenging and is o en an iterative co-creation process between the informal science educators and scientists. A good topic doesn’t have a right or wrong answer. A good topic is one that is accessible enough for participants to engage in and also contribute ideas and information that is helpful to scientists. There are more details about how to identify a topic in Chapter 4.
This framework for planning and designing a public engagement event was adapted from a framework for the design of chemistry communication events developed by the National Academies of Sciences, Engineering, and Medicine.1 Another framework that may be useful in event planning is the Wellcome Trust’s Planning your public engagement activities guide,which can be found in Appendix B.2
Begin by setting public engagement goals and outcomes for your target audience. Goals are broad statements tied to institutional work, societal or community needs, or to the broad benefits of PES activities (refer to Chapter 2: Strategic public engagement for more on the benefits of PES activities). Outcomes are specific changes in an individual, group, or community as a result of participation in the engagement event or activity. Some types of public engagement activities designed for citizen consultation must be framed carefully to be neutral and represent a wide range of perspectives, but most o ered by science museums for public audiences are more successful if they reflect the interests, needs, and characteristics of the participants.
You may have different goals and outcomes for the public participants and the scientist participants.
For each participant group, consider the following questions from Communicating Chemistry1 to help tailor your event to your goals and participants:
Science museum educators design and present other kinds of programs for their public visitors all the time. Educators may wish to design a public engagement event for one of their usual audiences: subset of a current audience, or a different group altogether.
For groups that include adults and children of various ages, public engagement events with multiple hands-on activities that can be browsed are likely a better fit than longer dialogue programs.
Scientist participants might be graduate or undergraduate students from local universities, industry professionals, early career scientists in research labs or the DIY community, among others. Each of these groups will have different backgrounds and things that they find compelling. It will be important to consider their perspectives as you set your goals and plan your event.
An additional component of a PES event might include data collection. It’s important to plan early in the process for what, if any, data will be collected from the public audiences or participating scientists in your event. Consider how this data will tie into the goals and outcomes set for the PES event overall and for its participants. Refer to Chapter 5 for more about evaluating public engagement outcomes and Chapter 6 for ideas on disseminating the results from your event.
TWO EXAMPLES OF GOALS AND AUDIENCE-SPECIFIC OBJECTIVES THAT MIGHT BE ASSOCIATED WITH A PUBLIC ENGAGEMENT WITH SCIENCE EVENT:
Event type: Forum-style public dialogue program that encourages in-depth discussion about a new technology for genetic engineering and its applications.
Goal: To provide participants opportunities to learn about an emerging science or technology and its societal impacts.
Goal: To increase participants’ confidence in making decisions and participating in public discourse about technologies that raise complex societal issues by giving them practice in considering different perspectives and sources of information in a deliberative problem-solving environment.
The choice of audience and goals will help determine which resources are best suited to support the event. Consider other PES projects and informational resources, physical materials, and organizations and individuals, including scientist experts. From this list of resources, one can identify gaps and opportunities for an event. A gap might be a content area not currently covered, a demographic group that is underserved by the activities or event, or a lack of scientists to facilitate hands-on activities. Once gaps have been recognized, an opportunity exists to develop content or activities, reach new audiences, or create partnerships to fill those gaps.
Informational resources to support the engagement event:
See Appendix A for examples of ready-to-use public engagement tools available from existing PES projects.
Physical resources such as:
Perhaps the most valuable resources are partnerships with individuals and organizations to:
Initiating these partnerships can be challenging. Start by identifying who would make a good partner, and see if any colleagues have contacts in the target organization. It’s usually easiest to send an email asking for some time to talk on the phone, although in some circumstances an in-person meeting can be more appropriate. During the conversation, make sure the potential partner understands what is being discussed, since this type of public engagement may not be familiar to them. Don’t assume they know anything about the core work of ISE institutions. Discuss any financial considerations that may be relevant—will the ISE institution have funds to cover the partner organization’s participation? Or will the partner organization be expected to contribute to the event? Be explicit about what the partner’s role will be in the event and what they will be expected to do, both in advance and at the event itself. Make sure to consider and discuss what they will get out of their participation, whether that’s support for their mission, publicity, an event for their constituents, or something else.
Useful references from the NISE Net include the Museum & Community Partnerships: Collaboration Guide,3 A Guide to Building Partnerships Between Science Museums and University-Based Research Centers,4 and Sharing Science: Workshop & Practicum for Early-Career Researchers.5
In preparing for a public engagement event, there are many different formats to choose from. This may be a familiar type of engagement event or an entirely new format based on the audience and goals.
The Building with Biology project,6 for instance, focused on two different types, or formats, of public engagement with science events. One brought public audiences and scientists together in meet-the-scientist type mini-festival events that included hands-on activities and stage presentations.
The other brought publics and scientists together at round tables for longer dialogue discussions and deliberation about synthetic biology and the use of genetic engineering to address environmental and other challenges, and the related societal and ethical implications.
The primary goal of both PES events was to create conversations between scientists and publics that both groups found valuable and informative.
There are many other formats for PES events. A 2011 study by the Museum of Science identified 11 types.7 Two formats used in the Building with Biology project had complementary strengths and weaknesses. Both formats facilitated conversations between publics and scientists in which each learned from the other.
It is important to note that some public participants may be more comfortable with traditional presentation methods used in public understanding of science (PUS) rather than PES. For these participants, it may take some coaxing to get them out of their comfort zones and into engagement. Sometimes PES advocates believe that people want engagement and want to talk about their own views, but that is not always the case. Event hosts need to create a setting where people feel comfortable and prepared to engage.
Once the goals and outcomes, target audience, and format(s) have been set, it is time for the next steps in preparing for your event. You may be able to download all the materials you need in the form of a prepackaged kit of materials on a particular topic. But in many cases you may need to prepare, or at least assemble, everything you’ll need yourself. Later chapters in this guide provide more details on developing hands-on materials, forum resources, and evaluation tools.
Five steps to prepare for a public engagement event
Depending on the type of event, it may make sense to bring in external facilitators, as described in Step 2 above. These should be individuals who are trained to facilitate conversations, rather than experts on the topic the PES event is covering. Forum events are more likely to require external facilitation than hands-on activities where content experts can be the facilitators. If it is not possible or desirable to bring in external facilitators, then at a minimum, the conveners of the event should be trained in facilitating effective engagement. This should include how to avoid having their personal preconceived notions of what the dialogue should look like get in the way of where the participants themselves want the experience to go.
Some types of PES events, like hands-on museum and festival events, can be presented for regular science museum visitors. Forums or dialogue programs require a different audience, such as one without young children and where participants have a couple of hours to commit to the event. Use what is known about the target audience to adapt marketing strategies and attract people to attend the event. Facebook groups, hobby or affinity groups, or neighborhood email lists can be helpful marketing resources, in addition to working with any partner organizations to market the event to their members. For more about running hands-on museum and festival events, see the Building with Biology Event Planning and Marketing Guide9 and the 2015 NanoDays Planning Guide.10
Recruiting scientist participants presents its own challenges. Depending on what types of institutions are in the area, it may make sense to reach out to universities or colleges, companies, community organizations, or government. Try googling the topic of the PES event and the name of the area and see what comes up. Ask colleagues to see if anyone knows of people working in or studying areas related to the topic. Search the websites of local universities and colleges.
After identifying potential scientists or connections, reach out by email or phone to explain the opportunity and how they might be involved. Be clear about the time commitment for the event and any training leading up to it. Mention whether they will need to do anything to prepare. Many scientists are excited to get involved in their local science museum or ISE institution, so it can be helpful to highlight that angle. If the target is graduate or undergraduate students, offering science communication training can help make the request more appealing (as will a free lunch on the day of their participation, if that’s possible). If there is a scientist who seems interested, ask if they know of others who might be interested in getting involved. Another option is to email professors and ask whether they would be willing to send their graduate students, since it can be easier for them to volunteer their students than for the students to volunteer themselves. Scientists working at companies may be interested if there is an opportunity to promote their company as part of their participation in the event. An easy way to do this is to o er them a table at the event, which they can use to have staff showcase their work or to put out materials that people can browse. Consider what benefits of participation in your event may be attractive to each type of scientist, and o er those benefits if possible.
Citizen consultation events are a very different type of event and may require recruitment strategies that provide demographic representation. For any type of public engagement event, it may take a special effort to recruit under-represented audiences to participate.
If the preparation is done well, the ISE organizers may not have much work to do during the actual event. Make sure to build in plenty of flexibility, redundancy, and backup plans for if and when things don’t go according to plan. Consider what will happen and what adaptations will be necessary if certain key people don’t show up, if speakers talk for longer than they’re scheduled, and so on. In advance of the event, make a list of people who will be available to help at the event and their roles, including if those roles change at different times during the event.
A er the event, complete any evaluation that was planned. Discuss the outcomes of the event with partners and make plans for future engagement activities. Did the public engagement event achieve its goals and outcomes? Consider what went well and what you would change for future events. It’s helpful for the event planning team to have a debrief session to go over lessons learned. Collect and review the opinions and views expressed by participants, and discuss the ways you will use them. Consider how they reflect the event’s goals and outcomes identified during the development process. To learn more about evaluating PES events and ways to disseminate event outcomes, refer to Chapters 5 and 6, respectively.
Public engagement with science depends on relationships. PES activities build new relationships between scientists and stakeholders, between PES practitioners and scientists, and between PES practitioners and stakeholders. No matter how clearly the scope and goals of an engagement activity are defined, there are always expectations that develop during these new relationships, especially regarding follow-up communication and engagement. It’s important to try to recognize those expectations and to think about how institutions and individuals can help meet those expectations.
Sometimes doing PES is challenging. PES practitioners learn many lessons the hard way. Here are some examples of things that could go wrong and how to head them off before they happen.
Not meeting expectations of participants— people come in with expectations that may not be aligned with the purpose of the event. Be clear about setting expectations and goals for the event in marketing material and in any communications in advance of the event. Will participants be expected to do some work and provide their own input? Will they contribute to outcomes or products? Also make sure they’re getting enough benefit out of their participation, especially if they’re expected to do work.
Agenda is too ambitious and people run out of time— it’s tempting to include a lot of components in an event or activity. Assume you have too much and try to cut back. One thing that can help is sharing a draft of the proposed agenda with a former participant to see what they think. Make sure your agenda is well designed and thought through to have enough small-group time, but also enough time as a whole group.
No one shows up— if there aren’t enough members of the public at the event, the event may feel disappointing to the organizers and pointless to any scientists who participate. Make sure there is enough publicity about the event in advance and recruit audiences who are likely to be interested.
No scientists/too many scientists— it’s not PES if there are not scientists at the event, but too many can overwhelm members of the public and make them feel like they can’t participate. Try to estimate how many members of the public will attend, and make sure there are enough scientists to interact with them. If there is a lot of interest from scientists, it may be necessary to cap the number who can participate.
Scientists are not prepared or do not want to listen to the public— scientists may feel like they should be the experts and convey their knowledge to the public. Make sure they understand that their role in the event is to listen. It may help to share this quote from the National Academies report Communicating Science Effectively:11 “How best to engage the public under different circumstances and on different issues is an important empirical question that merits additional research. What is known now, though, is that public engagement often is essential for acceptable decisions about science-related controversies. It is clear as well that even when an issue does not involve a widely known controversy, science communication is more effective when scientists are willing and able to listen carefully and respectfully to different points of view.”
People get upset or off- topic, and facilitators (if present) are unprepared—facilitators might (even unintentionally) be biased, especially if the discussion gets off the primary topic. They might also be insufficiently skilled to get the conversation back on track if people are upset or o -topic. Make sure that the facilitators have enough training to adapt to situations as they come up, and that they know what to do if they get into a situation they can’t handle. Can they call on the lead facilitator? Can the event coordinator come help?
Materials or scientists have wrong or too much/too little information— there is always a tension between providing enough information that people can have an informed conversation and overwhelming them with too much information. Decide how much content to provide and test it in formative evaluation.
+ - References
This chapter describes the overall planning and implementation cycle for conducting a public engagement with science event, including marketing, audiences, venues, recruitment and orientation of scientists.
By Caroline Lowenthal
This chapter goes into detail about how to plan, design, and revise an activity for public engagement. It primarily covers hands-on activities and forum conversation activities, as well as briefly mentioning citizen consultation activities. Examples are included from the Building with Biology project, as well as other projects.
Three characteristics of activities designed for public engagement with science (PES) make them distinct from activities designed for public understanding of science (PUS): topic, public participation, and scientist participation.
PES activities address socio-scientific topics rather than topics focused solely on science. They involve conversations about putting science to use in people’s daily lives and include values and life perspectives that non-scientists can contribute to the discussion. They include discussion of:
Good topics lead to discussion of questions that scientific data and analysis cannot answer on their own. They reflect authentic decision-making priorities for decision-makers and stakeholders, while remaining broadly accessible and engaging to a wide audience. A good PES question is robust, meaning a decision cannot be made in five minutes. The question should require some specific scientific knowledge as an input and have background materials readily available for use as a resource in the decision-making process.
PES activities require that public participants are involved in an active, meaningful way, contributing their knowledge and perspectives. As part of a PES activity or event, members of the public may:
PES activities engage scientists as participants in multidirectional communication with publics. This can include co-creating a question or topic for the public engagement event. Before, during, and a er a PES activity, scientists may:
Sites hosting an event can help scientists with these tasks by providing orientation and training for communicating with the public, and by guiding scientists to think about what input from the public might be most useful for them, along with how they might use the input.
To support the PES goal of mutual learning, the best hands-on PES activities allow public participants to touch, manipulate, use tools, and create things with the assistance of a scientist who facilitates the activity and talks with the participants. While either scientists or informal educators could develop activities of this sort themselves, there are benefits from working together to co-create the activities. Most useful is for scientists and informal educators to develop a set of shared goals and outcomes for the activity, covering content and learning outcomes for both public and expert, or scientist, participants. This helps assure that scientific content and educational pedagogy will play supportive roles in the design of the activity, and that there is a mutually agreeable plan for learning by both public and expert participants.
Developers will need to consider how the activity will support PES, using questions that address societal impact, values, priorities, and policy. The activity should put visitors in the role of making choices and discussing the reasons for their choices. It should be designed to be facilitated by scientists, though it may be facilitated by museum staff or volunteer educators. In addition to supporting conversations between public participants and scientists or educators, hands-on activities that support PES should raise or suggest questions that focus not only on what and how but also on why and should. When designing the activity, keep in mind that analogies, pictures, animations, and physical objects can help the public make sense of the science.
When the PES event is being planned, the organizers should plan to recruit scientists or science students to facilitate the activities and to provide them with event logistics and the details of the activity they will be facilitating. If possible, the team can also provide an in-person or online orientation and science communication training. This training should include not just how to talk about their work, but also how to listen to the public and engage in multidirectional conversation. For the Building with Biology project, this training was included in orientation materials designed by AAAS, which helped convey the rationale and methods for PES. In the case of forums, scientists do not facilitate directly, so it is important to make sure they understand their role. In general, scientists participate in forums to listen and discuss people’s values on scientific issues, on an equal footing with public participants.
As part of the activity development, it is important to include a plan for evaluation that will allow for revising the activity based on feedback. Chapter 5 focuses on evaluating public engagement outcomes.
Three phases of evaluation used in developing activities
There are five elements of a forum experience for participants:
Framing the question
The development of a forum begins with the process of determining the overarching question that participants will discuss. The design of the question can influence the success of the discussion. A successful discussion is more likely to result from a question that is not strictly scientific, but rather starts from a scientific background and depends on participants bringing in their own values and experiences to help them answer it. This is the difference between “Do vaccines cause autism?” (strictly a scientific question) and “Should we allow parents to opt out of vaccinating their children because of their personal beliefs?” (a socio-scientific question). Similarly, a successful discussion is more likely to result from a question where people disagree than where almost everyone agrees. For example, “Should we genetically engineer mosquitoes to be worse at transmitting disease?” (a question on which people tend to disagree) will work better than “Should we provide cognitive enhancing drugs to all college students?” (a question to which most people say no).
Facilitating the conversation
Once the question is determined, the next step is to create the materials that will be used in the discussion. A facilitation mechanism is needed to prevent the conversation going off track or being dominated by any one participant. An early decision to make is whether there will be facilitators or printed materials at each table to guide participants through the discussion. Live facilitators can be trained professionals, students, museum staff , or volunteers. They are responsible for keeping track of time and making sure that all voices are heard. In a self- facilitated game-like format, background knowledge can be shared by participants who take turns reading info cards, which gives everyone a speaking role from the start. You can use cards, a game board, or time limits, among other tools, to facilitate the discussion. The resources in Appendix B may be useful in the development of materials. In particular, the Everyday Democracy guide How to Develop Discussion Materials for Public Dialogue7 includes useful content. Make sure that scientist participants understand that they do not facilitate forums, but rather listen and discuss people’s values on scientific issues, on an equal footing with public participants.
Play Decide uses illustrated self-facilitation guides so that the materials can be used internationally regardless of the language participants speak8
Shared background knowledge
Shared background knowledge can come from a variety of sources, such as printed material, live presentations, videos, or theater. If developing a forum, think about what resources are already available or can be created that will give participants a shared background on the topic. This choice may depend on several factors such as time or money to create materials or a theater performance, access to scientists or experts who can speak on the topic, or whether videos or other materials exist that convey the desired information. Whatever type of material used for the background, its presentation needs to be easily understood by a general public audience, free of scientific jargon, and as balanced and unbiased as possible. It’s best if one scientist or more can review any materials or presentations that have been created for accuracy before sharing them with the public. If the event will include expert speakers, choose people who are comfortable speaking with a public audience, and instruct them to avoid scientific jargon and overly technical details.
Valuing public participant contributions
Being explicit that public participants have valuable knowledge to share is an important part of any forum, beginning with the design of the question, continuing through the language and images used for marketing the event, and specifically stating this as part of the welcome to the event. A forum topic that invites people to bring in their personal values and experiences welcomes participation. When marketing the forum event, be explicit that it will be an interactive experience. Phrases like “Share your views,” “Have a conversation,” or “Add your voice” work well.
The more people that can be recruited with diverse perspectives, the richer your discussions will be. This includes members of the public as well as scientists. It’s ideal if there can be both members of the public and scientists at every table for small-group discussions. It’s important to make sure that public participants feel empowered to speak and share their views with the scientists, and that they don’t hold back their own perspectives and defer to the experts. The goal is a multidirectional conversation, with all participants listening to each other. The forum host can encourage this explicitly in the welcome, include it on materials on the tables, and if using facilitators, have them remind participants that all views are valuable to the conversation.
Reporting out of resulting views
One goal of multidirectional conversations among participants with diverse perspectives is for new views to emerge from the discussions. Depending upon the design of the forum activity, those resulting perspectives may be in the form of individual views or collective views. Report-out mechanisms at the end of the discussion help collect participant views and add meaning to the activity.
Some options for reporting out at the end of a forum activity
Report-outs from every table share collective thoughts about the question posed
Citizen consultation activities for direct policy input
Citizen consultation activities for direct policy input Modified forum programs can provide policymakers with public input for governmental or institutional decision-making. Development of the type of forum programs described in this chapter was originally inspired originally inspired by Citizen Consensus Conferences organized by the Danish Board of Technology. Such consultations are really research activities for policy clients rather than educational programs to teach the participants, though participants learn a lot from their involvement. Rigorous attention to a variety of details is needed to ensure scientific validity of the outcomes, such as testing of background materials and processes for biases and recruitment of demographically representative groups of participants. Details of adapting forum programs for citizen consultation are not addressed in this guide, but to learn more about this kind of activity see the World Wide Views and ECAST resources in Appendix A.
+ - References
This chapter goes into detail about specific approaches for two kinds of PES activities: Hands-on activities for museums and science festivals, and forums for dialogue and deliberation. It briefly mentions a third approach, citizen consultation activities for direct policy input.
By Katie Todd, Sarah Pfeifle, and Elizabeth Kunz Kollmann
This chapter shares resources and discusses special considerations for evaluating public engagement with science (PES) events or activities. It focuses on defining appropriate outcomes for PES and selecting the right methods for measuring those outcomes.
Evaluation of PES activities is similar to other evaluation in the informal education field in that the process involves identifying the focus of the evaluation, planning for data collection, gathering data, making sense of the data, and sharing it with relevant stakeholders.1 While the majority of this chapter focuses on PES- specific evaluation, general resources about evaluation of informal education endeavors are easily available to professionals. These resources are relevant to evaluation of PES activities and especially helpful for those who are new to evaluation.
Distinctive Characteristics of PES Evaluation
There are two key things that differentiate evaluation of public engagement with science activities from other informal education evaluation:
The first step of a PES evaluation process is to work with the project team to identify goals for the PES endeavor. This should happen early, often when writing a grant or during the planning stages for the event or activity. Chapter 3 talks about the broad goals of the PES event. The evaluation should document these goals and record how PES aligns with the organization’s mission and vision. This will allow the team to use data to track how well the PES endeavor follows the organization’s values throughout the evaluation.
To construct the evaluation, the project’s broad goals need to be specified to the key components that will be investigated. Additionally, the target audiences – the people who will be involved in the PES program – must be identified. Traditionally, ISE activities have focused on impacting public participants.4, 5 Since PES is about mutual learning among all participants, planners of PES events should consider multiple professional audiences including but not limited to:
Broad goals may overlap among audiences, but the specifics will likely differ. For example, the PES project team may set learning goals for both public and scientist participants, but the focus of that learning might be scientific content for the public and science communication skills for scientists. The following page shares examples of PES goals for scientist, public, and informal education audiences. It is best to focus on a short list of objectives. Fewer goals allow for more intentional program design, which leads to stronger outcomes. The goals should be realistic and appropriate: if the program does not discuss ways participants could follow up afterwards, selecting a goal about follow-up behavior is inappropriate. Certain interactions may better support different outcomes, as well. Participants in Building with Biology hands-on activities reported greater learning about content knowledge, while forum participants indicated greater learning gains about what other people think about synthetic biology.5
Once the goals are set for each of the target audiences, the next step should be to state the evaluation questions. Each evaluation question should be directly related to the goal. The way the question is framed depends on the phase of evaluation being conducted. In the informal education field, evaluation is o en categorized into front-end, formative, and summative stages, as shown below. Other common evaluation types may be relevant depending on your activity. For instance, remedial evaluation could be applicable for a PES exhibition or other lasting experiences, but may be less appropriate for a one-time event. While some large-scale projects may undergo multiple stages of evaluation, smaller activities may choose to do only one or two, based on the team’s needs and funders’ expectations.
The following boxes share some specific PES examples of how one could turn a goal into an evaluation question for each evaluation stage. As mentioned previously, each study might address several goals and have a range of evaluation questions.
Front-end evaluation can help you make data-informed decisions. For example, a series of interviews could gather data from local scientists to learn what experience they have with outreach, and how their experiences are different from PES. This could help identify what types of training would meet scientists where they are and help them feel prepared to lead the type of interaction set forth by the activity goals.
Goal: Scientists will feel prepared to lead PES activities.
Front-end evaluation question: In what ways are scientists engaging in outreach? How, if at all, is their experience different from PES?
Formative evaluations test drafts of programs in order to gather data that will help improve the materials. For example, a formative evaluation could share draft facilitation guides or training materials with potential volunteers and ask them to identify areas for improvement.
Goal: Activity guides will stand alone so volunteers can lead the activity without additional training.
Formative evaluation question: Which aspects of the activity guide, if any, are confusing for volunteers or facilitators? What other information, if any, do facilitators need?
In a summative evaluation, a PES project might want to know whether people learned from the experience. While a traditional public understanding of science (PUS) project might look at content learning around facts, a PES project could look at learning about other people’s views, societal and ethical implications, and more.
Goal: Participants will gain knowledge about how science impacts society.
Summative evaluation question: To what extent do informal education professionals learn about conducting PES activities? To what extent do scientists learn about the public’s views? To what extent do public participants learn from the materials? From other participants? What types of learning do these audiences gain?
The previous boxes shared example questions for each evaluation stage, and focused on a single audience. An evaluation study will typically have multiple evaluation questions and there should be questions that address each target audience. Depending on the scope of the evaluation, studies o en address between two and five evaluation questions per audience. The fewer questions you have, the more focused the study will be; more questions may cover more topics, but think hard about what can actually be accomplished well. Especially when evaluating public outcomes, response fatigue sets in if too many questions are asked. If evaluation questions are kept concise, it will be easier to keep evaluation instruments manageable as well, leading to happier evaluation participants and higher-quality data.
Ethical considerations of PES evaluation
ISE evaluation projects that aim to create generalizable information or that use federal funding will likely require the approval and oversight of an Institutional Review Board (IRB). The purpose of IRB protocols are to ensure that research projects consider how they are gaining consent from the people they are collecting data from and whether the data they collect is overly invasive or risky to participants. Evaluations that exclusively serve to help teams understand the impacts of the program and remain internal to the ISE institution may be exempt from IRB oversight. However, even in these cases, it is best practice for anyone handling data or interacting with human subjects to complete human subjects training. NIH and CITI have human subjects training courses that certify professionals in best practices for protecting human subjects.
PES audiences require protection in the following ways:
Once evaluation questions are confirmed, one should develop a plan for collecting data. The appropriateness of different evaluation methods depends on the scale of the project, the evaluation questions, the audiences, and the type of program that is being evaluated. The following are types of instruments that could be useful in evaluating PES events and activities:
An analysis of existing PES evaluation instruments found that surveys were the most common. Most projects collected data following the event, with a minority using pre-post designs or process instruments.5
A er deciding what type of instrument best suits a PES program, one can develop an instrument from scratch or use or adapt an existing data collection tool. Diamond, Luke, and Uttal’s Practical Evaluation Guide outlines valuable guidance around these decisions.6 Basic information about creating a tool can be found in the Team-Based Inquiry Guide.1 There is a function on www.informalscience.org that allows you to search for existing instruments. AAAS and Karen Peterman Consulting have developed two validated scales for PES, including one about scientists’ outcome expectations7 and one for scientists’ self-efficacy for PES.8
The Building with Biology project tested multiple methods during its pilot phase. The box on the right summarizes the results. The evaluation team selected these three methods because all three could address the same evaluation questions about learning, interest, and value. The three survey types used the same questions and gathered data that was comparable in quality. In the end, the evaluation team pursued passport data collection. This method required enough staff to distribute passports and collect surveys, but it was easiest for the data collectors and had the least burden on participants because the survey was built into the activity. When visitors had completed their passport by collecting stamps for doing PES activities, they could return to the passport station, fill out a survey, and receive a temporary tattoo as a thank-you. Group members could take their passports home, as well. The project team also felt the passport activity was a positive guide that encouraged authentic PES with an embedded evaluation task.
To learn about the affordances of evaluation methods for public participants in PES, the Building with Biology evaluation team pilot tested three ways of collecting data:
There are many considerations when planning and conducting an evaluation for a public engagement with science activity or event, but the process need not be onerous. Keep the following tips in mind for a seamless evaluation:
+ - References
This chapter describes evaluation methods and tools for assessing public and scientist outcomes as a result of participation in PES activities and events.
By Caroline Lowenthal and David Sittenfeld
Organizers may want to:
Public engagement with science as a form of participatory democracy is still new to many ISE professionals and scientists, so there is a need to share purpose, goals, practices, and evaluation strategies for PES. For professionals who are already interested in public engagement, sharing both materials and content, as well as lessons learned and evaluation results, can be extremely valuable.
Possible ways to disseminate project results include through online websites and platforms, professional conferences or professional organizations, or publishing in a range of journals. Specific examples are provided below.
Various websites allow for posting of digital materials, guides, evaluation results, methods, and lessons learned:
Dissemination at professional conferences
Building with Biology project staff present a poster about the project.
Professional conferences o er opportunities to share insights and experiences gained from work on PES projects. During the three years of the Building with Biology project, team members presented several sessions at the ASTC conference on aspects of the project related to evaluation of PES, biotechnology events, forums, and cross-museum collaboration, among others. Following are some relevant professional conferences for PES work:
Dissemination through professional journals
Journals can be a good place to share evaluation methods, findings, and recommendations for future work. Publications that would be well-suited to sharing research on this type of PES work include:
In general, the dissemination of evaluation findings focuses on establishing a format that is actionable for the internal stakeholder commissioning the study. Evaluation reports can be posted on informalscience.org. Methods and findings are o en presented at conferences, especially the Visitor Studies Association conference (listed above). Whereas evaluation is designed to be context- specific so that its findings are directly addressing the impacts or outcomes of the specific project, research studies aim to produce generalizable knowledge that can be applied across the field. As with evaluation, dissemination of research results can be done through informalscience.org and conferences, as well as peer-reviewed journal articles including those listed above.
If a goal of the PES event is to be more formal about sharing public input and recommendations with scientists and policymakers, discussions can be recorded in a variety of ways, collected, and analyzed. In general, two primary methods of sharing outcomes from PES events with policymakers and scientists are written reports and live presentations, and o en using both together can work well. This works best if an expert or policy group is interested in public input, and is even better if this group participates in setting goals for what they would like to get out of consulting the public.
As an example, the Museum of Science in Boston and partners in the Expert and Citizen Assessment of Science and Technology (ECAST) Network worked with NASA in 2014 to develop a forum to get public input on options they were considering for missions related to asteroids. NASA collaborated on developing the PES questions that the public would answer, provided a variety of informational assets that could be used in the forum event, and then served as a client for a report based on the public’s input at two separate events.12
Another series of PES events, organized by the World Wide Views global citizen consultation initiative,13 consisted of simultaneous daylong citizen deliberations at dozens of sites in countries around the world in 2009, 2012, and 2015. Public participant responses were collected, analyzed, and presented to the United Nations at the relevant international conferences, as well as made available online for anyone to view. The United Nations was not a client for the World Wide Views events, but it welcomed the citizen input generated by them.
World Wide Views organizers presenting at a United Nations conference in India
PES participants, especially if they have to work hard addressing challenging questions, are more likely to feel that their effort was worthwhile if they know that someone is interested in and will potentially use their views and recommendations. The two examples above are at the high end of expert and policy groups interested in public input.
Such groups could also be composed of scientists at a local university or research lab or members of a town or city government or citizens group. For instance, the Director of Environmental Health for the city of Cambridge, Massachusetts, saw a forum on nanotechnology and asked the Museum of Science if it could develop a forum to focus on questions he was considering in connection with consumer products containing nanoparticles.
Key questions for designing PES events to collect and share views relate to how public participants will provide their views during the event and how the organizers collect them.
How the public provides their views
The public can provide views in a variety of ways that affect how outcomes can be disseminated:
Studies on Public Attitudes to Science in the U.K. published in 2011 and 2014 show that public audiences overwhelmingly think regulators, government, and scientists should be engaging in dialogue with the public about science, but about 70% don’t want to be involved in such activities themselves.14, 15 If the same is true in the U.S., public audiences not involved in public engagement events themselves may at least be interested in learning about the outcomes of such events.
However, a live museum program may not be a successful way to reach such interested audiences. A er World Wide Views on Climate and Energy in 2015, the Museum of Science held a dissemination event for members of the public to learn about the deliberations that took place, and to give members of the public who weren’t able to participate a chance to weigh in on the topic. Fewer than 10 people attended, a result that did not necessarily support the effort invested in the event.
While holding a follow-up public event may not reach audiences with a special interest in learning about the outcomes of PES events, other methods may be more successful. World Wide Views provides public access to outcomes online, including all of the questions that were asked of all participants, along with their answers, organized into bar charts, which can be broken down by country for comparison.
PES outcomes can also be shared through press releases or press coverage, or blog posts or online articles. For press coverage, it’s important to make sure that the presence of members of the press at events does not interfere with the deliberation process. It’s also good to make sure that any press coverage that may come out in advance of the PES event does not impact or bias participant views. These methods of documenting PES outcomes allow the information to reach audiences with special interest in the specific topic or question of the PES activity.
While all of these vehicles of dissemination can be valuable, the most effective way to disseminate project outcomes is through direct involvement in the project. Even though the term dissemination does not normally refer to reaching the project participants themselves, if a project can gather the necessary resources, direct involvement can have a stronger effect on people than simply being presented with the results of the engagement. Participating in public engagement can provide an impactful personal experience.
Professionals interested in PES that are good candidates for direct project involvement include ISE professionals, STEM undergraduate and graduate students, early career scientists, and policymakers. They can be involved in many ways and at many stages of the process, described in detail in Chapters 3 and 4 as well as the example below.
The Building with Biology project16 was able to leverage the National Informal STEM Education Network (NISE Network)17 to involve hundreds of organizations and recruited partners to be part of the experience. This section provides examples of several direct involvement strategies for dissemination from the Building with Biology project.
The core team, consultants, and advisors were chosen not only for the expertise they would provide for carrying out the work of the project, but also for their ability to take learning from the project and incorporate it into their own work with specific professional communities. Project team participants were leaders in the NISE Network’s community of science and children’s museums, Arizona State University’s social and political science community, and AAAS’ scientific and outreach communities.
Scientists and science museum educators formed 12 pairs of professionals in the first year of the project to develop and test prototype Building with Biology hands-on activities in an intensive, collaborative, and iterative process aimed at raising their knowledge about, and capacity for, developing activities that focused on public engagement with science. The expectation was that both the scientists and museum educators would carry their increased knowledge and capacity beyond the current project into future work. To get buy-in from these project participants, it’s important to have funding to support their work.
Distribution of physical resources and/or funding is another strong dissemination strategy if the project has a budget large enough to support the cost of this approach. The Building with Biology project subcontracted various work groups from the NISE Network, including the kit development and fabrication teams to produce 200 physical kits, and the community team that consists of national regional hub leaders to provide direct communication with museum partners all across the country. The Building with Biology project also awarded 32 stipends to organizations willing to support evaluation and data collection activities for the forum programs included in the Building with Biology materials distributed. Building with Biology physical kit recipients included science museums, children’s museums, universities, iGEM (Internationally Genetically Engineered Machine Foundation) teams, community bio labs, and members of the Teen Science Café Network.
While the Building with Biology project had sufficient funding to implement these three strategies at a large national scale, the same approaches can also be effective at a more modest scale.
PES is most satisfying to participants if their recommendations are acknowledged and used by someone, so there is a need to share their input with those who can use it. That can happen in several ways depending upon the overall goals of the engagement activity.
Participants themselves, including both scientists and members of the public, benefit from thinking through PES questions with input from others and working toward conclusions either individually or collectively.
Participants mention their own learning, both about the topic of the event and about the views of other participants, as a valuable outcome of PES events. They also cite increased interest in the topic and suggest presenting the PES program to other audiences. For example, some teachers who participated as members of the public in a Building with Biology forum asked to use the forum materials with the students in their own classes. There have also been a few cases where participants have changed careers and become professionally involved in the topic due to learning about it through a forum program.
Impacts on scientist participants have ranged from gaining a greater appreciation for public opinions, to increased familiarity with the topic, to honing their own science communication skills, to thinking about their own research differently or writing grant proposals to pursue funding for further work on PES projects with museum partners.
At PES events, and especially at forums, participants can share their ideas in a variety of ways. They can participate in small-group discussions in which participants share their own thoughts and give verbal report-outs of the collective views of each discussion group, vote individually on paper or electronically and have that data visualized, or record collective narratives or answers to questions on flip charts or physical materials designed for the specific PES event.
For scientists, direct participation in discussions and deliberation with public participants can provide input for their own work and impact their career trajectories. To maximize the impact, however, it’s important to structure the program carefully. Science students and other scientists not identified as lead experts in the particular topic of the discussion may be able to participate in the small-group discussions without hindering the active participation of others. However, when particular participants are identified as experts, present to the whole group, and then join small-group discussions, there’s a risk that their participation may inhibit full participation by non-experts (public participants). To reduce this risk, it should be clearly stated that both scientists and public participants share an equal voice in discussions and deliberations. If the PES event is going to provide a report of public views to an interested group a er they event, then it may need to be designed so that identified participating experts serve as resources to be called upon, but don’t sit in on group discussions. Then groups that include members of the public can present their outcomes and recommendations to expert scientists following the deliberation.
– Scientist participant, Building with Biology forum
+ - References
This chapter provides tools and methods for disseminating PES projects and results with informal STEM educators and others interesting in implementing PES activities and programs, and sharing public views and recommendations that result from PES.
By David Sittenfeld
The preceding chapters have identified and communicated points of promise with respect to the practice, adoption, evaluation, and theoretical frameworks of PES. This chapter communicates ideas and recommendations for strategic directions for broadening the adoption, participation, and range of PES offerings within the informal science education field.
While the variety and depth of public engagement methods in the context of informal science education environments is growing rapidly, more work is still needed to embed PES methods and perspectives more deeply within the culture of the informal science education community, so that the meaning and merits for PES as a way of collaborating with scientists are well-recognized. Further, future innovations, more diverse participation, and broader implementation could help to strengthen the impacts of PES activities.
As one of the final activities of the NSF-funded Multi-Site Public Engagement with Science (MSPES) project, approximately 45 PES scholars and practitioners from the fields of informal science education, natural and physical sciences, and science communication met at the 2017 American Association for the Advancement of Science (AAAS) Annual Meeting in Boston to suggest and consider future directions for PES. Here we share recommendations that were generated by that group, as well as other priorities for future PES efforts that have emerged as part of and are concurrent with the work of the MSPES project. These recommendations are summarized in the accompanying box and are explored more fully in the rest of this chapter.
The central characteristic of these recommendations is co-creation. “Co-creation” refers to collaborative topic selection to establish a roadmap for shared PES agendas that are of relevance to both scientists and publics. These recommendations reflect the idea that bringing together publics, researchers, and community partners to envision new PES topics and activities will serve to broaden participation and also to sustain and more deeply embed PES into the fabric of ISE practices. ISE institutions are natural conveners for these kinds of co-created PES activities: they are skilled at translating complex scientific ideas for audiences from very different backgrounds, they are welcoming institutions with connections to community organizations, and they are respected and trusted by scientific and policy institutions, as well as by diverse publics.
Some of the ideas recommended here have been tried in pilot settings before, and others would be challenging to implement, but could lead to new innovations in the co-production of knowledge by publics and researchers in ISE settings.
The American Association for the Advancement of Science (AAAS) identifies four kinds of engagement on its PES website:1
World Wide Views organizers presenting at a United Nations conference in India
While the focus of this guide has been primarily on “policy deliberation” and “public dialogue,” the techniques of these kinds of engagement can also serve the purposes of “knowledge co- production” and “cooperative engagement.”
For example, public dialogue or policy deliberations could be integrated with elements of knowledge co-production to amplify upon the mutual learning among and between different kinds of PES audiences. This could be accomplished through co-creation of new agendas for PES activities and/or areas where new research knowledge would be beneficial for both public communities and for scientific understanding. Further, cooperative engagement activities, which connect research institutions with stakeholder communities of particular interest or concern, could be combined with public forums or other kinds of PES mechanisms to work with communities in new and innovative ways on solutions to societal problems.
Other experiments exploring the redesign of existing PES formats and products could help to inform future PES innovations in the informal science education community. For example, PES could be built into exhibits more explicitly to help connect them to other kinds of ISE activities.
An increasing number of exhibits are building elements of PES into physical or virtual exhibits. One interesting experiment with this is the Norsk Teksnisk Museum’s exhibit called TING: Technology & Democracy, which received an ASTC Leading Edge Award in 2015.2 TING visitors are confronted with 100 objects that represent the intersection of technology and society. Visitors then consider a series of emerging technological objects, discuss them with fellow visitors through moderated conversations, and vote with a wooden cube on whether and how they should be used in society.3
The Provocative Questions exhibit in the Museum of Science’s Hall of Human Life similarly engages visitors in discussions about science policy questions such as taxation of sugary drinks or sharing genetic information of unborn children. Visitors learn about the science, consider a range of social values and personal perspectives, and discuss tradeoffs with a partner before making a decision and viewing the responses of others.
While many ISE projects such as Portal to the Public and Building with Biology have designed and used engagement materials that were co-created by scientists and informal educators, a potentially strategic next step in the evolution of PES could be to include public participants in the co-creation of PES agendas for PES engagement through collaborative selection and refinement of topics and questions for discussion. By co-creation, we refer to mutually selecting topics for PES, whereas knowledge co-production refers more generally to publics and researchers working together to produce new understanding.
One example of an institution that employs collaborative selection of issues for mutual learning is the Public Laboratory for Science (publiclab.org), which was established a er the 2010 BP oil spill in the Gulf of Mexico.4 The researchers at Public Lab worked with local communities to hear from them about environmental issues of local concern. Similarly, the scientists at the Hubbard Brook Research Institute have held iterative series of discussions with local community and industry stakeholders to hear their ideas, concerns, and priorities that can help to inform future investigations at the Long-Term Ecological Research station where their research is conducted.5
There is additional precedent for this work in the citizen science and public health fields. The Harvard Catalyst, which devotes funding to establishing connections between communities and researchers who can help to address community concerns, states that community-engaged research “centers around fostering collaborations with and among groups of people affiliated by geographic proximity, special interest, or similar situations with the goal of addressing issues that affect the well-being of the people within the group.”8 This philosophy reflects a change in power dynamic; topics from mutual engagement arise from communities and scientists working together to decide what is important for science to learn and address, rather than flowing from the priorities of researchers and their institutions and including communities later in the process.
The Center for Nanotechnology in Society at Arizona State University and the Nurture Nature Center in Easton, Pennsylvania, have each developed and implemented a series of programs that integrate co-creation of PES agendas with elements of curious exploration and deliberative dialogue. With a goal of creating a more “inclusive, sustainable, and integrated public engagement experience,” the project organizers of the Futurescape City Tours led half-day tours of six cities focusing on urban planning topics such as energy, food systems, architecture, and transportation. The organizers wrote that “citizens drive the agenda and participate in conversations as active, experienced, and equal contributors” with civic planners, policymakers, and stakeholders. Tour participants explored innovation spaces, public art installations, and other types of urban landscapes, extemporaneously co-developing the tour route and capturing photographs of “visions of change” that the participants felt raised societal issues they would like to discuss. The images were used to spark facilitated dialogues with decision-makers and partners from community institutions.6
Easton Matters: What Environmental Issues Matter to You? was a similar two-year project led by the Nurture Nature Center.7 Building on a number of prior participatory dialogues, the project organizers surveyed local residents, community organizations, and policymakers to identify issues of local environmental concern such as traffic, access to food and water, and disposition of urban wastes. The next phase of the project was a series of facilitated public dialogues that elicited a range of attitudes about potential solutions and their tradeoffs. Participants’ ideas were presented to city officials and exhibited through physical exhibits and interactive walking tours.
The authors of the 2009 CAISE inquiry report Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education classify various types of public participation in scientific research projects (commonly referred to as “citizen science”) into three categories: “contributory,” “collaborative,” and “co-created.” While contributory projects are primarily envisioned and shaped by the researchers, co-created PPSR activities differ in that they are mutually conceived by community members and researchers as a defining step in the process. Researchers at the Cornell Lab of Ornithology observed that they saw “evidence that co- created projects that are initiated to meet specific community needs can draw concerned citizens into the scientific process who might not otherwise be involved in science-related activities.” Further, “some PPSR participants become more engaged in community politics and more confident about asking for a place at the table in making decisions about community planning.”9
In The Participatory Museum, Nina Simon extends the co-created concept to the building of ISE education projects more broadly: “In co-creative projects, community members work together with institutional staff members from the beginning to define the project’s goals and to generate the program or exhibit based on community interests.”10 ISE institutions could similarly become more responsive to the needs and priorities of community members by including co- creation in the design of PES activities, targeting areas where both groups see relevance and potential for mutual learning. For example, parent groups might be interested in helping to provide input on future research activities in developmental psychology, or people who share a disability or disease might come together to propose new ideas and share concerns regarding research that could impact others who share their condition.
The following table proposes a number of possible benefits for each PES audience from mutually co-created PES activities.
Another potential way to increase the impact of PES is to provide avenues for upstream public input on proposed research activities through the inclusion of community, local, or indigenous knowledge through the mutual learning that occurs in PES. One of the most popular ideas for new PES activities at the Multi-Site Public Engagement with Science project meeting in February 2017 focused on local community guidance of local research. In this model, informal science institutions would provide a venue for local researchers, community stakeholders, and diverse publics to come together and shape the future of proposed research that could impact a community.5 Scientists would describe their proposed research and their motivations for doing it, and community members would communicate their values, concerns, and priorities. Researchers and publics would work iteratively to consider research that is useful for generating usable scientific knowledge, while responding to local community values and perspectives.
One example proposed by the meeting attendees centered around the idea of introducing engineered organisms to fight the spread of a locally problematic vector-borne disease, or to eliminate a local invasive species as part of an ecological management plan. Similar conversations could happen around laboratories studying highly communicable infectious diseases or next-generation nuclear reactors, where tradeoffs between local economic growth, scientific advancement, and other community concerns might create conflict within a community.
The practice of PES among informal science institutions has seen rapid growth over the last few years. This growth and innovation brings potential challenges with respect to maintaining connections between PES practitioners who focus on different formats or do their work at different kinds of institutions. To advance the work of PES in ISE, it would be helpful if PES practitioners could share the work they are doing with one another and with the field more broadly as they develop and implement new PES innovations. To that end, one motivation for this guide is to encourage PES practitioners to disseminate their PES activities and experiments to the broader ISE field, in order to build the knowledge base and to increase the range of tools available for implementing PES. We envision the website developed as part of the MSPES project as one way for PES practitioners and scholars to share with others in the ISE community (publicengagementwithscience.org) Other resources, such as AAAS’ Trellis community and ASTC’s Forum on Public Engagement with Science can also help practitioners and researchers share new knowledge and PES resources between and among practitioners in their fields.
Since scientist participation is a key component of PES, building upon existing connections between scientific and ISE communities will be crucial as PES is adopted more broadly within the ISE community. In the Building with Biology project, AAAS worked with a range of scientific societies and institutions to “match” local scientists from the synthetic biology field with ISE institutions implementing Building with Biology events. In addition, a partnership with the Synthetic Biology Engineering Research Center (SYNBERC) connected participants with a number of researchers in the field of synthetic biology who were supportive of public engagements. Furthermore, the International Genetically Engineered Machine Competition (iGEM) provided a number of young scientists at the high school and college level, who were required to address societal and environmental impacts of their work and to engage with public audiences as part of their projects.12 These partnerships helped the Building with Biology project build connections between scientists and ISE organizations at all levels of involvement in the project across the country.13
PES projects focused on other topics and scientific disciplines may not be as well positioned for PES activities as the Building with Biology project. For certain emerging technologies, there may not be many scientists working in the field in any specific area of the country, and there may be few or none with expertise in the certain area of research that raises a particular societal question. Also, the field may not have a pool of available scientists who are tasked with or enthusiastic about participation in public engagement. Other fields may not have resources like iGEM, or strong support for public engagement. Therefore, institutional capacity for connecting scientists with ISE organizations is an important area of work for the PES community of practice.
World Wide Views organizers presenting at a United Nations conference in India
Communication across the ISE and natural and social science fields is a natural step to building this institutional capacity. For example, partners from the Multi-Site Public Engagement with Science project gave a number of presentations at AAAS Annual Meetings and at synthetic biology conferences in recent years, describing the rationale and meaning of PES in the ISE context and eliciting partnerships and participation from the scientific community as part of the scientist recruitment phase. These kinds of cross-disciplinary communications will be important to proceed as ISEs continue and expand their PES efforts.
PES activities are a lot of work for all the participants: scientist and public participants, and ISE institutions convening the events. A few ways to increase the value to participants are:
Providing social context and rewards for public participation in PES
Public participants who take part in PES activities frequently express interest in learning about the processes for sharing their ideas with scientists and policymakers. While conversations at PES events between scientists and the public are highly useful in that scientists are physically present at the event and can hear directly from public participants, methods for analyzing and communicating outcomes more broadly or to specific target audiences could help to validate the experience for public audiences and increase motivations for public participation, such as the online data presentation and reports to the United Nations that were produced in the World Wide Views projects. World Wide Views and similar events, however, have strict demographic targets for participant group composition. Important questions concerning the demographic representation of self-selected participation in PES events at informal science education institutions should be examined before the outcomes of PES events of this sort can be presented as expressing the views of the larger public.
Increasing context and rewards for PES within the scientific community
Public engagement might spread more widely throughout the scientific community if scientists were given more recognition for their PES participation, and if there were avenues for sharing with their colleagues what scientists learn from public audiences through PES activities. The American Association for the Advancement of Science has done important work over the last few years to increase the prominence and understanding of PES within the scientific community, but still many individual institutions and/or scientists may be unfamiliar with the concept of PES or unsure of the specific benefits for them. Transforming the scientific reward system to be more cognizant and responsive to the value of PES activities so that researchers are acknowledged in their funding and tenure for participation in PES is a challenging task that lies beyond the scope of this guide. It might help, however, if local ISE institutions that work with researchers to conduct public engagement events could communicate the motivations and potential benefits of PES with a researcher’s institution, rather than just with individual scientist partners. In some cases, this could be accomplished effectively in parallel, with public events targeted at local publics and policymakers, with presentations at national scientific conferences to target the scientific community. The resources created by AAAS1 and projects such as Portal to the Public14 are important stepping-stones that can be leveraged locally as ISE institutions work with their local researchers to conceive of and implement PES activities.
World Wide Views on Global Warming participants speaking at an event to reflect on their experiences in 2009
Increasing prominence of PES in a civic and civil society
Further, there are ways that could help to increase the prominence and motivations for PES while also serving to validate the experience for scientist and public participants alike. For example, PES activities that are endorsed by well-known scientists, personalities or institutions could help to broaden participant recruitment and also serve to increase the prominence of PES. PES activities could be combined with other educational or civic activities to increase their impact for public and scientist participants. For example, PES dialogues at science centers could be aligned with formal education in high schools or universities to help connect and empower the subjects of formal STEM learning. Or ballot initiatives could be informed by citizen dialogues at local ISE institutions.
One option for more deeply embedding PES in the civic, social, and educational communities would be to try to make a targeted PES campaign go “viral.” What if a public forum taking place at science centers around the country were spurred on by a video on social media, in which celebrities encouraged community members to help make a decision, and promised to be there when the results were announced with participation from civic leaders? People who are frustrated with the currently polarized nature of discourse in society may be drawn to participate in a more constructive experiment to connect scientists with the public. Such an effort could heighten the standing of PES efforts among public, civic, and scientist audiences and spur on future PES activities.
The Building with Biology project is an example of a national-scale PES activity, utilizing a set of standardized materials. ISE institutions in the European Union have participated in a number of similar large-scale projects, including Sea for Society15 and VOICES.16 Global projects such as World Wide Views17 or Play Decide have engaged participants in deliberation around the world, employing high- quality materials translated into dozens of languages, eliciting comparable responses and recommendations from participants.
These kinds of large-scale projects have clear benefits associated with them: the materials can be designed by networks of institutions with broad reach in their respective communities. By developing the materials centrally, the developers can efficiently get and use input from leading experts involved in PES. This helps to ensure that mutual learning will result from PES engagements with the vetted materials. The consistency of the materials allows for comparison and evaluation across institutions. Finally, dissemination can be powerful, since it can be achieved at a broad and visible scale.
However, the development of these large-scale engagements has primarily been top-down. Issue identification, framing, and content development are established at national or global scale and thus may not reflect immediate community priorities at any specific location. The issues most important to local communities working with their ISE partners may not be addressed by the nature of questions sought by scientific and policy leaders that may look to future needs rather than current ones.
One potential way to take advantage of the reach and power of large-scale projects while also addressing immediate community priorities would be to develop a battery of PES materials around a given topic that could be complemented with co-created questions of local relevance. In an ongoing National Oceanic and Atmospheric Administration (NOAA)- funded project led by Arizona State University and the Museum of Science (NA15SEC0080005) that is convening eight deliberations at ISE institutions around weather and climate-related hazards, four standard hazard modules are augmented with the inclusion of a locally focused resilience planning question, to be determined in collaboration by educators at the hosting institution and resilience planning partners.
One way of achieving the immediate community benefits in a national project might be for each participating ISE institution to create a citizen advisory board that could help to recommend locally relevant additions to large-scale PES materials. In this manner, national or global PES activities could spark a national or global conversation but also work to engage local PES audiences in somewhat different multi-directional conversations that would be relevant to the communities of the various host sites. ASTC’s recent International Science Center and Science Museum Day focused on the overarching topic of the United Nations’ Sustainable Development Goals, but left the actual activities for engagement up to the participating institutions.18
The European Commission in recent years has increasingly described PES in the context of Responsible Research and Innovation (RRI), with the objective “to foster mutual understanding and co-create research and innovation outcomes and policy agendas effective in tackling societal challenges.”19 An example of this was the PLACES project, organized by ECSITE as part of the Commission’s Horizon 2020 effort. PLACES created local partnerships between scientists, publics, and policymakers in an effort to “enhance the three-way conversation between science, policymakers and society.” Participants representing various perspectives and multifaceted expertise came together to create local action plans for cities and identified problems to solve. This model could be employed more broadly and paired with other kinds of ISE and PES offerings, such as citizen science or participatory exhibits, to tackle issues that are of mutual interest to community, scientific, and policy partners in new ways.20
Another possible way for PES materials to be customized would be for ISE institutions to create templates for more “evergreen” engagement materials, designed specifically for adoption by researchers over time at ISE institutions that can be made more specifically relevant by ISEs working locally. An example is the adoption of the PES materials by iGEM (International Genetically Engineered Machine) teams. iGEM encourages teams to take the creations of others and tweak or hack them for their own aims, so many iGEM teams adopted the Building with Biology materials in ways that better connected to their own individual work and projects.12 Could such “hackable” templates be designed at the outset?
To help drive forward some of the ideas described here, a group of ISE institutions could apply an array of PES materials centrally created in recent years (including this guide) to local researcher- community-ISE partnerships. These local partnerships would each convene events centered on co-created PES activities around issues that are of shared interest to local researchers and civic and community partners. Evaluation efforts could assess the mutual learning that occurs between these audiences through the co- created PES materials, and help to inform the creation of future templates and materials for locally focused PES activities.
Opportunities for PES in a fast-changing, polarized world
Topics that lie at the intersection of science and society are crucial for multidirectional engagement at a time when scientific and technological breakthroughs have wide-range impacts on everyone in our world. These societal issues, along with the fractured and polarized nature of public discourse, create a clear need for multidirectional learning between and among publics and scientists. Informal science education institutions have an important role to play in connecting communities with researchers and engineers, so that future products of science and technology reflect the priorities and concerns of a diverse and engaged public.
World Wide Views organizers presenting at a United Nations conference in India
One possible avenue for addressing this polarization is the co-created Forum model presented earlier in this chapter. The Museum of Science has received a recent NSF award to develop and explore this co- created Forum model (NSF DRL# 1811118) with science center partners from the Oregon Museum of Science and Industry, the Michigan Science Center, and the North Carolina Museum of Life and Science. The project will develop, test, refine and disseminate a model in which civic, community, and scientist partners collaboratively set the agenda for co-created forum activities. The convening role of the ISE institution in this process is depicted in the accompanying graphic.
A subsequent NOAA award (NA18SEC008000) to the one mentioned earlier will engage participants in collecting data pertaining to hazards selected in partnership with local civic planners, and discussing potential resilience strategies through deliberative dialogue. We are hopeful that combining citizen science with complementary PES dialogue components can help to inform future innovations in the PES landscape while helping to strengthen connections between science centers and local civic institutions.
+ - References
This chapter summarizes future directions and priorities for public engagement with science.
Hands-on Activities and Forums
The Museum of Science has created an online archive of Forum products at https://www.mos.org/pes/forum-archive. This resource includes basic descriptions of Forum programs as well as products and guides for hosting Forum programs on a variety of topics.
Building with Biology includes resources for hands-on activities and forum dialogue programs about synthetic biology.
NISE Net Forums include resources for four forums about nanotechnology, a discussion program focused around two 10-minute video-taped plays, a Science Café guide, the original NISE Net Forums Manual, and the Building with Biology materials noted above. http://www.nisenet.org/search/product_category/forums-13
NISE Net Science and Society training materials and hands-on activities bring PES content and practices to table-top cart activities. http://www.nisenet.org/catalog/nano-and-society-training-materials
The World Biotech Tour (WBT) was a multi-year initiative to bring biotechnology to life at select science centers and museums worldwide. The program, supported by the Association of Science- Technology Centers (ASTC) and Biogen Foundation, was scheduled to run from 2015–2017. The WBT increased the impact and visibility of biotechnology among youth and the general public through hands- on and discussion-led learning opportunities. http://www.worldbiotechtour.org/activities
World Wide Views has published resources it used for its citizen consultation events and which you could use to replicate them.
The PlayDecide website includes discussion games on a wide variety of controversial issues that can be downloaded and printed out. They are self-facilitating and allow you to add the results of your deliberations to an international online database. http://www.playdecide.eu
Climate Interactive creates interactive, scientifically rigorous tools that help people see connections, play out scenarios, and explore what works to address the biggest challenges we face. They have resources available for free including a mock-UN climate negotiation role-play simulation, online training in systems thinking to address climate change, webinars, and more. https://www.climateinteractive.org/
In 2011, the Museum of Science collected a sample of 201 activities submitted by 125 organizations as representative of their e orts at the time to step beyond public understanding of science into topics and processes that explore public engagement with science. The case summaries fell into 10 categories: art and theater, citizen science, exhibits, festivals, forums and science cafés, inquiry, media, meet the scientist, on-site research, reference, and take action. The catalog has not been updated since 2011 but can be downloaded at https://dimensionsofpes.wikispaces.com/Catalog+of+Case+Summaries
The Expert and Citizen Assessment of Science and Technology (ECAST) Network put on a program about asteroids for NASA, and the online version is available for use. https://ecastonline.consider.it/
Consider.it has other online dialogues on a range of topics and can also help you make your own. Participants can learn about a topic, share how much they agree or disagree, and give pros and cons for why they answered the way they did. https://consider.it/
Issues Guides and Training Materials
National Issues Forums Institute has issues guides (some free, some paid) on a variety of science and non-science issues. https://www.nifi.org/
National Coalition for Dialogue and Deliberation (NCDD) has a wealth of resources for beginners and experienced practitioners of dialogue and deliberation. http://ncdd.org/
Essential Partners (formerly Public Conversations Project) provides free training guides and dialogue resources, mostly on political topics, but some scientific topics. http://www.whatisessential.org/resources?type=All&title=&author=All
National Network for Ocean and Climate Change Interpretation (NNOCCI) is working to establish a national network of professionals who are skilled in communicating and translating climate and ocean science to broad public audiences. Their goal is to change the nature of public conversation about issues of climate change to be inviting, empowering, and solutions oriented. The website has guides, frameworks, recommendations, a multimedia toolkit, and a solutions bank. http://climateinterpreter.org/about/projects/NNOCCI
The Portal to the Public (PoP) provides strategies and resources to build local communities of scientists and educators who are dedicated to public engagement with current science. Their Implementation Manual is available for purchase ($40 as of May 2017). https://popnet.pacificsciencecenter.org/https://popnet.pacificsciencecenter.org/
Let’s Talk was a project at the University of Washington funded by IMLS. Their evaluation report and synthesis paper can be found here. http://www.informalscience.org/lets-talk-meta-conversation- about-dialogue-evaluation and https://www.nemanet.org/ files/8914/4587/2472/Lets_Talk_Symposium_DRAFT.pdf
The Expert and Citizen Assessment of Science and Technology (ECAST) Network has reports and toolkits about public participation in technology assessment. https://ecastnetwork.org/category/topics/toolkit/
Everyday Democracy guide: How to Develop Discussion Materials for Public Dialogue. https://www.everyday-democracy.org/sites/default/ files/attachments/How-Develop-Discussion-Materials-Public-Dialogue_Everyday-Democracy.pdf
Many Experts, Many Audiences: Public Engagement with Science is a seminal paper on public engagement with science in the informal science education field. http://www.informalscience.org/many- experts-many-audiences-public-engagement-science
National Coalition for Dialogue and Deliberation (NCDD) has a wealth of resources for beginners and experienced practitioners of dialogue and deliberation. http://ncdd.org/rc/
Public Agenda has “discussion starters” on science education, climate change, energy, health care, evangelical Christians and scientists, parental involvement in education, afterschool programs, and other topics. http://www.publicagenda.org/pages/choicework-homepage
Many Approaches to Public Engagement, from the AAAS Center for Public Engagement with Science & Technology. https://www.aaas.org/pes/value-public-engagement
Wellcome Trust, Planning your public engagement activities: Step by step guide. https://wellcome.ac.uk/sites/default/files/planning-engagement-guide-wellcome-nov14.pdf