Ken Goldberg and Brandie Nonnecke and Sanjay Krishnan and Camille Crittenden
The CAFE platform was awarded the 2017-2018 Chancellor’s Award for Research in the Public Interest, which recognizes research that addresses critical needs and issues affecting local, state, national, or global communities.
*The CAFE project has concluded after 9 years
The Collaborative Assessment and Feedback Engine (CAFE) is an open source, e-participation platform that provides participants dynamic visual feedback about their position on key social issues, relative to other participants. The platform applies statistical models and collaborative filtering to rapidly discover emerging trends as data is collected. By fostering open-ended dialogue and facilitating a more nuanced assessment of public opinion about complex issues, CAFE enables more informed organizational decisions while increasing participant engagement in decision-making processes.
The CAFE platform has been used to assess government performance in California and Mexico, to crowdsource insights on the effectiveness of family planning trainings in Uganda, to gather ideas on how to improve typhoon preparedness in the Philippines, and to collect timely feedback from students on how MOOCs can be improved.
In contrast to polls and surveys that are extremely valuable for analysts, CAFE is user-facing: the CAFE platform provides direct, immediate visual feedback to participants, allowing them to instantly understand where they stand on key issues with respect to other participants and encourage open-ended / qualitative responses that can enhance interaction between users. Like a coffeehouse, CAFE operates as a grassroots, user-friendly, self-organizing environment that allows the wisdom of crowds to identify, highlight, and display patterns, trends, and insights as they emerge. To facilitate interactions, CAFE incorporates techniques from deliberative polling, collaborative filtering, and multidimensional visualization.
DevCAFE
DevCAFE is a participatory platform that engages citizens and communities from developing regions to collectively assess conditions, needs, and outcomes of development projects
DevCAFE was used to assess the effectiveness of family planning and reproductive health trainings at three health centers in Uganda in June 2014 and July 2015. The platform features a visual- and voice-based user interface that allows women participants to evaluate the opportunities and barriers to family planning adoption, audio record their suggestions for improving the effectiveness of FPRH trainings, and evaluate the importance of others’ suggestions.
DevCAFE applies statistical models and collaborative filtering to enable rapid identification of key insights while in the field. Current insights reveal continued fear of particular family planning methods, but that participants would like to increase the frequency of trainings to reduce local misconceptions. More information is at https://citris-uc.org/uganda-cafe-platform/.
This work was supported by the Blum Center for Developing Economies and the Development Impact Lab (USAID Cooperative Agreement AID-OAA-A-12-00011), part of the USAID Higher Education Solutions Network; UC Berkeley’s Algorithms, Machines, and People Lab; and the UC CITRIS Connected Communities Initiative.
Malasakit
As the second most at-risk-country for disasters worldwide, the Philippines has established a nationwide Disaster Risk Reduction Framework and Plan, yet this initiative still lacks an efficient and effective mechanism for evaluating disaster risk, response and rehabilitation. Building on our previous work on the Development Collaborative Assessment and Feedback Engine (DevCAFE) platform developed for crowdsourcing opinions, ideas, and other input from local populations on the effectiveness of development interventions, we developed a disaster risk reduction (DRR) digital toolkit and methodology called “Malasakit” (meaning “to care” in Filipino). Malasakit is a customizable participatory assessment platform that collects and integrates quantitative assessment, qualitative feedback, and peer-to-peer collaborative filtering on ways local communities can become better prepared for typhoons and floods. You can access Malasakit, our data, and publications at ibahagi.national-u.edu.ph.
Malasakit is supported by the Philippine-California Advanced Research Institutes (PCARI) (Project #IIID-2015-07) and the UC CITRIS Connected Communities Initiative.
Malasakit was the 2018 runner-up for the Pacific Telecommunications Council (PTC) Best Quality of Life Improvement Award, which recognizes a telecommunications product or network-centric ICT application that has had the most outstanding impact on improving the quality of life for people in the Pacific Rim.
QuakeCAFE
QuakeCAFE (http://quakecafe.org) is a web-based platform that combines peer-to-peer review with statistical models to streamline and structure public feedback. The platform facilitates self-assessment of disaster preparedness, provides feedback about ways people can better prepare, encourages participants to suggest ideas for ways the California State government can improve earthquake preparedness, and allows participants to rate the usefulness of each others’ suggestions.
QuakeCAFE emerged from the California Report Card (californiareportcard.org), an English and Spanish mobile platform that allows participants to assess and give advice on timely policy issues at the state level. Over 22,000 participants from all 58 counties have assigned over 40,000 grades and suggested over 500 issues for increased priority. Among these, disaster preparedness generated consistent interest across the State, prompting Lt. Gov. Newsom to encourage CITRIS to develop QuakeCAFE.
QuakeCAFE is free to use, works on any mobile device or personal computer connected to the Internet.
Quake CAFE is a collaborative effort between the Connected Communities Initiative at the Center for Information Technology Research in the Interest of Society (CITRIS), based at UC Berkeley and the Office of the Lieutenant Governor of California.
The California Report Card
Launched in early February 2014, the California Report Card (http://californiareportcard.org) is an innovative, bilingual social engagement tool that provides an alternative to opinion surveys and social media channels often dominated by partisans. The California Report Card is a mobile-friendly platform that combines peer-to-peer review with statistical models to streamline and structure public feedback.
The California Report Card uses the latest IT and mobile technology to connect voters with state leaders and build new bridges between California’s Spanish- and English-speaking communities. The CRC rapidly translates between English and Spanish, reaching the 30% of Californians who speak Spanish at home.
Since its launch, over 22,000 Californians from all 58 counties have assigned more than 50,000 grades and suggested topics for future Report Cards. The platform has the potential to increase public engagement by tapping the collective intelligence of Californians. It provides a forum for residents to express their views and insights, consider the views of others and encourages elected leaders to act on the best of their ideas.
México Participa
To engage voters and alert them to topical issues, the CITRIS Connected Communities Initiative is collaborating with faculty at Tec de Monterrey and the Mexico National Electoral Institute to design and implement México Participa (mxparticipa.org), an innovative civic engagement platform.
México Participa allows participants to voice their concerns in an anonymous public forum, where they can share their opinions and priorities with fellow citizens and the leaders who represent them. Participants are asked to rate the country’s performance on timely issues, including: corruption, public safety, educational opportunities, formal employment, public health services, and presence of violence. Participants also suggest an issue that deserves increased national priority and rate the suggestions of others, offering candidates and incumbents timely feedback on the changing priorities of their constituents.
México Participa was launched in mid-April 2015 and currently has over 3,000 participants. More information is available at https://citris-uc.org/mexico-participa-platform/.
M-CAFE
Lack of efficient ongoing course evaluation platforms and the outdated end-of-course paper-based evaluations necessitate a new, more efficient and effective feedback platform.
The MOOC Collaborative Assessment and Feedback Engine (M-CAFE) is a mobile-friendly platform that encourages students to check in weekly to numerically assess their own performance, provide textual ideas about how the course might be improved, and rate ideas from other students. M-CAFE applies statistical models and collaborative filtering to provide rapid feedback to students on how they are positioned relative to others and display trends and potentially valuable ideas for instructors.
M-CAFE has been used in two edX MOOCS and an on-campus undergraduate course at UC Berkeley. Feedback from over 700 students has been collected, including nearly 1,500 quantitative evaluations and over 500 textual ideas.
Weekly updates on participation statistics, visualizations of the quantitative ratings and top-rated student ideas provide instructors with timely feedback to enable more informed decision making throughout the course.
More information is at http://m-cafe.org.
Opinion Space: Social Media for Collective Discovery
Developed at UC Berkeley, “Opinion Space” is the precursor to CAFE. It is a social media
technology designed to help communities generate ideas and discover insights into important issues and policies. A version of Opinion Space was used by the U.S. State Department for 3 years, attracting thousands of participants from around the world to contribute, visualize, and review constructive new ideas on foreign policy.
“Opinion Space will harness the power of connection technologies to provide a unique forum for international dialogue. This is…an opportunity to extend our engagement beyond the halls of government directly to the people of the world.” – U.S. Secretary of State Hillary Clinton 2010
Opinion Space is a self-organizing system that uses an intuitive graphical “map” that displays patterns, trends, and insights as they emerge and employs the wisdom of crowds to identify and highlight the most insightful ideas.
Opinion Space helps organizations and companies: 1) Understand the diversity of their communities, 2) Solicit feedback and creative suggestions on specific topics, 3) Rapidly identify the most insightful ideas and suggestions, and 4) Increase satisfaction and engagement with their communities.
Opinion Space helps participants: 1) Visualize their relationships to other participants, 2) Express thoughtful ideas and suggestions about emerging issues and products, 3) Engage in friendly competition with other participants, and 4) Learn and gain insights from other participants.
Opinion Space uses a game model that incorporates techniques from deliberative polling, collaborative filtering, and multidimensional visualization.
It works like this: new participants use graphical sliders to express the degree to which they agree or disagree with five baseline statements such as: “I’m very interested in issue A”, “I am an active user of product B”. These responses are combined to display the new participant as a unique point in a map. The map is not based on predetermined categories, but on similarity of interests, behavior, and perspectives. It is designed to “depolarize” discussions by including all participants on a single level playing field.
Participants click on the points of other community members to read ideas and suggestions on discussion topics such as “What new approaches could be used to address issue C? “What features would you like to see in a new version of product D?” Participants evaluate the ideas of others and enter their own ideas. Participants earn points as Reviewers based on how they evaluate the ideas of others and earn points as Authors based on how others rate their own ideas.
More info at http://opinion.berkeley.edu/.
Publications
Malasakit 2.0: A participatory online platform with feature phone integration and voice recognition for crowdsourcing disaster risk reduction strategies in the Philippines. Brandie Nonnecke, Shrestha Mohanty, Andrew Lee, Jonathan Lee, Sequoia Beckman, Justin Mi, Thanatcha Panpairoj, Jeffrey Rosario Ancheta, Hilary Martinez, Nathaniel Oco, Rachel Edita Roxas, Camille Crittenden, and Ken Goldberg. IEEE Global Humanitarian Technology Conference (GHTC). San Jose, CA. October 2018. [.pdf]
Malasakit 1.0: A Participatory Online Platform for Crowdsourcing Disaster Risk Reduction Strategies in the Philippines. Brandie Nonnecke, Shrestha Mohanty, Andrew Lee, Jonathan Lee, Sequoia Beckman, Jonathan Mi, Sanjay Krishnan, Rachel Edita Roxas, Nathaniel Oco, Camille Crittenden, & Ken Goldberg. Proceedings of the IEEE Global Humanitarian Tech Conference (GHTC). San Jose, CA. Oct. 2017 [.pdf]
DevCAFE 1.0: A Participatory Platform for Assessing Development Initiatives in the Field. Brandie Nonnecke*, Sanjay Krishnan*, Jay Patel, Mo Zhou, Laura Byaruhanga, Dorothy Masinde, Maria Elena Meneses, Alejandro Martin del Campo, Camille Crittenden, Ken Goldberg. Proceedings of the IEEE Global Humanitarian Technology Conference (GHTC), Seattle, WA. Oct 2015.[.pdf][Recipient of Best Paper Award]
M-CAFE 1.0: Motivating and Prioritizing Ongoing Student Feedback During MOOCs and Large on-Campus Courses using Collaborative Filtering. Mo Zhou, Alison Cliff, Sanjay Krishnan, Brandie Nonnecke, Camille Crittenden, Kanji Uchino, Ken Goldberg. Proceedings of the 16th Annual ACM Conference on Information Technology Education, SIGITE 15, Chicago, September, 2015. [.pdf]
Motivating and Prioritizing Ongoing Student Feedback During MOOCs and Large on-Campus Courses using Collaborative Filtering. Mo Zhou, Alison Cliff, Sanjay Krishnan, Brandie Nonnecke, Camille Crittenden, Kanji Uchino, Ken Goldberg. Proceedings of the 16th Annual ACM Conference on Information Technology Education, SIGITE 15, Chicago, September, 2015. [.pdf]
M-CAFE: Managing MOOC Student Feedback with Collaborative Filtering. Mo Zhou, Alison Cliff, Allen Huang, Sanjay Krishnan, Brandie Nonneche, Kanji Uchino, Sam Joseph, Armando Fox, Ken Goldberg. Proceedings of the 2nd ACM Conference on Learning At Scale (L@S 2015), Vancouver, BC, Canada, March 2015. [.pdf]
A Mobile Platform for Participatory Assessment of Development Programs and a Case Study in Uganda. Brandie Nonnecke, Sanjay Krishnan, Amit Akula, Alison Cliff, Allen Huang, Angela Lin, Shir Nehama, Laura Byaruhanga, Dorothy Masinde, Camille Crittenden, Ken Goldberg. Humanitarian Technology Conference: Science, Systems and Global Impact. HumTech2015. Elsevier. Boston, MA. May 2015. [.pdf]
Comparing Three Online Civic Engagement Platforms using the Spectrum of Public Participation. Matti Nelimarkka, Brandie Nonnecke, Sanjay Krishnan, Tanja Aitamurto, Allen Ching-Chang Huang, Gavin Newsom, Conrad Gregory, Jay Patel, Daniel Catterson, Camille Crittenden, John Scott, Chris Garland, and Ken Goldberg. University of Oxford: Internet, Policy, and Politics Conference on Crowdsourcing for Politics and Policy (IPP2014). Oxford, UK. Sept, 2014. [.pdf]
Using a Social Media Platform to Explore How Social Media Can Enhance Primary and Secondary Learning. Sanjay Krishnan, Yuko Okubo, Kanji Uchino, and Ken Goldberg. Learning International Networks Consortium (LINC) 2013 Conference. MIT, Cambridge, MA. June 2013. [.pdf]
Distributed Spectral Dimensionality Reduction for Visualizing Textual Data. Sanjay Krishnan and Ken Goldberg. International Conference on Machine Learning (ICML) Workshop on Spectral Learning Methods, Atlanta, GA, June 2013. [.pdf]
This work was supported by the Blum Center for Developing Economies and the Development Impact Lab (USAID Cooperative Agreement AID-OAA-A-12-00011), part of the USAID Higher Education Solutions Network; UC Berkeley’s Algorithms, Machines, and People Lab; and the UC CITRIS Connected Communities Initiative.
