Projects / The Collaborative Assessment and Feedback Engine (CAFE)

The Collaborative Assessment and Feedback Engine (CAFE)

Connected Communities

The CITRIS Connected Communities Initiative supports collaborative discovery, design, and governance through new technologies that enhance...
Screenshot-CRCintro California Report Card

The Collaborative Assessment and Feedback Engine (CAFE)

The Collaborative Assessment and Feedback Engine (CAFE) is an open source, e-participation platform that provides participants dynamic visual feedback about their position on key social issues, relative to other participants. The platform applies statistical models and collaborative filtering to rapidly discover emerging trends as data is collected. By fostering open-ended dialogue and facilitating a more nuanced assessment of public opinion about complex issues, CAFE enables more informed organizational decisions while increasing participant engagement in decision-making processes.

The CAFÉ platform has been used to assess government performance in California and Mexico, to crowdsource insights on the effectiveness of family planning trainings in Uganda, and to collect timely feedback from students on how MOOCs can be improved.

QuakeCAFE

QuakeCAFE in handQuakeCAFE (http://quakecafe.org) is a web-based platform that combines peer-to-peer review with statistical models to streamline and structure public feedback. The platform facilitates self-assessment of disaster preparedness, provides feedback about ways people can better prepare, encourages participants to suggest ideas for ways the California State government can improve earthquake preparedness, and allows participants to rate the usefulness of each others’ suggestions.

QuakeCAFE emerged from the California Report Card (californiareportcard.org), an English and Spanish mobile platform that allows participants to assess and give advice on timely policy issues at the state level. Over 22,000 participants from all 58 counties have assigned over 40,000 grades and suggested over 500 issues for increased priority. Among these, disaster preparedness generated consistent interest across the State, prompting Lt. Gov. Newsom to encourage CITRIS to develop QuakeCAFE.

QuakeCAFE is free to use, works on any mobile device or personal computer connected to the Internet.

Quake CAFE is a collaborative effort between the Connected Communities Initiative at the Center for Information Technology Research in the Interest of Society (CITRIS), based at UC Berkeley and the Office of the Lieutenant Governor of California.

The California Report Card

Screenshot-CRCintro California Report CardLaunched in early February 2014, the California Report Card (http://californiareportcard.org) is an innovative, bilingual social engagement tool that provides an alternative to opinion surveys and social media channels often dominated by partisans. The California Report Card is a mobile-friendly platform that combines peer-to-peer review with statistical models to streamline and structure public feedback.

The California Report Card uses the latest IT and mobile technology to connect voters with state leaders and build new bridges between California’s Spanish- and English-speaking communities. The CRC rapidly translates between English and Spanish, reaching the 30% of Californians who speak Spanish at home.

Since its launch, over 22,000 Californians from all 58 counties have assigned more than 50,000 grades and suggested topics for future Report Cards. The platform has the potential to increase public engagement by tapping the collective intelligence of Californians. It provides a forum for residents to express their views and insights, consider the views of others and encourages elected leaders to act on the best of their ideas.

 

México Participa

MexicoParticipaTo engage voters and alert them to topical issues, the CITRIS Connected Communities Initiative is collaborating with faculty at Tec de Monterrey and the Mexico National Electoral Institute to design and implement México Participa (mxparticipa.org), an innovative civic engagement platform.

México Participa allows participants to voice their concerns in an anonymous public forum, where they can share their opinions and priorities with fellow citizens and the leaders who represent them. Participants are asked to rate the country’s performance on timely issues, including: corruption, public safety, educational opportunities, formal employment, public health services, and presence of violence. Participants also suggest an issue that deserves increased national priority and rate the suggestions of others, offering candidates and incumbents timely feedback on the changing priorities of their constituents.

México Participa was launched in mid-April 2015 and currently has over 3,000 participants. More information is available at http://citris-uc.org/mexico-participa-platform/.

 

DevCAFE

Uganda-CAFEDevCAFE is a participatory platform that engages citizens and communities from developing regions to collectively assess conditions, needs, and outcomes of development projects

DevCAFE was used to assess the effectiveness of family planning and reproductive health trainings at three health centers in Uganda in June 2014 and July 2015. The platform features a visual- and voice-based user interface that allows women participants to evaluate the opportunities and barriers to family planning adoption, audio record their suggestions for improving the effectiveness of FPRH trainings, and evaluate the importance of others’ suggestions.

DevCAFE applies statistical models and collaborative filtering to enable rapid identification of key insights while in the field. Current insights reveal continued fear of particular family planning methods, but that participants would like to increase the frequency of trainings to reduce local misconceptions. More information is at http://citris-uc.org/uganda-cafe-platform/.

This work was supported by the Blum Center for Developing Economies and the Development Impact Lab (USAID Cooperative Agreement AID-OAA-A-12-00011), part of the USAID Higher Education Solutions Network; UC Berkeley’s Algorithms, Machines, and People Lab; and the UC CITRIS Connected Communities Initiative.

 

M-CAFE

MOOC-CollaborativeAssessmentFeedbackEngineLack of efficient ongoing course evaluation platforms and the outdated end-of-course paper-based evaluations necessitate a new, more efficient and effective feedback platform.

The MOOC Collaborative Assessment and Feedback Engine (M-CAFE) is a mobile-friendly platform that encourages students to check in weekly to numerically assess their own performance, provide textual ideas about how the course might be improved, and rate ideas from other students. M-CAFE applies statistical models and collaborative filtering to provide rapid feedback to students on how they are positioned relative to others and display trends and potentially valuable ideas for instructors.

M-CAFE has been used in two edX MOOCS and an on-campus undergraduate course at UC Berkeley. Feedback from over 700 students has been collected, including nearly 1,500 quantitative evaluations and over 500 textual ideas.

Weekly updates on participation statistics, visualizations of the quantitative ratings and top-rated student ideas provide instructors with timely feedback to enable more informed decision making throughout the course.

More information is at http://m-cafe.org.

Links:

  1. The California Report Card: http://californiareportcard.org
  2. Mexico Participa: http://mxparticipa.org
  3. M-CAFE: http://m-cafe.org
  4. Uganda CAFE: http://citris-uc.org/uganda-cafe-platform/

Project Collaborators:

Publications:

DevCAFE 1.0: A Participatory Platform for Assessing Development Initiatives in the Field. Brandie Nonnecke*, Sanjay Krishnan*, Jay Patel, Mo Zhou, Laura Byaruhanga, Dorothy Masinde, Maria Elena Meneses, Alejandro Martin del Campo, Camille Crittenden, Ken Goldberg. Proceedings of the IEEE Global Humanitarian Technology Conference (GHTC), Seattle, WA. Oct 2015. [.pdf] [Recipient of Best Paper Award]

M-CAFE 1.0: Motivating and Prioritizing Ongoing Student Feedback During MOOCs and Large on-Campus Courses using Collaborative Filtering. Mo Zhou, Alison Cliff, Sanjay Krishnan, Brandie Nonnecke, Camille Crittenden, Kanji Uchino, Ken Goldberg. Proceedings of the 16th Annual ACM Conference on Information Technology Education, SIGITE 15, Chicago, September, 2015. [.pdf]

Motivating and Prioritizing Ongoing Student Feedback During MOOCs and Large on-Campus Courses using Collaborative Filtering. Mo Zhou, Alison Cliff, Sanjay Krishnan, Brandie Nonnecke, Camille Crittenden, Kanji Uchino, Ken Goldberg. Proceedings of the 16th Annual ACM Conference on Information Technology Education, SIGITE 15, Chicago, September, 2015. [.pdf]

M-CAFE: Managing MOOC Student Feedback with Collaborative Filtering. Mo Zhou, Alison Cliff, Allen Huang, Sanjay Krishnan, Brandie Nonneche, Kanji Uchino, Sam Joseph, Armando Fox, Ken Goldberg. Proceedings of the 2nd ACM Conference on Learning At Scale (L@S 2015), Vancouver, BC, Canada, March 2015. [.pdf]

A Mobile Platform for Participatory Assessment of Development Programs and a Case Study in Uganda. Brandie Nonnecke, Sanjay Krishnan, Amit Akula, Alison Cliff, Allen Huang, Angela Lin, Shir Nehama, Laura Byaruhanga, Dorothy Masinde, Camille Crittenden, Ken Goldberg. Humanitarian Technology Conference: Science, Systems and Global Impact. HumTech2015. Elsevier. Boston, MA. May 2015. [.pdf]

Comparing Three Online Civic Engagement Platforms using the Spectrum of Public Participation. Matti Nelimarkka, Brandie Nonnecke, Sanjay Krishnan, Tanja Aitamurto, Allen Ching-Chang Huang, Gavin Newsom, Conrad Gregory, Jay Patel, Daniel Catterson, Camille Crittenden, John Scott, Chris Garland, and Ken Goldberg. University of Oxford: Internet, Policy, and Politics Conference on Crowdsourcing for Politics and Policy (IPP2014). Oxford, UK. Sept, 2014. [.pdf]

This work was supported by the Blum Center for Developing Economies and the Development Impact Lab (USAID Cooperative Agreement AID-OAA-A-12-00011), part of the USAID Higher Education Solutions Network; UC Berkeley’s Algorithms, Machines, and People Lab; and the UC CITRIS Connected Communities Initiative.