A new open online course, “Program Evaluations for Connected Learning,” is being offered by the Digital Media and Learning (DML) Research Hub.
The free course, running from Jan. 11 through May 1, will be taught by William Penuel, professor of educational psychology and learning sciences at the University of Colorado at Boulder. The course will explore different theories and approaches to evaluate connected learning in the field. Connected learning is a way to approach education in the 21st century that takes advantage of today’s abundance of information and social connection.
“Since the 1960s, program evaluation has played an important role in educational policy making,” Penuel said. “Evaluation involves making evidence-based judgments about the worth and value of projects, programs, and policies. Sometimes, these judgments are intended to provide evidence of impact to policy makers at the federal, state, and local level and other stakeholders. Other judgments are intended to inform ongoing processes for continuous improvement of projects and programs.”
Course participants will be provided with practical experience in the process of educational program evaluation by exploring “real world” problems brought forth by youth-serving programs. The intent is to study real world scenarios and identify practical evaluation solutions.
“This course is geared toward people who are working on evaluations of their individual programs and wanting to gain some new insight, tricks of the trade about how to bring stakeholders into evaluation processes to design measures to writing better reports,” Penuel said. “It’s also for their partners who want to be smart consumers of evaluation or maybe looking to hire an evaluator. They will see great examples of evaluations and collaborations.”
Topics to be covered include: a theory of change for university-based programs housed in a center to promote civic and community engagement; how teams can collaborate to develop a set of survey tools intended to measure the outcomes of connected learning in a wide variety of programs; how large-scale evaluation studies can answer causal questions about program impact; and how to effectively communicate evaluation findings to stakeholders.
The focus will be on:
- how to negotiate the focus and scope of an evaluation;
- how to figure out if it’s a good idea to undertake an evaluation;
- how to design under real-world constraints of small budgets and limited human resources for evaluations;
- how to develop multiple forms of evidence about outcomes and implementation; and
- how to go about analyzing data and constructing reports that people can use to improve their programs.
The strength of connected learning programs — that they are free choice, voluntary, and people come and go as they please — is what makes them challenging to evaluate, Penuel said. “With people coming and going, we have to devise strategies for capturing the data we can and not interfere with what makes connected learning programs exciting.”
“Program Evaluations for Connected Learning” is DML Hub’s third open course offering. The others — “Connected Courses”; and DML Commons: “Professional Pathways” and “Design Research” — still are available online.
“Program Evaluations” will feature webinars and online discussions. Once the live course ends in May, it will remain online at dmlcommons.net/2016-course.
About the Professor
Penuel has conducted dozens of small- and large-scale evaluations of projects in education in the domains of mathematics, science, technology, and literacy. His expertise is relevant to evaluation in the areas of study design, assessment design and implementation research. He is on the editorial board for the American Journal of Evaluation.
To learn more about the course and sign up, visit dmlcommons.net.
In 2013, the DML Hub, as part of the MacArthur Foundation-supported Digital Media and Learning Initiative, launched Reclaim Open Learning, an effort to explore the intersections between higher education, open learning, and the connected learning model in the midst of MOOC mania. The effort focused on returning to core pedagogical and learning principles, the ethos of the open web, and end-to-end faculty and student innovation when attention was shifting to large institutionalized initiatives and old-school, top-down pedagogical and learning models.
In 2014, the DML Hub offered “Connected Courses,” an open, online class teaching higher education faculty ways of developing and teaching connected courses.
About DML Hub
The Research Hub was created in 2009, and its mission is to advance research in the service of a more equitable, participatory, and effective ecosystem of learning, keyed to the digital and networked era. Located at the systemwide University of California Humanities Research Institute at UC Irvine, it is an international research center that is committed to promoting compelling research collaborations about best participatory learning practices, applications, programs and their assessments that engage digital media. All of its activities — which include original research, blogs, websites, a webinar series, publications, online courses and an annual conference — are supported by the John D. and Catherine T. MacArthur Foundation’s Digital Media and Learning Initiative.
Mimi Ko Cruz