How Should I Evaluate Quality?

Even prior to searching for content, it is important to think about what separates high-quality instructional materials from the average. Why consider quality before searching? Consider these statistics:

"Improving the quality of curriculum is 40x more effective than class-size reduction" (Boser et al., 2015).

In a single school year, the average student spends 80% of their available time on assignments that are not high quality (TNTP, 2018).

 

In this lesson, we'll look at a number of lenses you can use to evaluate learning materials, as well as established quality assurance rubrics to assist. It is important to note that items appearing in Open Space have achieved the standards in our Open Space Submission Rubric and the majority of content found in Educator Tools are also standards-aligned.

 

image with four circles and the words research-based strategies, flexability and adaptability, usability, and inclusive teaching

RemotEDx Rubric for Instructional Materials

The RemotEDx Instructional Material Review rubric contains four categories of criteria: Usability, Flexibility and Adaptability, Inclusive Teaching, and Research-Based Strategies. Materials that have been reviewed using this rubric can be found on in the RemotEDx collection on Open Space. The detailed reviews themselves can be searched on the RemotEDx Exchange platform. Let's take a look at a selection of the criteria that you can keep in mind before beginning your search, with a couple of more essential items highlighted yellow:

Research-Based Strategies

  • Content is aligned to Ohio Learning Standards.
  • Content provides opportunities for authentic learning.
  • Content engages students and supports building toward independence. 
  • Content includes options for differentiation to meet needs of all learners.

Usability

  • Captions and/or transcripts for video or audio content.
  • Allows for low-bandwidth instructional strategies.
  • Features read-aloud option and ability to adjust font size if text-based.
  • Visual design is clean and coherent.

Flexibility and Adaptability

  • Materials are flexible to allow students to access and complete work online or offline as needed.
  • Material include teacher supports, documentation, and/or guides for effective use.
  • Material is in a format that allows for modifications.

Inclusive Teaching

  • Content is presented with an objective view on topic and is free of bias.
  • Content creates student experiences that enable all children to reach empowering and rigorous learning outcomes regardless of their race or income.
  • Content cultivates an awareness and acceptance of a variety of ages, cultures, races, religions, and gender roles and identities.

 

logo for Achieve

Achieve.org OER Rubrics

To help states, districts, teachers, and other users determine the degree of alignment of Open Educational Resources (OER) to college- and career-ready standards and to determine other aspects of quality of OER, Achieve has developed eight rubrics in collaboration with leaders from the OER community.

  • Degree of Alignment to Standards - How closely aligned to the learning standard is this resource? Standards could be common core, next generation science, or other.
  • Quality of Explanation of the Subject Matter - How thoroughly is the subject matter explained in this resource?
  • Utility of Materials Designed to Support Teaching - Is this resource designed to support teachers in planning and presenting the resource? Is it easy for teachers to understand and use?
  • Quality of Assessment - Are there assessments included in this resource that determine what a student knows before, during, and after the subject is taught?
  • Quality of Technological Interactivity - Is technological interactivity included in this resource?
  • Quality of Instructional and Practice Exercises - Are exercises designed to provide an opportunity to practice and strengthen specific skills and knowledge?
  • Opportunities for Deeper Learning - Are students engaged in any of the deeper learning skill areas below? 
  • Assurance of Accessibility - Is the material accessible to all students, including students identified as hearing or visually impaired.

Access the complete Achieve.org OER Rubrics document to view all details.

 

Ohio Materials Matter and EdReports

Ohio Materials Matter Reviews, available from the RemotEDx Curriculum Library, is Ohio's curriculum review database. The Ohio Department of Education, EdReports, and INFOhio collaborated on the project. The database contains reports on English language arts, mathematics, and science curriculums from EdReports, a nonprofit that provides reviews of K-12 instructional materials. Learn more about how to use EdReports and its curriculum adoption process in the INFOhio Learning Pathway class, EdReports.  

 

Additional Lenses

Accessibility and Inclusive Design

Ensure your materials and activities are accessible by all learners so nobody is left out. The quality assurance standards and mechanisms already mentioned above include criteria for accessibility. For thoroughness and to provide you with additional helpful resources, consider these other resources when looking at issues of accessibility and inclusive design: Universal Design for Learning (UDL) Guidelines and Making Learning Accessible for All: An Educator's Guide.

OER Starter Kit 

The OER Starter Kit handbook offers a number of criteria consider, including:

  • Clarity, Comprehensibility, and Readability
  • Content and Technical Accuracy
  • Adaptability and Modularity
  • Appropriateness and Fit
  • Accessibility

For convenience, they have made these criteria into a Evaluating OER Checklist available to copy via Google Docs.

OER Course on OpenLearn

The Creating Open Educational Resources course on OpenLearn suggests that good OER is:

  • findable – it can be in multiple locations
  • clearly described
  • clearly licensed (normally through Creative Commons)
  • from a source you trust
  • easy to modify
  • free-standing – it does not assume knowledge of other resources
  • free of copyright content
  • being used by/recommended by people like you
  • imperfect – it just needs to work for you

When searching for and evaluating new materials, the attributes that OpenLearn suggests including are:

  • accuracy
  • reputation of author/institution
  • standard of technical production
  • accessibility
  • fitness for purpose
  • clear rights declarations, e.g. Creative Commons.

 

books icon

Additional Resources

To help make more sense of evaluating instructional materials and how this idea fits into larger curricular adoption decisions, consider taking INFOhio's High-Quality Instructional Materials to Supplement Your Curriculum class, available at no cost and for 5 contact hours upon completion.

 

This is a quotation box
 
Need Help?
CONTACT SUPPORT
Open ISearch  
ISearch - Advanced Library & Resource Search
Fetch - Library Catalog Search

Fetch is avaiable to INFOhio automated schools. If you are an INFOhio school, please log in with your school username/password using the button at the top-left corner of this page.

For more information about Fetch, please visit the Fetch information page or contact INFOhio support at https://support.infohio.org.