DLESE Peer Review Criteria

DLESE Scientific and Pedagogical Review Questions

Purpose of Review

To evaluate the scientific accuracy and currency of the resource.

To evaluate the pedagogical effectiveness of the resource.

To evaluate the usability of the resource.

Scientific accuracy and currency

Scientific accuracy and currency of educational resources is a primary concern of K-16 educators who may be teaching out of subject or have a need to teach about topics new to them. In these situations, users desire assurances from experts of the quality of the science content.

Accuracy includes the specific content and how it is presented. For example, maps and graphs must be properly labeled for one to ascertain whether they are accurate and current. Resources that include models, simulations and research results must be accompanied by information about the assumptions and methods used in their development, and should communicate the uncertainty and unknowns surrounding a problem or issue. Additionally, the science should be without political, religious or social bias, or if a bias is intended, it should be clearly indicated.

Currency is a measure of how well a topic represents current understanding, debate, and new directions of research about a topic. In rapidly evolving content areas such as global climate change, a resource may become outdated in just a few years, while in more mature fields the fundamental science content is better known and often is being refined rather than defined. In some cases, the science content may be out-of-date, yet nevertheless useful for teaching about how scientific thought has evolved, so long is this is clearly identified.

It is recognized that some digital resources may deal solely with pedagogical approaches to learning; in such cases, an evaluation of science accuracy and currency may not be relevant.

Reviewers are asked to consider the following in their evaluation of scientific accuracy and currency.

Is the scientific information factually accurate as evaluated in the context of the intended audience? What is the nature of any factual or other errors?

Have the science methods and any mathematical development been clearly and accurately presented for broad understanding?

Is the science content consistent with current scientific knowledge? If not, describe instances of out-of-date information. If appropriate, indicate whether the resource is nevertheless potentially useful when teaching about how scientific thought has evolved.

Are the graphs, charts, images, and animations clearly labeled and narrated, including color keys where appropriate? Are the units of measurement identified and or described?

Is the resource fair and accurate in describing scientific problems and issues? Does the resource appropriately communicate the uncertainty and unknowns surrounding a problem or issue? Describe any instances where the user would benefit from an improved treatment within the resource.

Are there any political, religious, or commercial messages in the resource? If so, describe them and provide your opinion on whether they detract from the value of the resource.

Usability

Usability refers to whether the resource as presented can readily be applied in an educational setting. It also includes an evaluation of the organization and logic of the resource design, and the ease with which it is navigated and understood in itself and within the context of the portal in which it is embedded.

Reviewers are asked to consider the following in their evaluation of the usability of the resource.

How well is the subject matter in the resource organized and logically presented?
Please describe any sections within the resource that could be shortened, deleted, or expanded to improve its educational usability for the intended audience.
Can most educators use this resource as written, without need for revision or enhancement? If not, explain weaknesses in the design that prevent use of the resource as written.
How easy is it to logically and efficiently navigate within the resource and within any larger site or portal environment in which the resource is found. Describe any changes that might be made to improve navigation and thus understanding of the resource presentation.
To the best of your knowledge, do similar resource exist, and if so, please comment on the strengths and weaknesses of the current resource as compared to others with similar content. Please provide the URL or citation for the other resource(s).

Pedagogical Effectiveness

Pedagogical effectiveness can be evaluated from two perspectives: 1) the effectiveness of the resource in supporting an educator desiring to teach with the resource and 2) the pedagogical effectiveness for student learning.

Reviewers are asked to consider the following in their evaluation of pedagogical effectiveness.

How well does this resource incorporate the pedagogical methods known to be effective for the learning proposed? How might the pedagogical methods be improved?
Will students be actively (mentally) engaged in practicing or applying the concepts to be learned in this resource? Is there a logical progression from lower order thinking and process skills to higher order thinking and process skills?
Does the resource contain sufficient pedagogical support for the educator to effectively implement it in a learning environment?
If the resource contains student work, is it clearly identified as such? Describe the value of the student work to users of this resource.

Recommendation

Reviewers are asked to select one of the following as an overall recommendation:

Accept
Accept with minor revisions
Accept with major revisions
Reject

Summary

Reviewers are asked to provide short answers to the following questions:

Does this resource make an important contribution to Earth system education?
Would you recommend this resource to a colleague?
If you recommend major revisions for this resource, would you be willing to review it again when resubmitted?