Evaluation Methods

Digital storytelling is more engaging, interesting, and significant for students educational needs than that of traditional storytelling. Sara Armstrong, an educator and consultant, believes that through digital storytelling students are provided the opportunity to personalize their education and learning. It encourages children to be inquisitive and ask questions related to the story or concept. Why I am telling this story? What are the elements or major themes of this story? How accurate is my portrayal? The most profound way to evaluate a digital storytelling program is to determine if it can tie into the curriculum and strengthen students' critical thinking, comprehension, report writing, and media literacy skills. Pre-testing and post-testing is one possible solution to collecting data for evaluative purposes. However, a rubric can be used to successfully evaluate the program. It is an instrumental tool that is used to evaluate the quality of the students' work on a complex project. An example rubric was taken directly from http://www.coe.uh.edu/digitalstorytelling/.

To use the rubric to evaluate student work on Digital Storytelling, you should determine the number of points achieved in each of 10 categories. Then add the points up for all 10 categories and multiply by 2.5. The lowest score possible is a "25" and the highest is "100." Please note that a score of at least a "3" in a category must be achieved for the quality of the student's work to be considered satisfactory in that category.

 

Category 4 Points 3 Points 2 Points 1 Point
1. Purpose of Story Establishes a purpose early on and maintains a clear focus throughout. Establishes a purpose early on and maintains focus for most of the presentation. There are a few lapses in focus, but the purpose is fairly clear. It is difficult to figure out the purpose of the presentation.          
2. Point of View The point of view is well developed and contributes to the overall meaning of the story. The point of view is stated but does not connect with each part of the story, although an attempt is made to connect it to the overall meaning of the story. The point of view is stated but no attempt is made to connect it to the overall meaning of the story. The point of view is only hinted at, or is difficult to discern.
3. Dramatic Question A meaningful dramatic question is asked and answered within the context of the story. A dramatic question is asked but not clearly answered within the context of the story. A dramatic question is hinted at but not clearly established within the context of the story. Little or no attempt is made to pose a dramatic question or answer it.
4. Choice of Content Contents create a distinct atmosphere or tone that matches different parts of the story. The images may communicate symbolism and/or metaphors. Contents create an atmosphere or tone that matches some parts of the story. The images may communicate symbolism and/or metaphors. An attempt was made to use contents to create an atmosphere/tone but it needed more work. Image choice is logical. Little or no attempt to use contents to create an appropriate atmosphere/tone.
5. Clarity of Voice Voice quality is clear and consistently audible throughout the presentation. Voice quality is clear and consistently audible throughout the majority (85-95%) of the presentation. Voice quality is clear and consistently audible through some (70-84%)of the presentation. Voice quality needs more attention.
6. Pacing of Narrative The pace (rhythm and voice punctuation) fits the story line and helps the audience really "get into" the story. Occasionally speaks too fast or too slowly for the story line. The pacing (rhythm and voice punctuation) is relatively engaging for the audience. Tries to use pacing (rhythm and voice punctuation), but it is often noticeable that the pacing does not fit the story line. Audience is not consistently engaged. No attempt to match the pace of the storytelling to the story line or the audience.
7. Meaningful Audio Soundtrack Music stirs a rich emotional response that matches the story line well. Images coordinated with the music. Music stirs a rich emotional response that somewhat matches the story line. Images mostly coordinated with the music. Music is ok, and not distracting, but it does not add much to the story. Not coordinated with images. Music is distracting, inappropriate, OR was not used.
8. Quality of Images Images create a distinct atmosphere or tone that matches different parts of the story. The images may communicate symbolism and/or metaphors. Images create an atmosphere or tone that matches some parts of the story. The images may communicate symbolism and/or metaphors. An attempt was made to use images to create an atmosphere/tone but it needed more work. Image choice is logical. Little or no attempt to use images to create an appropriate atmosphere/tone.
9. Economy of Story Detail The story is told with exactly the right amount of detail throughout. It does not seem too short nor does it seem too long The story composition is typically good, though it seems to drag somewhat OR need slightly more detail in one or two sections. The story seems to need more editing. It is noticeably too long or too short in more than one section. The story needs extensive editing. It is too long or too short to be interesting.
10. Grammar and Language Usage Grammar and usage were correct (for the dialect chosen) and contributed to clarity, style and character development. Grammar and usage were typically correct (for the dialect chosen) and errors did not detract from the story. Grammar and usage were typically correct but errors detracted from story Repeated errors in grammar and usage distracted greatly from the story.

 

Another example of a rubric that may be used or may function as a template to creating one's own rubric for evaluative purposes can be found by following this link. http://www.umass.edu/wmwp/DigitalStorytelling/Digital%20Storytelling%20Main%20Page.htm

Digitales provides a detailed overview for evaluating digital storytelling projects as well. Four different feedback models are used for evaluating through peer review as well as teachers, librarians, administrators, etc. These include Informal Reflecting and Formal Reflecting, the two used for peer review and Informal Evaluation and Formal Evaluation, the two used for teachers, librarians, administrators, etc. Also presented on Digitales is a rubric that may be used for grading or evaluating a project to determine if it was successful.
http://www.digitales.us/

This link provides an example of a grant proposal for a digital storytelling program. It provides an evaluation section that may be of use when writing a grant proposal on digital storytelling.
http://www.mcli.dist.maricopa.edu/mlx/slip.php?item=1610

Alphonse Trezza provides a complete guide to evaluating library programs in An Evaluation of State Supported Library Programs for the Disadvantaged in Illinois. This report is a useful tool when applying or reapplying for a state funded grant. Topics include purposes, methods and techniques, the framework of evaluation, descriptions of library programs, and equipment and materials used.

Evaluating Library Programs and Services: Tell It! By Douglas Zweizig et. al is a manual that was developed under the contract of the U.S. Department of Education to compile evaluation procedures for library programs. Several evaluation methods are discussed that include questionnaires, interviews, numbers-gathering, use of existing data, observation, and other self-reporting activities. A detailed account of choosing an evaluation method is also provided as well as establishing your library's vision, and alternative program design.

 

 

Page completed by Nicole Kaffel

 

Nadene Eisner
Nell Fleming
Nicole Kaffel
Janet Vogel (webmaster)

Graduate School of Library and Information Science
University of Illinois at Urbana-Champaign
LIS 506
November 2007