MARS ATTACKS!
SUMMATIVE EVALUATION
Evaluator
Facilitator
DATE
Target
Technique
Instruments
Expert Evaluation Phase
Instructional Design Expert
Rita Karl
January
2000
Interface
Congruence Analysis
Checklist
Subject Matter Expert (Mars Scientist)
Rita Karl
January 2000
Instructional Materials
Content Analysis
Interview
Questionnaire
Human Factors Expert
NASA Education/Outreach Specialist
January
2000
Instructional Materials
Design Analysis
Interview
Questionnaire
Distance Education Expert
Rita Karl
January 2000
Instructional Materials
Feasibility
Checklist
Field Trial Phase
Staff and Administrators: 
CCISD & NASA Education Staff and Administrators
Rita Karl
February 2000
Student Performance 
Feasibility 
Cost
Managerial Analysis
Observations
Interviews Questionnaires
Sample of CCISD Students: 
Secondary
NASA Education/Outreach Specialist
February
2000
Student Reactions
Outcome Analysis
Surveys 
Observations
Interviews 
Sample of CCISD Students: 
Secondary
NASA Education/Outreach Specialist
February
2000
Student Learning
Outcome Analysis
Portfolio Assessment Rubrics 
Performance Checklists 
Sample of CCISD Students, NASA and CCISD Staff, & NASA and CCISD Administrators
Rita Karl
February
2000
Student Behaviors
Outcome Analysis
Surveys 
Observations
Interviews 
Sample of CCISD Students, NASA and CCISD Staff, & NASA and CCISD Administrators
Rita Karl
February
2000
Organizational Impact
Outcome Analysis
Surveys 
Interviews 


Summative Evaluation Instrument


 


Evaluator: Human Factors Expert

Facilitator: NASA Education/Outreach Specialist

Date: January 5, 2000

Target: Instructional Materials

Technique: Design Analysis

Expert will meet with facilitator to address the evaluation after completing a review of the course materials and observing students completing a variety of activities.

Instrument: Interview/Questionnaire (ARCS Mode)

Interview and Questions:

Rating Scale

Not at all  - 0
In a few cases - 1
To some extent - 2
In many instances - 3
To a large degree  - 4

Attitude
 
Are the instructional strategies ones which can excite student interest in the exploration
of Mars?

Rating: _____
Comments:(list)__________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

Are the strategies employed consistently motivating throughout the course of instruction?
 
Rating: _____
Comments:(list)__________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

Are strategies challenging?
 
Rating: _____
Comments:: ___________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
 

Relevance

Is the level of the instruction applicable for secondary students?
 
Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Are units too hard?
 
Rating: _____
Comments: (Which ones and why?)__________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Are units too easy?
Rating: _____
Comments: (Which ones and why?)___________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Are the assignments realistic?  Challenging?
 
Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Will students considering science and engineering careers find the information
relevant to their future careers?

Rating: _____
Comments:____________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________

Confidence

Are the activities structured so that students will generally meet with success?
 
Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Are the goals and objectives clearly stated?
 
Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Are the assignments and performance criteria reasonable and fair?

Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Is the sequence of instructional events logical and progress in difficulty?

Rating: _____                          
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Does one unit lead to the next, building upon the last?
 
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Is the vocabulary appropriate for secondary students?

Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Satisfaction

Are the rewards within this course of instruction (one on one correspondence with NASA, work
published on the website, recognition of excellence by NASA) satisfactory and appealing to students
in terms of personal feelings of accomplishment?

Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Will completing one unit (or all units) in this course of study help students gain more of an interest in
the space program?

Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________

Will completing this course influence students' attitudes towards careers in math, science, and
technology?

Rating: _____
Comments:____________________________________________________________________
_____________________________________________________________________________
_____________________________________________________________________________
 

HOME