Adobe Acrobat PDF version

Prepared by a subgroup of SUCCEED's Outcome Assessment Focus Team
JULY, 1997

Bill Shelnutt - UNC Charlotte
Kim Buch - UNC Charlotte
John Muffo, Virginia Tech.
Don Beasley, Clemson

The Capstone experience can be a rich source of assessment information for outcomes related to an institution's strategic goals and to the requirements of accrediting agencies such as ABET. Such information can be divided into three categories:

1) measurements of the success of the Capstone experience in meeting the goals of the course itself, such as student learning to apply techniques of synthesis to an open ended problem posed in realistic terms; or

2) measurement of the degree to which the student successfully applies specific tools, techniques, knowledge, and resources learned in earlier work to the capstone project;

3) measurement of the maturity level of the student in applying successfully skills a and capabilities cultivated throughout his or her curriculum, such as team skills, communication skills, personal time management, resourcefulness, self assessment, and personal development -- all used as an indicator of the level of capability brought forth upon graduation.

We have drafted five assessment instruments which can be used for gathering data to satisfy one or more of the three purposes above:

A) a Capstone Experience Student Survey for students completing a capstone experience;

B) a Capstone Experience Instructor/Coach Survey for faculty teaching a capstone course or serving as a coach of students or student teams in such a course;

C) a Capstone Project Industrial Sponsor Survey for industry partners serving as company representative/coach for the project;

D) a Peer Evaluation Instrument for Teams for students to provide feedback to their fellow team members on their level of contribution to the team;

E) a Team Project Reviewer Scoring Rubric for assessing the success of Capstone Project teams (or individuals) making final project presentations.

All student group projects are able to document the use of mathematical principles in implementing the capstone course project.

At least X% of individual group members are able to explain in an understandable manner to faculty and/or external experts how mathematical principles are used in the team problem- solving process.

At least X% of the students are able to explain in an understandable manner to faculty and/or external experts how mathematical principles are used in the project problem-solving process.

All groups are able to design and conduct at least X experiments in a satisfactory manner that provide information which improves the final projects in some way.

All groups are able to analyze and interpret in a satisfactory manner multivariate data that result from experiments conducted in support of the senior projects.

At least X% of the students are able to design and conduct Y experiments in a satisfactory manner that provide information which improves the final project in some way.

At least X% of the students are able to analyze and interpret in a satisfactory manner multivariate data that result from experiments conducted in support of the senior project.

All groups are able to design a system, component, or process which substantially improves engineering performance of a problem situation.

At least X% of the students are able to design a system, component, or process which substantially improves engineering performance of a problem situation.

At least X% of team members are rated by other team members as "excellent" or "good" in team leadership skills.

At least X% of team members are rated by other team members as "excellent" or "good" in understanding of problems faced by those with assignments different from their own.

At least X% of team members are rated by faculty members and/or external experts as "excellent" or "good" in being major contributors to team success.

All teams are able to identify problems so as to allow for formulation and solution.

All teams can formulate their problems in a logical manner so as to enable progress towards a solution.

All teams make substantial progress on solving the problem presented to them (or) making substantial improvements in the performance of the engineering process presented.

At least X% of the students are able to identify a problem in such a manner as to permit formulation and solution.

At least X% of the students can formulate the problem in a logical manner so as to enable progress towards a solution.

At least X% of the students (or all students) make substantial progress in solving the problem presented to them (or) in making substantial improvements in the performance of the engineering process presented.

At least X% of the students can present a cogent written synopsis of the ethical issues involved in applying their solutions to the problems presented.

At least X% of the students can make a cogent oral presentation regarding the ethical issues involved in applying their solutions to the problems presented.

At least X% of the students respond in an acceptable manner ethically when presented with other scenarios for solving the problem presented to them in the senior project.

At least X% of the students making the final oral presentation (assuming that each team member has responsibility for a different aspect of the final presentation) score an average of "good" or "excellent" on Y% of the following: presentation planning clarity logical sequencing ability to answer questions etc.

All groups produce written reports that are coherent and present the data in a logical, convincing fashion.

At least X% of the students making final oral presentations score "excellent" or "good" by a majority of faculty and/or external experts on the following:

• logic
• grammar
• understandability
• technical accuracy

All groups present at least X impacts that their solutions might have on each of the following:

• the environment
• their local communities
• global economic competition
• etc.

At least X% of the students can suggest Y impacts that their project solutions will have on

• the environment
• their local communities
• global economic competition
• etc.

All groups show the ability to use the following tools in completing their projects:

• the library - print media
• the library - electronic media
• the Internet
• other information sources

All groups demonstrate creativity in seeking out information sources.

At least X% of the students perform at an acceptable level in using Y% of the following resources in completing their projects: . the library - print media

• the library - electronic media
• the Internet
• other information sources

At least X% of the students demonstrate creativity in seeking out information sources.

All students are required to create tentative career development plans as part of the course. At least X% include various forms of continuing education in those plans.

(j) a knowledge of contemporary issues

In written and/or oral presentations, X% demonstrate a level of awareness of issues in the discipline that are rated "excellent" or "good" by the faculty and/or external reviewers.

(k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice

All groups demonstrate the following skills at least at an adequate level in their project

reports:

• computer skills
• technical skills (specify)
• time management skills
• other management skills (specify)
• other

At least X% of the students demonstrate skill levels at adequate or higher on Y% of the

following:

• computer skills
• technical skills (specify)
• time management skills
• other management skills (specify)
• other

Capstone Experience

Student Survey

Please respond to each of the following statements by writing a number (at left) from 1 to 5 corresponding to your degree of agreement with the statement using the scale below. (NOTE: YOUR RESPONSES WILL NOT AFFECT YOUR GRADE IN THIS COURSE AND ARE ANONYMOUS).

Asterisked (*) items indicate program-specific goal questions.)

1 2 3 4 5

totally disagree neither agree totally

disagree agree nor agree

disagree

Based on my experience in this course and in preceding courses:

_____ 1. I am confident in my abilities to apply my knowledge of mathematics to solve engineering problems.

_____ 2. I am confident in my abilities to apply my knowledge of science to solve engineering problems.

_____ 3. I am confident in my abilities to apply my knowledge of engineering to solve engineering problems.

_____ 4. I am confident in my abilities to design and conduct

statistically valid experiments and to interpret the data.

_____ 5. I am confident in my abilities to design full and fractional factorial experiments, conduct them, and to interpret the results.

_____ 6. I am confident in my abilities to function on multi-disciplinary teams.

_____ 7. I am constantly aware of team process and dynamics for good team performance.

_____ 8. I am able to reinforce and support ideas from team members.

_____ 9. I am able to negotiate agreements and handle conflict.

_____ 10. I am able to encourage open discussion of ideas.

_____ 11. I am able to work for and accept consensus or compromise.

_____ 12. I am able to plan work and set goals.

_____ 13. I am able to stay on task toward a timely completion of goals.

_____ 14. I am able to define and apply a systematic approach to tasks.

_____ 15. I am able to communicate effectively with persons from other disciplines.

_____ 16. I am confident of my ability to identify, formulate, and solve engineering problems.

_____ 17. I am able to define an engineering problem in succinct terms which express its essential elements and needed context.

_____ 18. I am able to use the tools of creative problem solving (such as brainstorming, withholding judgment, force-fitting of unconventional ideas, etc.) to produce a roster of creative solutions to a problem.

_____ 19. I am able to use organized methods of comparing alternative solutions to problems (such as the Pugh method) to evaluate and evolve progressively better solutions before final selection.

_____ 20. I am able to "sell" my ideas or design solutions by effective technical presentations.

_____ 21. I am able to "sell" my ideas or design solutions by effective written reports.

_____ 22. I am confident in my abilities to be aware of the issues I will likely face in my career and to make ethical decisions and to behave responsibly in all aspects of my occupation.

_____ 23. I am confident in my understanding of the impact of engineering solutions in a global and societal context.

_____ 24. I have begun a plan for remaining current in my field.

_____ *25. I have acquired a knowledge of contemporary issues relevant to my field, including energy conservation and the environment.

_____ *26. I have demonstrated an ability to use effectively tools of 3D modeling including Pro-engineer software.

_____ *27. I have demonstrated an ability to use effectively tools of finite element modeling including IDEAS software.

_____ *28. I have demonstrated an ability to use effectively tools of project planning including Microsoft Project.

_____ *29. I have demonstrated an ability to use effectively tools of business process re-engineering including Process Mapping using Benchmarker software.

Capstone Course or Experience

Faculty Course Instructor or Coach Survey

Course No. ________ Section _______ Number of students _______

Course Title ____________________________________________________________

Number of students working in teams _______ team sizes __, __, __, __

Number of students working individually _____

Number of industrially defined projects ____

Other faculty working on this course section with you:

____________________________________________

Please respond to each of the following statements by writing a number (at left) from 1 to 5 corresponding to your degree of agreement with the statement using the scale below.

(NOTE: your responses will be used for outcomes assessment of this program, which includes a number of courses and experiences leading to this capstone experience. It is expressly not for assessment of your performance as an instructor in this course.)

Asterisked (*) items indicate program-specific goal questions.)

1 2 3 4 5

totally disagree neither agree totally disagree (or untrue agree nor (or true agree

(or untrue of most disagree of most (or true

of virtually students) (or true of students) of virtually

all students) about half every student)

of students)

Based on my observation of student work in this course,

_____ 1. These students were able to apply their knowledge of mathematics to solve engineering problems.

_____ 2. These students were able to apply their knowledge of science to solve engineering problems.

_____ 3. These students were able to apply their knowledge of engineering to solve engineering problems.

_____ 4. These students were able to design and conduct statistically valid experiments and to interpret the data.

_____ 5. These students were able to design full and fractional factorial experiments, conduct them, and to interpret the results.

_____ 6. These students were able to function on multi-disciplinary teams.

_____ 7. These students realized the importance of being constantly aware of team process and dynamics for good team performance.

_____ 8. These students were able to reinforce and support ideas from various team members.

_____ 9. These students were able to negotiate agreements and handle conflict constructively.

_____ 10. These students practiced encouraging open discussion of ideas.

_____ 11. These students worked for and accepted consensus or compromise.

_____ 12. These students were able to plan work and set goals effectively.

_____ 13. These students were able to stay on task toward a timely completion of goals.

_____ 14. These students were able to define and apply a systematic approach to tasks.

_____ 15. These students were able to communicate effectively with persons from other disciplines.

_____ 16. These students were able to identify, formulate, and solve engineering problems.

_____ 17. These students were able to define an engineering problem in succinct terms which expressed its essential elements and needed context.

_____ 18. These students were able to use the tools of creative problem solving (such as brainstorming, withholding judgment, force-fitting of unconventional ideas, etc.) to produce a roster of creative solutions to a problem.

_____ 19. These students demonstrated abilities to use organized methods of comparing alternative solutions to problems (such as the Pugh method) to evaluate and evolve progressively better solutions before final selection.

_____ 20. These students were able to "sell" their ideas or design solutions by effective technical presentations.

_____ 21. These students were able to "sell" their ideas or design solutions by effective written reports.

_____ 22. These students exhibited awareness of the issues which they will likely face in their careers and seemed able and ready to make ethical decisions and to behave responsibly in all aspects of their occupations.

_____ 23. These students exhibited a good understanding of the impact of engineering solutions in a global and societal context.

_____ 24. These students showed commitment to life-long learning in their profession, and have begun plans for remaining current in their fields.

_____ *25. These students demonstrated a knowledge of contemporary issues relevant to their fields, including energy conservation and the environment.

_____ *26. These students demonstrated an ability to use effectively tools of 3D modeling including Pro-engineer software.

_____ *27. These students demonstrated an ability to use effectively tools of finite element modeling including IDEAS software.

_____ *28. These students demonstrated an ability to use effectively tools of project planning including Microsoft Project.

_____ *29. These students demonstrated an ability to use effectively tools of business process re-engineering including Process Mapping using Benchmarker software.

Capstone Course or Experience

Industrial Coach or Project Contact/Sponsor Survey

(To be filled out for each project sponsored by industrial partner by the company contact person responsible for the project.)

CONFIDENTIAL - THIS SURVEY WILL BE USED TO ASSESS THE OUTCOMES OF VARIOUS COURSES AND EXPERIENCES UP TO AND INCLUDING THIS CAPSTONE EXPERIENCE. IT WILL NOT AFFECT THE GRADE OF STUDENTS IN THIS COURSE NOR THE EVALUATION OF THE INSTRUCTOR'S PERFORMANCE IN THE COURSE.

Dear (industrial sponsor or project coach):

We appreciate the time you contribute to our engineering program by filling out this survey. Please be assured that your responses will help us in assessing our degree of success in meeting our strategic goals and in meeting the requirements of ABET criteria. We want your honest and thoughtful feedback which only you can give on the project in which you were involved.

Project Title ______________________________________________________________

________________________________________________________________________

Date project completed

Person completing survey ___________________________________________________

Title

________________________________________________________________________

Approximately how often did you meet with the student team or with its representatives?

Approximately how often did you have other forms of communication (identify) with the students? ________________________________________________________

________________________________________________________________________

How well defined was the project to the students?

________________________________________________________________________

________________________________________________________________________

To what degree do you feel that the goals of the project were achieved?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Please identify any stumbling blocks to successful project completion which were encountered by the student team, but not necessarily of their own responsibility:

________________________________________________________________________

_______________________________________________________________________

Did you review the final written report? ______ If so, please comment on the final written report quality:

________________________________________________________________________

________________________________________________________________________

Did you attend the final project presentation? _____ If so, please comment on the quality of the final oral presentation:

________________________________________________________________________

________________________________________________________________________

Please respond to each of the following statements by writing a number (at left) from 1 to 5 corresponding to your degree of agreement with the statement using the scale below.

Asterisked (*) items indicate program-specific

goal questions.)

1 2 3 4 5

totally disagree neither agree totally

disagree (or untrue agree nor (or true agree

(or untrue of most disagree of most (or true

of virtually students) (or true of students) of virtually

all students) about half every student)

of students)

(Please enter NA in the blank if you have no basis for any observation or opinion for that statement)

Based on my observation of student work on this project:

_____ 1. These students were able to apply their knowledge of mathematics to solve engineering problems.

_____ 2. These students were able to apply their knowledge of science to solve engineering problems.

_____ 3. These students were able to apply their knowledge of engineering to solve engineering problems.

_____ 4. These students were able to design and conduct statistically valid experiments and to interpret the data.

_____ 5. These students were able to design full and fractional factorial experiments, conduct them, and to interpret the results.

_____ 6. These students were able to function on multi-disciplinary teams.

_____ 7. These students realized the importance of being constantly aware of team process and dynamics for good team performance.

_____ 8. These students were able to reinforce and support ideas from various team members.

_____ 9. These students were able to negotiate agreements and handle conflict constructively.

_____ 10. These students practiced encouraging open discussion of ideas.

_____ 11. These students worked for and accepted consensus or compromise.

_____ 12. These students were able to plan work and set goals effectively.

_____ 13. These students were able to stay on task toward a timely completion of goals.

_____ 14. These students were able to define and apply a systematic approach to tasks.

_____ 15. These students were able to communicate effectively with persons from other disciplines.

_____ 16. These students were able to identify, formulate, and solve engineering problems.

_____ 17. These students were able to define an engineering problem in succinct terms which expressed its essential elements and needed context.

_____ 18. These students were able to use the tools of creative problem solving (such as brainstorming, withholding judgment, force-fitting of unconventional ideas, etc.) to produce a roster of creative solutions to a problem.

_____ 19. These students demonstrated abilities to use organized methods of comparing alternative solutions to problems (such as the Pugh method) to evaluate and evolve progressively better solutions before final selection.

_____ 20. These students were able to "sell" their ideas or design solutions by effective technical presentations.

_____ 21. These students were able to "sell" their ideas or design solutions by effective written reports.

_____ 22. These students exhibited awareness of the issues which they will likely face in their careers and seemed able and ready to make ethical decisions and to behave responsibly in all aspects of their occupations.

_____ 23. These students exhibited a good understanding of the impact of engineering solutions in a global and societal context.

_____ 24. These students showed commitment to life-long learning in their profession, and they have begun plans for remaining current in their fields.

_____ *25. These students demonstrated a knowledge of contemporary issues relevant to their fields, including energy conservation and the environment.

_____ *26. These students demonstrated an ability to use effectively tools of 3D modeling including Pro-engineer software.

_____ *27. These students demonstrated an ability to use effectively tools of finite element modeling including IDEAS software.

_____ *28. These students demonstrated an ability to use effectively tools of project planning including Microsoft Project.

_____ *29. These students demonstrated an ability to use effectively tools of business process re-engineering including Process Mapping using Benchmarker software.

Capstone Experience

CONFIDENTIAL Student Peer Survey

(NOTE: this survey may be administered on the WWW if appropriate security measures are taken, providing advantages of easy and timely response back to the team members after perusal by a faculty member to disallow flaming of any team member).

To the student:

Please take the time to provide feedback to the members of your project team in this course by filling out this survey form. You may be asked to complete this survey several times for each team member during the course term. Your ratings and comments will be relayed anonymously to each team member along with those of other team members for the purpose of improvement and program assessment.

This particular survey will be used only for the purpose of getting anonymous information back to your team members for their personal development plan and for overall outcome assessment of the success of the capstone experience in this engineering program. It will not be used as a part of any student's grade unless explicitly so stated in place of this paragraph. Also, any feedback you receive on your performance will be treated confidentially as well.

Please respond truthfully and constructively. If you rate a student particularly high or particularly low, then you should feel obligated to mention in the comments section some specific example of behavior which illustrates the basis for your opinion.

-------------------------------------------------------------------------------

Anonymous Peer feedback on Team Member's performance on team project.

Team member completing this rating (not reported to person being rated):

Team name or team identifier:

Team member being rated:

Please rate this team member on each of the following statements by writing a

number (at left) from 1 to 5 corresponding to your degree of agreement with the

statement for that team member using the scale below.

1 2 3 4 5

totally disagree neither agree totally

disagree agree nor agree

(Please enter NA in the blank if you have no basis for any observation or opinion for that statement)

Based on my observation of this team member's work on this project:

_____ 1. This student was able to apply knowledge of mathematics to solve engineering problems.

_____ 2. This student was able to apply knowledge of science to solve engineering problems.

_____ 3. This student was able to apply knowledge of engineering to solve engineering problems.

_____ 4. This students was able to design and conduct statistically valid experiments and to interpret the data.

_____ 5. This student was able to design full and fractional factorial experiments, conduct them, and to interpret the results.

_____ 6. This student was able to function effectively on our multi-disciplinary team.

_____ 7. This student realized the importance of being constantly aware of team process and dynamics for good team performance.

_____ 8. This student was able to reinforce and support ideas from various team members.

_____ 9. This student was able to negotiate agreements and handle conflict constructively.

_____ 10. This student practiced encouraging open discussion of ideas.

_____ 11. This student worked for and accepted consensus or compromise.

_____ 12. This student was able to plan work and set goals effectively.

_____ 13. This student was able to stay on task toward a timely completion of goals.

_____ 14. This student was able to define and apply a systematic approach to tasks.

_____ 15. This student was able to communicate effectively with persons from other disciplines.

_____ 16. This student was able to identify, formulate, and solve engineering problems.

_____ 17. This student was able to define an engineering problem in succinct terms which expressed its essential elements and needed context.

_____ 18. This student was able to use the tools of creative problem solving (such as brainstorming, withholding judgment, force-fitting of unconventional ideas, etc.) to produce a roster of creative solutions to a problem.

_____ 19. This student demonstrated abilities to use organized methods of comparing alternative solutions to problems (such as the Pugh method) to evaluate and evolve progressively better solutions before final selection.

_____ 20. This student was able to "sell" ideas or design solutions by effective technical presentations.

_____ 21. This student was able to "sell" ideas or design solutions by effective written reports.

_____ 22. This student exhibited awareness of the issues which he or she will likely face in a career and seemed able and ready to make ethical decisions and to behave responsibly in all aspects of the occupation.

_____ 23. This student exhibited a good understanding of the impact of engineering solutions in a global and societal context.

_____ 24. This student showed commitment to life-long learning in his or her profession, and he or she has begun plans for remaining current in the field.

_____ *25. This student demonstrated a knowledge of contemporary issues relevant to his or her fields, including energy conservation and the environment.

_____ *26. This student demonstrated an ability to use effectively tools of 3D modeling including Pro-engineer software.

_____ *27. This student demonstrated an ability to use effectively tools of finite element modeling including IDEAS software.

_____ *28. This student demonstrated an ability to use effectively tools of project planning including Microsoft Project.

_____ *29. This student demonstrated an ability to use effectively tools of business process re-engineering including Process Mapping using Benchmarker software.

Project Scoring Rubric for Final Project Presentations

This instrument can be used by teams of faculty and industrial observers present for final presentations of the Capstone project. It is deliberately concise in order to be scored in two minutes or so, since that is a reasonable amount of time in which to ask scores of such presentations to do their evaluations. Typically two or three teams may present per hour, and the results of the ratings compiled for assessment of the course progress and of certain program goals.

Date ________________ Scorer Name: _______________________________________

Scorer Title

Scorer Company or Affiliation

Team Name or identifier:

Project Title _____________________________________________________________

Please respond to each of the following statements by writing a number (at left) from 1 to 5 corresponding to your degree of agreement with the statement using the scale below.

Asterisked (*) items indicate program-specific goal questions.)

1 2 3 4 5

totally disagree neither agree totally

disagree (or untrue agree nor (or true agree

(or untrue of most disagree of most (or true

of virtually students) (or true of students) of virtually

all students) about half every student)

of students)

(Please enter NA in the blank if you have no basis for any observation or opinion for that statement)

Based on my observation of the team's final oral presentation:

_____ 1. These students were able to apply their knowledge of mathematics, science, and engineering to solve engineering problems.

_____ 6. These students were able to function effectively on multi-disciplinary teams.

_____ 12. These students were able to plan work and set and meet goals effectively.

_____ 16. These students were able to identify, formulate, and solve engineering problems.

_____ 20. These students were able to "sell" their ideas or design solutions by effective technical presentations.

_____ 23. These students exhibited a good understanding of the impact of engineering solutions in a global and societal context.

____ *29. These students demonstrated an ability to use effectively tools of business process re-engineering including Process Mapping using Benchmarker software.