Successful Alumni Surveys Used in CENG

by Fred DePiero - Fall 2008

We have used alumni surveys as part of our ABET accreditation process for over 10 years. Recent improvements to our methodology have resulted in less administrative work to setup, run, and analyze results. Our response rates have also improved to 18% - 38%. Given the style of our alumni survey (voluntary responses) we are content with these response rates. The following points summarize our approach:

Top three most important items for a successful survey

• Short, short, short

Specific questions are easier to answer and yield more precise measures

• Ask specific questions related to alumni accomplishments, e.g. to measure.
  • Leadership: Received awards? Patents? Promotion? Published articles?
  • Life-long learning: Attend graduate school? Training courses? Attended conferences? Member of a professional society?
  • Integrated Learning: Value GE courses more/less as an alumnus?
• It may also be necessary to include less specific questions, e.g. using a likert scale to describe degree of accomplishment for each program goal. While this is nonspecific, this style question can ensure baseline coverage of all goals.

We use the professional version of SurveyMonkey.com to administer surveys.

• $20/month or $200/year
• Respondents are prompted via an email having a link to a web-based survey.
• Results include simple statistics as well as verbose responses.
• Provides anonymity
• Can include respondents outside CalPoly (unlike BlackBoard surveys).

Features of our email prompt to respondents

• Prompt is upbeat, emphasizes anonymity and value of respondent's input
• The sender appears to be the department chair (as seen by the recipient)
• Subject line does not include keywords likely to be filtered as spam
• Offer a gift (for immediate download) and a subsequent prize drawing

Integration with PolyLink - A social networking tool for CalPoly alumni

• (Note: The University controls access to alumni email addresses. PolyLink is the only available mechanism, replacing 'Output Requests' from Univ Advancement. Each college should have an established PolyLink contact person.)
• Email prompts are sent via the PolyLink system.
  • Lead time is brief (few days, better than the previous system)
  • Bounce-back statistics available
• Respondents are encouraged to update their PolyLink profiles. This creates a synergy with PolyLink (and helps refresh email addresses).
• Unfortunately because this must be done via PolyLink, re-prompts must be done en masse, not more selectively to only those who have not responded.

Offer a gift and prize drawing

• Gift is immediately available for download, after completing the survey. (Gift is a SuDoku puzzle game, donated by Fred DePiero. It would be nice if all colleges contributed downloadable gifts!)
• Prize drawing follows survey process. To enter the drawing, respondents give up their anonymity by providing contact info.

On a related point - Martin Shibata (Dir. Career Services) provides very good employer surveys. Martin tailors these surveys to meet the needs of a college. He also has extensive contact info and works to achieve good response rates.

As a final FYI, CENG also runs surveys in another, separate, senior-level assessment process. Some Programs choose to require student participation in a given course (yielding 100% response rates). Other Programs simply encourage students (yielding ~75%). For these surveys, we capture email addresses from the Data Warehouse and upload the addresses into SurveyMonkey. The Monkey then prompts each respondent and keeps track of those who have replied. Students can then be easily reminded via additional email prompts. Both the alumni and senior surveys are administered in a centralized fashion to reduce burden on individual programs.

Example Email Sent to Alumni - Written by Amy Hewes, CENG

(Apparent) Email Sender: Don White, Chair
Email Subject: CAL POLY ALUMNI

Dear Cal Poly Alum:

A prize for your thoughts . . .

As an alumnus/alumna of the Cal Poly Industrial Engineering program your assessment of our program is extremely valuable and a necessary to our continuous efforts to improve and excel.

To show our appreciation for providing us your feedback, we'd like to offer you a free gift for taking the survey posted at http://www.surveymonkey.com/s.aspx?sm=

The input we receive from this survey is an important part of our engineering accreditation process. Your comments and responses will be kept anonymous.

If you have already replied to this survey, we thank you. We are now expanding the survey pool for the widest possible response rate.

In thanks for completing the survey by January 1st, 2008, we'd like to offer you a FREE GIFT. In addition, you may register for a PRIZE DRAWING. The free gift is a computer-based SuDoku game that you may download immediately after completing the survey. To become eligible for the prize drawing, fill out the contact information at the end of the survey.

Please take the survey now at:
http://www.surveymonkey.com/s.aspx?sm=

On behalf of Cal Poly's Industrial Engineering program, thank you for your time and your continued support. Your participation will help secure our accreditation.

Sincerely,
Dr. Don White
IME Chair