Child pages
  • Teaching, Learning and Assessment Portfolio (TLAP) for Industrial Engineering
Skip to end of metadata
Go to start of metadata
User Scenario - Teaching, Learning and Assessment Portfolio for Industrial Engineering.

User

Role

Girish

tenure track faculty member in Engineering, part of a team of faculty awarded a grant to introduce portfolios as a teaching, learning and assessment tool for their program.

Fatik

1st year undeclared student

Courtney

4th year Industrial Engineering major

Anderson

Dean (College of Engineering)

Garrett

Assessment Coordinator

Mary Kate

Instructional Designer - has to design, develop and implement all the portfolio grant projects working collaboratively with each grantee. She supports Sakai on campus for all users.

Dillon

Journalism instructor, part of the assessment team to evaluate General Education outcomes for the University.

David

An external evaluator on the ABET Review Team

Background

The Industrial Engineering program was awarded a grant from the Office of Educational Assessment to design, implement, assess, and sustain a teaching, learning and assessment portfolio as a required part of their majors' undergraduate experience. The portfolio spans every year of a student's time in the major, is focused upon integrative and reflective learning, and demonstrates student competency in achieving the program's student learning goals and at least three general education goals. General education goals at the University are embedded in each program, they are not separate courses that a student takes while attending the university. The portfolio is used to assess student learning goals and student mastery of general education competencies.

The department is working on this grant as a whole and has about 6 faculty currently using the portfolio. These faculty have been working with the Assessment Coordinator and Instructional Designer mapping the current curriculum to ABET outcomes and the institutional general education outcomes. They have linked which assignments students must complete and submit into the portfolio. They have created several reflective questions encouraging students to think more about what they are learning and how it relates back to other courses they have taken, projects they have completed, why it is important to the profession they are entering, and other experiences both within and outside of the department. They use general feedback in the form of comments and rubrics to evaluate student work.

At least 3 of the General Education goals must be evaluated as part of the portfolio. For institutional assessment the Assessment Coordinator will randomly select several portfolios to review. Using a group of people trained in using the VALUE rubrics or similar rubrics, this group will assess whether the general education goals have been met. Access needs to be given for this set of random reviewers to evaluate the portfolios, they do not need to see and probably should not see the instructors evaluation of that submission. Most of the reviewers will be faculty selected to be an assessment fellow. Data will be aggregated from the portfolio to create reports needed for the department, program, and institution. As an Engineering discipline, many of these reports will be shared with ABET, which is the recognized accreditor for college and university programs in applied science, computing, engineering, and technology.

Users

Fatik is a freshman undeclared student, introduced to the concept of portfolios in his First Year Experience class. He has been asked to upload into his portfolio matrix a project he completed. Besides uploading the artifact he is supposed to answer some reflection questions. Fatik takes a long time to answer questions that really make him think. He was never very good at writing. He started completing a form with the questions once, and timed out, he can get a little distracted. Not seeing any warning, he lost a lot of his work. Now he copies down the questions, and first answers them in Word. He doesn't mind cutting and pasting his work into the form because it gives him more time, but depending on his browser he sometimes get's errors when he copies the text. When he is ready, he knows he has to submit his work. It would be nice if there was some sort of automatic save, then he wouldn't do all that cutting and pasting. Once his work is submitted he knows he can not do any more editing. He checks back later to see his instructors feedback. He knows feedback has been provided when the color in his matrix cell has changed - it would be nice if he got notified via an email or something.

Courtney is a senior Industrial Engineering major, and besides uploading artifacts into her portfolio matrix and answering reflective questions, which she has done now for four years, she has to complete a presentation portfolio for her capstone class. She likes the template provided in Sakai, but wishes there was a little more flexibility where she could upload her own designs and styles or replace the university banner with one she makes. As she plans her presentation, she carefully selects samples of her work from her matrix to include as evidence the knowledge and skills she has gained through her coursework and coop assignments.  She needs to be able to add some video and audio, not sure of the best way to embed this. She forgets to make sure her images and files she has attached are made public so people she shares her portfolio with can see them. After she creates her portfolio, she is asked to share her portfolio with some classmates for peer review. She also has to share it with her instructor. For feedback they add comments right on the page. She is worried that all this work she has done in her portfolio can't be taken with her when she graduates. She has been told that she can download it, but she is afraid that she may need to update it and is not sure if she can do that with the software she has on her computer. Courtney is currently trying to get a job. She is going to send her portfolio to a couple of places in the industry. She doesn't want the system to send the email, because the link is so long and she doesn't think it looks very professional. She made her portfolio public, shortened the URL and then sent the URL in an email. She likes that she can have multiple versions of her portfolio. She likes to customize them for different potential employers, or uses (she is thinking about going to grad school). She is so glad she was told to save her artifacts since Freshman year.  She wishes that she had the ability to save a snapshot of her portfolio each year so that she could easily review previous versions when asked to reflect on her intellectual development.

Girish is one of the instructors participating in the grant and using the portfolio. He thought it would be good to participate since he is going up for tenure soon. He has been working with Garrett and Mary Kate and other Engineering faculty mapping the current curriculum to ABET outcomes and the institutional general education outcomes. He added to his syllabus information about what artifacts (products of assignments) should be added to the portfolio and when. He knows and will explain to his students how these artifacts address the programs outcomes. Someone in the department made a video explaining the purpose of the portfolio, which he decided to use to help introduce the portfolio to his class on the first day. He also asked Mary Kate, to come to his classroom to show students how to upload artifacts, complete forms and answer the reflection questions. She made sure the students understood how to submit their work and where to go for help if needed. He asked his TA's to come to Mary Kate's training session so they understand the process and could help students if needed. He is a little uncomfortable with the technology but is trying to adjust. He doesn't want to be bothered with any technical problems though.

Mary Kate is the Instructional Designer for the grant projects and will be using the matrix tool to organize each department's program outcomes and which courses or experiences the faculty will be using to demonstrate competency for that outcome. They like the visual aspect of the matrix. She, or someone in her group has sat down with the grantees to help them map their program competencies with their courses and assignments. She has provided training or guidance on how to write reflective questions and give feedback. The engineering team of faculty are providing the reflective questions they want the students to complete, instructions on which artifacts to upload, etc. They have also provided rubrics to be used for evaluation. Mary Kate or one of the instructional technologists or associates are creating all forms that will gather the students and evaluators data. Currently this is one portfolio site which all the Industrial Engineering majors are using since it spans all their time in the program, from entering as freshmen till they graduate. Seems there is some slowness as the sight gets larger - the number of students entering the program is growing. We are now considering breaking them down by cohorts and creating multiple portfolio sites with the same matrix based on either the year they entered or when they are expected to graduate. There are concerns that there will be too many active sites which would make it confusing for the instructors. Mary Kate as instructional designer also has to help them report out on their findings. The deans and department chairs for these portfolio projects want to be able to generate report(s) on how students are meeting competencies, where in the program there may be problems, etc.

Anderson, as Dean has signed off on the grant that his faculty will implement the e-portfolio for the program. He wants to be able to generate report(s) on how students are meeting competencies, where in the program there may be problems, etc. He is looking forward to what data can be extracted and used to make improvements to the program. He also is responsible to report their findings back to the Office of Educational Assessment. He is fully expecting that the data that is gathered in these portfolios should help when he has to go through his ABET accreditation in a couple of years. He has concerns that most of his faculty, like himself, are not tech savvy in the use of these tools and will need lots of help and guidance throughout the process. The simpler the process can be for himself, his faculty and students the better in his mind.

Garrett, as Assessment Coordinator for the University, has met with the grantees to help them map their program competencies with their courses and assignments. He learned early on from some pilots that he needs to provide training and guidance on how to write reflective questions and give feedback. Students also need help with how to write a good reflection. So he is working with the Instructional Designer to provide this training. He has also selected a team of faculty to assess whether the general education competencies have been met in all the portfolio projects. They will be trained in the evaluation tools in Sakai, how matrices work, etc. as well as the VALUE rubrics or other rubrics they will be using.  They will also complete an interrater reliability/norming exercise to ensure they are rating student work consistently; Garret wishes the portfolio software would automatically detect and alert him to rater inconsistencies, but for now he relies on the training alone. He wants to make sure he has access to all the matrices in the various programs. Right now he is going in manually to randomly select student submissions, etc. His goal is to add a column to each matrix just for students to select final artifacts that demonstrate meeting the general education goals. All his faculty assessment fellows need access to the portfolio sites. It would be helpful if while evaluating student work, his evaluators could not see instructor feedback, but later they would need to when they explore the quality of feedback students are receiving. He has lots of reporting needs, within programs, but especially across programs for institutional assessment of the general education goals and external accreditors like ABET.

Dillon is a Journalism instructor and also an assessment fellow for the University. As a faculty member he started using portfolios in his program last year, and is very familiar with how the matrix and evaluation tools work. His concern is that reading through the student's submission and trying to complete the evaluation form (rubric or comments) can take a long time. He believes in providing useful feedback and wants to do a quality job. Since things are not auto-saved and he can get timed out, he has to be careful not to lose his work.

David was Courtney's supervisor while she cooped at his company her Junior year.  She writes weekly reflections about her coop experience and David has been asked to review and comment on them.  Since he's not on the staff at Courtney's institution, he needs a means of accessing the secure area of her portfolio and the ability to provide feedback on specific items, but not all items.

  • No labels

1 Comment

  1. Excellent work!!!  Think it might be best to make David an external accreditor-or someone who needs access to more secure areas of portfolio.  Then we can state explicitly that, even though David isn't associated with the institution, he needs to be able login to Sakai to view representative examples of student work as well as reports summarizing student growth and achievement in each of the ABET outcomes.