Artifact 2: Rethinking Group Work in Program Evaluation

Meta-Evaluation of a Case Study Evaluation

For my second artifact, I chose the group assignment from MDDE 617, Program Evaluation. This assignment was completed with a partner and involved doing a critical review of a fictitious evaluation report regarding a new online educational program. The program evaluation was provided by our instructor as a case study and was based on John Owen’s evaluation structure, which involves five categories, or Forms. The critical review was in the form of a paper that evaluated the case study based on which of Owen’s Forms it had used, descriptive elements of its developmental stage, and analysis of how well the Form was applied, and whether that application was sound given the program context. I chose this assignment for my portfolio because it was one of the areas in the program where my group work and communication skills were challenged and required some self-reflection and growth. My partner did not have a good grasp of the assignment and we didn't realize that until it was time to put our work together. In the end, the communication breakdown caused a lot more work for both of us. This issue could have been avoided by a more thorough collaboration process. It also made me consider the importance of clarity when creating instructional materials in my own practice.

Excerpt From Outline

Great things are never done by one person, they are done by a team. Steve Jobs

My Learning Process

When diving into this assignment, I read into it like any other group assignment. It was a 10-12 page paper focusing on program evaluation. We were provided with a fictitious program evaluation that had been completed to assess a distance education program and had to assess and critique the evaluation in terms of how well it was implemented. The assignment was done with a partner who I didn’t know and hadn’t worked with before. This person was a school administrator and seemed well educated and articulate based on our first text based interactions. I immediately assumed they wouldn’t have issues with the assignment and would be a ‘good’ partner whose work I wouldn’t have to overly correct or edit. This assumption was the root of my issue with this assignment and gave me reason to reflect on my own biases and downfalls when working as a team (1.1, 1.2).

My partner and I made our initial contact through Moodle and text, as those were the most accessible for both of us. We decided to do a brainstorm session prior to writing each of our sections of the paper. Our initial text exchanges were clear and concise, and we ended up using a Google Doc to collaborate and write out the main ideas for the paper (3.1, 3.2). We chose Google Docs because we both had experience with it, and we wouldn’t have to spend time learning a new program.  Google Docs is also free to use, which makes it appealing for student work (3.5). Once we had the main points down in our outline, we divvied up the sections and went to work. We each chose one of the more basic sections (introduction and conclusion) and then split the more detailed sections including which of Owen’s Forms was used, if the Form was applied well, and other details on the Form application. Once we agreed who would write each portion of the paper, we both individually went to work. I used Word to create my document, because it has more features than the online Google Docs version, but remains highly compatible to transfer between the full program with simple copy and paste. At the end of our deadline, we each shared our writing on a shared Google Doc, for the purpose of making it cohesive and editing each other’s work. Completing my portion of the assignment was straightforward, including navigating Owen’s Forms, researching related literature, and analyzing the case study and relevant program evaluation samples (5.2, 5.7).

When I look back at our outline, I can see where we ran into some trouble, as the outline didn’t really go deep into the details of the analysis. In fact, it was quite basic and general. What I didn’t realize at the time, was that my partner had missed the critique portion of the paper and was focusing on simply describing and explaining what the evaluation had covered.

Luckily, we had given ourselves enough time to edit each other’s work before the final due date. On the date we chose to compile our sections together, I read my partner’s portion and found it was significantly lacking in very important details and critical assessment (1.1, 1.5). While they did a decent job of describing the evaluation case study, they did not evaluate how well the evaluation had been executed (1.2). This was a vital part of the assignment. I panicked! Firstly, I get uncomfortable when critiquing the work of a peer, and this peer definitely needed critical feedback. This stems from some misguided feeling of guilt that pops up when the power dynamic from a teacher-student relationship is absent. As a teacher I am very comfortable with giving students feedback. They submit assignments to me expecting me to grade and critique their work, and in this realm, I know how to be professional and thoughtful. However, when giving feedback to a peer, or even a professor in my own academic journey, the rules seem somewhat fuzzier. It’s more important to avoid hurting feelings or ego when the person you’re giving advice to is your equal or superior (4.4, 4.5). These are relationships that may play into my professional life and academic opportunities in the future, so more is at stake when giving feedback! Secondly, I panicked because wasn’t sure if we would be able to get the assignment rewritten in time. We had spent most of the allotted time preparing the first draft, expecting minimal editing to be necessary. Lastly, I panicked simply because was angry with myself for not realizing there was an issue sooner and because I recognized that this mistake had created a load of extra work for both of us, in the eleventh hour (1.10).

My partner and I had an emergency meeting using Teams, which we chose because of its easy-to-use format and link to our school accounts (1.1, 1.6). I knew that we should talk face-to-face, or at least have a phone conversation to figure out what happened and what to do next. Writing long text messages back and forth would be too time consuming and has a higher risk of misconstruing subtleties in meaning when communicating important details (3.3, 3.6).

In our meeting we both got clarity on a few things. Mainly, I explained that the assignment was meant to be a meta-evaluation, or an evaluation of the evaluation itself. I also apologized because I had discovered my own pitfalls in being a supportive partner. By not clarifying the assignment together before we got started with our writing, we had let each other down. It was poor use of both of our time because they had spent all the time writing something that now had to be redone. We had a thoughtful and kind discussion about the breakdown and made a plan to move forward. I added the necessary details and removed redundancies while my partner got busy with editing (1.12, 1.13). In the end, we were still able to hand in the assignment on time and scored well, but it was a lesson in what cohesive group work really looks like, and what it doesn’t look like (4.9).

One other topic that came up in my conversation with my partner was the importance of clear assignment details and instructions for the sake of the learner. In my own practice it was a reminder to follow good instructional design guidelines, such as relevant academic language for the lessons, examples of good workmanship, and a clear rubric (2.8, 6.2). Good instructional design sets the stage for student success.

Ultimately, I know that I have strong writing skills, and can create purposeful and coherent documents. In this case I co-authored a meta-evaluation of a simulated educational program evaluation (2.3, 4.1). What I missed at the beginning of this assignment was collaborating and supporting my partner’s learning as well. Once I realized my mistake, I was able to connect with them using appropriate technology, and ensure they understood the nature of the assignment and together we were able to create steps to move forward and still create an effective and relevant paper through a process of constructive feedback and supportive collaboration (4.4, 4.5). We had to step back and re-write a large section of our paper to include everything required by the assignment and work more cohesively than we were expecting to, with the aid of communication technology such as video and voice conferencing. In my past group projects for this program, divide and conquer had been the main strategy, but this experience was an excellent lesson in what true collaboration and cooperation should look like. I came to understand the importance of ensuring all team members are on the same page and equipped to handle their duties within a project early on, and how regularly checking in throughout the process is vital to everyone’s success (4.6, 4.7).

Competencies

1. Problem Solving, Analysis, & Reflective Decision Making

1.1, 1.2, 1.5, 1.6, 1.12, 1.13

2. Instructional Design & Development for Equity

2.3, 2.8

3. Communication Technologies and Networking

3.1, 3.2, 3.3, 3.5, 3.6

4. Communication & Interpersonal Skills

4.1, 4.4, 4.5, 4.6, 4.7, 4.8, 4.9

5. Research

5.2, 5.7

6. Management, Organization and Leadership

6.2

Moving Forward

This group assignment was an important learning lesson for me because I always considered myself to have strong teamwork and leadership skills. Getting to the final stages of an assignment and realizing that someone misread the instructions felt like a failure for me. I came to appreciate that being a leader means I can’t assume any part of my team understands simply because they are well educated or articulate. Being a good team member and leader involves regular check ins, asking appropriate questions to ensure understanding, and most definitely requires the ability to effectively provide critical feedback to everyone, regardless of their status.

In addition, this experience made me consider the importance of clear assignment guidelines for students, because although this assignment was labeled as a critique, it could have been made clear that this was indeed a meta-evaluation, and that might have lessened the confusion for multiple students, not just my partner. Assignment instructions should be written clearly, concisely, and with the vocabulary that students are learning about in the course. This helps to create connections and encourage contextualization of learning materials. Finally, a detailed rubric indicating all the required components of an assignment gives students a place to check their work and self-assess prior to submitting for summative feedback. This experience grounded my belief and understanding of how instructional materials can be designed intentionally to help students succeed.

Comments

K Hansen
06 November 2023, 9:49 AM

Went through again with a finer tooth comb, I hope it reads a little smoother. Thank you!
Kim

BonnieM
09 November 2023, 3:25 PM

Good job, artifact two is complete.

Bonnie 

12 comments