Evaluation is a crucial component in interdisciplinary fields, particularly in distance education. Throughout the semester, I gained valuable insights into program evaluation theories, methodologies, approaches, and applications. One of the key takeaways from this course was the critical importance of using flexible evaluation frameworks tailored to dynamic environments and capable of rapidly adapting to key audience needs. 

Throughout this course, we engaged in various assignments that allowed us to practice applying different evaluation methods and forms. By studying numerous evaluation models and techniques, I've gained a deeper understanding of how to apply and combine them effectively based on specific case requirements. This experience has undoubtedly reinforced the importance of extensive practice in evaluating educational programs.

For Assignment 3, I utilized Kirkpatrick's four-level model and participant-focused approaches. In the group assignment, we employed an interactive form with a specific emphasis on Responsive evaluation. This exercise proved invaluable for enhancing my understanding of program evaluation methodologies.

The final assignment of this course required creating an evaluation plan based on the provided case study, Ithika University's Instructional Design Diploma (IDD) Program.  This is a very comprehensive exercise as we had to include the evaluand, the purpose, the client and primary audiences, the evaluation resources, the focus, the model and/or form, the key questions, the preliminary standards and criteria, data management, dissemination plan, codes of behavior, and timeline. 

After careful consideration specifically the state of the program and the client's evaluation purposes, I selected the Context, Input, Process, and Product (CIPP) model approach for the case study of Ithika University's Instructional Design Diploma Program. This decision was based on the client's request to get a holistic view of their program, extending beyond mere impact assessment. 

The evaluation focused on four key areas:

1. Program Viability: This will involve assessing the overall feasibility and sustainability of the IDD program, taking into account its long-term prospects and potential for continued operation.
2. Academic Quality: The committee will evaluate the rigor of the curriculum, the effectiveness of the technologies and instructional methods used, and their alignment with industry standards and best practices. This evaluation aims to ensure that the program maintains high academic standards.
3. Learner Relevance: The assessment will determine whether the program meets the needs of its target audience, ensuring it remains relevant and valuable to learners. This evaluation will help maintain the program's appeal and effectiveness in addressing learner needs.
4. Cost Effectiveness: The committee will analyze the program's resource utilization, efficiency, and overall value in terms of return on investment. This evaluation will assist in optimizing resource allocation and ensuring the program's financial sustainability.

The IDD evaluation framework consists of the following four dimensions. The dimensions of this framework work together to create a comprehensive assessment, providing a holistic view of the IDD program.  

• Context – In the IDD program, context refers to the foundational elements that shape its design and delivery. It encompasses the results of needs assessments, opportunities that influence program goals, and various factors that impact the educational experience such as learner demographics, organizational culture, and external environmental influences that affect the educational experience.
• Input - Input involves the necessary resources, infrastructure, curriculum, and content needed to implement teaching-learning processes effectively. This includes materials, facilities, budget, technology, and other assets required for successful program delivery.
• Process - The process includes all activities necessary for implementing different teaching methods, evaluations, and formative assessments. It covers how instructors deliver content, facilitate discussions, and assess student progress throughout the program.
• Product - Product evaluation focuses on assessing skills, values, attitudes, and results achieved by learners. It examines whether the program has met its intended outcomes and contributed to learners' personal and professional development.

After that, I created a comprehensive list of evaluation questions for each dimension of the CIPP model.

After creating these questions, I created the preliminary standards and criteria. These criteria serve as indicators of compliance and identify areas needing improvement. Standards and criteria may evolve throughout evaluation as new issues emerge. 

I also drafted the questions for the assessment instruments by following the guidelines in the course guide. 

Reflection 

Throughout the Spring semester, engaging with the MDDE 620 course's instructional materials, assignments, live sessions, online forums, and peer reviews has profoundly deepened my grasp of both the theoretical groundwork and the modern-day challenges inherent in Education 4.0 technologies, alongside the shifting needs of 21st-century learners. Additionally, the course offered me an invaluable opportunity to explore change management tactics and enhance my abilities in advocating my selected EdTech application within an organization through a compelling video presentation.

My semester journey through the MDDE 620 course not only strengthened my skills and confidence in teaching within the digital realm of 21st-century education but also highlighted the importance of continuous learning and flexibility amidst the constant changes in educational settings.

Reflecting on my participation in the MDDE 620 course, I actively contributed to both online forum discussions and live sessions. My involvement in the online forum discussions surpassed the typical expectations; I found these discussions helpful in enhancing my comprehension of the subject matter.

Unlike other courses where each unit might be broken down into sub-components like learning outcomes, key terms for each unit, and new information or knowledge, the MDDE 620 course structured its content differently. This unique approach meant that the online forum discussions and live sessions became crucial for me in grasping the core concepts of each unit in addition to the unit’s reading materials.

The reading materials assigned for each unit significantly enhanced my readiness for both the forum discussions and assignment work. Notably, Fahy’s publication, "Technology in Education and Training" (2015)[1], served as an invaluable reference throughout the course, akin to a personal dictionary.

The live sessions also stood out as the most enjoyable aspect of the course. I participated in all the live sessions and one optional office hour. Beyond participating in virtual meetings and interacting with Professor Mae and my peers, I was particularly impressed by Professor Mae's approach to integrating instructional materials. She utilized Mentimeter beforehand to gather our insights or comments on specific subjects, and her presentations often featured advanced technological tools. Additionally, these sessions not only deepened our comprehension of each assignment but also ensured we were thoroughly prepared.

I found immense satisfaction in completing the A2 video assignment. This task demanded a blend of creativity and technical expertise, pushing me to devise an engaging narrative and design. The process was both challenging and rewarding, as it allowed me to synthesize the knowledge acquired throughout the course with practical application of technology and design skills. Additionally, one of the most enriching aspects of this assignment was the chance to observe and learn from the exceptional work of my peers, despite being asked to review only three videos. My curiosity led me to view every video, each showcasing remarkable talent and innovation.

Summary

In conclusion, I would rate my overall engagement in this course as an outstanding 10 out of 10 across all dimensions of class participation. Through my active participation in this course, I have gained the skills needed to incorporate technology into my pedagogical strategies and continuously update my knowledge and skills to keep my teaching relevant and effective in the digital age.

Thank you, Prof. Mae, for all your dedication, kindness, and guidance throughout this journey. Your exceptional mentorship has not only enriched my academic experience but also inspired me to pursue excellence in education.  I hope to see you again in another course.

[1] Fahy, P. J. (2015). In Earlier study guide for MDDE 620. Athabasca University

The following video explains why you should use ScreenPal to provide meaningful feedback for online learners.

https://youtu.be/byi1oAHmN_s?si=WSf8QnocCJkfHytD

Feedback plays a crucial role in enhancing the quality of learning experiences. It serves as a potent tool for strengthening the relationship between instructors and learners, creating an environment that fosters growth and continuous improvements.  However, some research showed that written feedback in an online learning environment can be even more challenging as learners can overlook instructors' messages and fail to make meaningful revisions to assignments. 

Crook et al. (2012, as cited in Joseph-Edwards & Edwards, 2022) and Kilickaya (2016) highlighted that the integration of technology into feedback processes promises to deliver feedback more effectively, engage learners more deeply, and ensure prompt delivery, ultimately leading to superior learning outcomes. 

Therefore, I wrote a paper as part of the MDDE620 assignment to analyze the use of ScreenPal, formerly Screencast-O-Matic, for delivering video feedback. For a detailed overview, kindly refer to the attached document or visit this link to read my full paper. 

Hello Everyone,

I would like to share my learning journal for my journey in MDDE619, Trends and Issues in Instructional Design for Fall 2023. Please check it out this link: https://sites.google.com/view/galuh-elearningjournal/home 

“AI will create a learning revolution, and it’s only just getting started. It has the potential to help teachers and students. While we won’t achieve perfection in its implementation, suppressing AI would be misguided. Instead, we must focus on empowering students and teachers to utilize AI ethically and responsibly, providing them with the necessary support to navigate this evolving landscape.”

(Alexander McNeece, Director of Instructional Services Garden City Public Schools)

The quote from McNeece above describes where we, as an educator, should stand on AI. Artificial intelligence (AI) is here to stay. If we want our learners to be workforce-ready when they graduate, we must teach the ethical use of AI across the curriculum rather than focus on the potential misuse of AI e.g., plagiarism. Plagiarism or cheating is not the reason to not teach or use AI in the curriculum. Academic integrity is about developing ethical decision-making skills that learners can use daily in and beyond the classroom. Academic integrity is a foundation for professional practice (Eaton, 2023).

Based on my experience using AI tools throughout this semester, I believe AI can be used as our digital assistant to accelerate our work as instructors to become increasingly productive e.g., create lesson plans, quizzes, modules, or discover ideas/resources/materials for our lectures and to improve our writing. For learners, AI can assist learners in collecting and analyzing data/information; be helpful assistance when the instructor is not available; or help learners individualize their learning opportunities e.g., translating materials to their language or revising the materials to suit the learners’ level or skill.

However, we have to make sure to check the accuracy of the AI outputs because AI sometimes produces outputs seemingly intelligent responses that are nonsensical (AI hallucination or confabulation). AI makes a guess based on training, data used, and learned associations resulting in unpredictable outputs and can be biased. ChatGPT is one of the AI tools that I believe will continue to grow in education and as of today, this tool is trained to provide you outputs for information and resources up to January 2022. You can check the accuracy by checking with the subject matter expert, other AI tools, or Google Scholar.

Privacy is another concern in AI. Without us knowing it, technologies that we use e.g., smartphones and the internet have made AI easier to collect personal data. I often feel that am being monitored by my smartphone, tablet, and laptop e.g., I suddenly receive offer coupons based on my past purchases or internet displayed suggested sites based on my search histories. 

If we want our learners to be workforce-ready when they graduate, we must teach the ethical use of AI across the curriculum rather than focus on the potential misuse of AI e.g., plagiarism. Plagiarism or cheating is not the reason to not teach or use AI in the curriculum. Academic integrity is about developing ethical decision-making skills that learners can use daily in and beyond the classroom. Academic integrity is a foundation for professional practice (Eaton, 2023).

To build the capacity for the proper use of AI, we need to start educating ourselves first to understand more about AI including its benefits, limitations, and challenges, and know how to mitigate the risks of AI. In the next step, we can equip our learners on how to use AI responsibly and ethically by creating an AI ethical guideline and engaging learners with interactive routine discussions about their experience in using AI to foster their ability to recognize AI ethical risks and concerns.

When I started this course, I had very limited knowledge about AI or emerging technology. I do not think I could write my reflection about AI above. Now, I know a number of emerging technologies e.g., AI, gamification, and immersive VR/AR. The hands-on experience or learning exercise and blog critique specifically the multimedia are my favorite parts of this course. These tasks allowed me to learn and practice how to use them, and to foster my critical pedagogical thinking on how to apply those tools to my teaching and learning practices.

The synchronous sessions are my favorite as well. Dr. Levina provided lectures about the materials of the week. The information and discussions with my classmates about the topics gave me a better perspective and understanding than just reading the course materials.

For future development, it would be great if the assignments for this course could be focused on the learning exercises and blog critique multimedia (hands-on experience).  The discussion forums also focus on those hands-on experience activities. I think it would be great to learn from other experiences (as the Unit 4 Primer Discussion).

In summary, it has been a great learning journey this semester. Thank you, Dr. Levina, for all your dedication, patience, and support throughout this course.  You are an amazing professor and I hope I will see you again in another course. 

References:

Eaton, S. E. (2023). Using Generative AI Ethically: Teaching, Learning, and Assessing in a Postplagiarism Era. [Conference Slides]

Hwang, G. J., & Chen, N. S. (2023). Exploring the Potential of Generative Artificial Intelligence in Education: Applications, Challenges, and Future Research Directions. Educational Technology & Society, 26(2).