Evaluation is a crucial component in interdisciplinary fields, particularly in distance education. Throughout the semester, I gained valuable insights into program evaluation theories, methodologies, approaches, and applications. One of the key takeaways from this course was the critical importance of using flexible evaluation frameworks tailored to dynamic environments and capable of rapidly adapting to key audience needs. 

Throughout this course, we engaged in various assignments that allowed us to practice applying different evaluation methods and forms. By studying numerous evaluation models and techniques, I've gained a deeper understanding of how to apply and combine them effectively based on specific case requirements. This experience has undoubtedly reinforced the importance of extensive practice in evaluating educational programs.

For Assignment 3, I utilized Kirkpatrick's four-level model and participant-focused approaches. In the group assignment, we employed an interactive form with a specific emphasis on Responsive evaluation. This exercise proved invaluable for enhancing my understanding of program evaluation methodologies.

The final assignment of this course required creating an evaluation plan based on the provided case study, Ithika University's Instructional Design Diploma (IDD) Program.  This is a very comprehensive exercise as we had to include the evaluand, the purpose, the client and primary audiences, the evaluation resources, the focus, the model and/or form, the key questions, the preliminary standards and criteria, data management, dissemination plan, codes of behavior, and timeline. 

After careful consideration specifically the state of the program and the client's evaluation purposes, I selected the Context, Input, Process, and Product (CIPP) model approach for the case study of Ithika University's Instructional Design Diploma Program. This decision was based on the client's request to get a holistic view of their program, extending beyond mere impact assessment. 

The evaluation focused on four key areas:

1. Program Viability: This will involve assessing the overall feasibility and sustainability of the IDD program, taking into account its long-term prospects and potential for continued operation.
2. Academic Quality: The committee will evaluate the rigor of the curriculum, the effectiveness of the technologies and instructional methods used, and their alignment with industry standards and best practices. This evaluation aims to ensure that the program maintains high academic standards.
3. Learner Relevance: The assessment will determine whether the program meets the needs of its target audience, ensuring it remains relevant and valuable to learners. This evaluation will help maintain the program's appeal and effectiveness in addressing learner needs.
4. Cost Effectiveness: The committee will analyze the program's resource utilization, efficiency, and overall value in terms of return on investment. This evaluation will assist in optimizing resource allocation and ensuring the program's financial sustainability.

The IDD evaluation framework consists of the following four dimensions. The dimensions of this framework work together to create a comprehensive assessment, providing a holistic view of the IDD program.  

• Context – In the IDD program, context refers to the foundational elements that shape its design and delivery. It encompasses the results of needs assessments, opportunities that influence program goals, and various factors that impact the educational experience such as learner demographics, organizational culture, and external environmental influences that affect the educational experience.
• Input - Input involves the necessary resources, infrastructure, curriculum, and content needed to implement teaching-learning processes effectively. This includes materials, facilities, budget, technology, and other assets required for successful program delivery.
• Process - The process includes all activities necessary for implementing different teaching methods, evaluations, and formative assessments. It covers how instructors deliver content, facilitate discussions, and assess student progress throughout the program.
• Product - Product evaluation focuses on assessing skills, values, attitudes, and results achieved by learners. It examines whether the program has met its intended outcomes and contributed to learners' personal and professional development.

After that, I created a comprehensive list of evaluation questions for each dimension of the CIPP model.

After creating these questions, I created the preliminary standards and criteria. These criteria serve as indicators of compliance and identify areas needing improvement. Standards and criteria may evolve throughout evaluation as new issues emerge. 

I also drafted the questions for the assessment instruments by following the guidelines in the course guide. 

Reflection 

Throughout the Spring semester, engaging with the MDDE 620 course's instructional materials, assignments, live sessions, online forums, and peer reviews has profoundly deepened my grasp of both the theoretical groundwork and the modern-day challenges inherent in Education 4.0 technologies, alongside the shifting needs of 21st-century learners. Additionally, the course offered me an invaluable opportunity to explore change management tactics and enhance my abilities in advocating my selected EdTech application within an organization through a compelling video presentation.

My semester journey through the MDDE 620 course not only strengthened my skills and confidence in teaching within the digital realm of 21st-century education but also highlighted the importance of continuous learning and flexibility amidst the constant changes in educational settings.

Reflecting on my participation in the MDDE 620 course, I actively contributed to both online forum discussions and live sessions. My involvement in the online forum discussions surpassed the typical expectations; I found these discussions helpful in enhancing my comprehension of the subject matter.

Unlike other courses where each unit might be broken down into sub-components like learning outcomes, key terms for each unit, and new information or knowledge, the MDDE 620 course structured its content differently. This unique approach meant that the online forum discussions and live sessions became crucial for me in grasping the core concepts of each unit in addition to the unit’s reading materials.

The reading materials assigned for each unit significantly enhanced my readiness for both the forum discussions and assignment work. Notably, Fahy’s publication, "Technology in Education and Training" (2015)[1], served as an invaluable reference throughout the course, akin to a personal dictionary.

The live sessions also stood out as the most enjoyable aspect of the course. I participated in all the live sessions and one optional office hour. Beyond participating in virtual meetings and interacting with Professor Mae and my peers, I was particularly impressed by Professor Mae's approach to integrating instructional materials. She utilized Mentimeter beforehand to gather our insights or comments on specific subjects, and her presentations often featured advanced technological tools. Additionally, these sessions not only deepened our comprehension of each assignment but also ensured we were thoroughly prepared.

I found immense satisfaction in completing the A2 video assignment. This task demanded a blend of creativity and technical expertise, pushing me to devise an engaging narrative and design. The process was both challenging and rewarding, as it allowed me to synthesize the knowledge acquired throughout the course with practical application of technology and design skills. Additionally, one of the most enriching aspects of this assignment was the chance to observe and learn from the exceptional work of my peers, despite being asked to review only three videos. My curiosity led me to view every video, each showcasing remarkable talent and innovation.

Summary

In conclusion, I would rate my overall engagement in this course as an outstanding 10 out of 10 across all dimensions of class participation. Through my active participation in this course, I have gained the skills needed to incorporate technology into my pedagogical strategies and continuously update my knowledge and skills to keep my teaching relevant and effective in the digital age.

Thank you, Prof. Mae, for all your dedication, kindness, and guidance throughout this journey. Your exceptional mentorship has not only enriched my academic experience but also inspired me to pursue excellence in education.  I hope to see you again in another course.

[1] Fahy, P. J. (2015). In Earlier study guide for MDDE 620. Athabasca University

The following video explains why you should use ScreenPal to provide meaningful feedback for online learners.

https://youtu.be/byi1oAHmN_s?si=WSf8QnocCJkfHytD

Feedback plays a crucial role in enhancing the quality of learning experiences. It serves as a potent tool for strengthening the relationship between instructors and learners, creating an environment that fosters growth and continuous improvements.  However, some research showed that written feedback in an online learning environment can be even more challenging as learners can overlook instructors' messages and fail to make meaningful revisions to assignments. 

Crook et al. (2012, as cited in Joseph-Edwards & Edwards, 2022) and Kilickaya (2016) highlighted that the integration of technology into feedback processes promises to deliver feedback more effectively, engage learners more deeply, and ensure prompt delivery, ultimately leading to superior learning outcomes. 

Therefore, I wrote a paper as part of the MDDE620 assignment to analyze the use of ScreenPal, formerly Screencast-O-Matic, for delivering video feedback. For a detailed overview, kindly refer to the attached document or visit this link to read my full paper. 

Hello Everyone,

I would like to share my learning journal for my journey in MDDE619, Trends and Issues in Instructional Design for Fall 2023. Please check it out this link: https://sites.google.com/view/galuh-elearningjournal/home 

“AI will create a learning revolution, and it’s only just getting started. It has the potential to help teachers and students. While we won’t achieve perfection in its implementation, suppressing AI would be misguided. Instead, we must focus on empowering students and teachers to utilize AI ethically and responsibly, providing them with the necessary support to navigate this evolving landscape.”

(Alexander McNeece, Director of Instructional Services Garden City Public Schools)

The quote from McNeece above describes where we, as an educator, should stand on AI. Artificial intelligence (AI) is here to stay. If we want our learners to be workforce-ready when they graduate, we must teach the ethical use of AI across the curriculum rather than focus on the potential misuse of AI e.g., plagiarism. Plagiarism or cheating is not the reason to not teach or use AI in the curriculum. Academic integrity is about developing ethical decision-making skills that learners can use daily in and beyond the classroom. Academic integrity is a foundation for professional practice (Eaton, 2023).

Based on my experience using AI tools throughout this semester, I believe AI can be used as our digital assistant to accelerate our work as instructors to become increasingly productive e.g., create lesson plans, quizzes, modules, or discover ideas/resources/materials for our lectures and to improve our writing. For learners, AI can assist learners in collecting and analyzing data/information; be helpful assistance when the instructor is not available; or help learners individualize their learning opportunities e.g., translating materials to their language or revising the materials to suit the learners’ level or skill.

However, we have to make sure to check the accuracy of the AI outputs because AI sometimes produces outputs seemingly intelligent responses that are nonsensical (AI hallucination or confabulation). AI makes a guess based on training, data used, and learned associations resulting in unpredictable outputs and can be biased. ChatGPT is one of the AI tools that I believe will continue to grow in education and as of today, this tool is trained to provide you outputs for information and resources up to January 2022. You can check the accuracy by checking with the subject matter expert, other AI tools, or Google Scholar.

Privacy is another concern in AI. Without us knowing it, technologies that we use e.g., smartphones and the internet have made AI easier to collect personal data. I often feel that am being monitored by my smartphone, tablet, and laptop e.g., I suddenly receive offer coupons based on my past purchases or internet displayed suggested sites based on my search histories. 

If we want our learners to be workforce-ready when they graduate, we must teach the ethical use of AI across the curriculum rather than focus on the potential misuse of AI e.g., plagiarism. Plagiarism or cheating is not the reason to not teach or use AI in the curriculum. Academic integrity is about developing ethical decision-making skills that learners can use daily in and beyond the classroom. Academic integrity is a foundation for professional practice (Eaton, 2023).

To build the capacity for the proper use of AI, we need to start educating ourselves first to understand more about AI including its benefits, limitations, and challenges, and know how to mitigate the risks of AI. In the next step, we can equip our learners on how to use AI responsibly and ethically by creating an AI ethical guideline and engaging learners with interactive routine discussions about their experience in using AI to foster their ability to recognize AI ethical risks and concerns.

When I started this course, I had very limited knowledge about AI or emerging technology. I do not think I could write my reflection about AI above. Now, I know a number of emerging technologies e.g., AI, gamification, and immersive VR/AR. The hands-on experience or learning exercise and blog critique specifically the multimedia are my favorite parts of this course. These tasks allowed me to learn and practice how to use them, and to foster my critical pedagogical thinking on how to apply those tools to my teaching and learning practices.

The synchronous sessions are my favorite as well. Dr. Levina provided lectures about the materials of the week. The information and discussions with my classmates about the topics gave me a better perspective and understanding than just reading the course materials.

For future development, it would be great if the assignments for this course could be focused on the learning exercises and blog critique multimedia (hands-on experience).  The discussion forums also focus on those hands-on experience activities. I think it would be great to learn from other experiences (as the Unit 4 Primer Discussion).

In summary, it has been a great learning journey this semester. Thank you, Dr. Levina, for all your dedication, patience, and support throughout this course.  You are an amazing professor and I hope I will see you again in another course. 

References:

Eaton, S. E. (2023). Using Generative AI Ethically: Teaching, Learning, and Assessing in a Postplagiarism Era. [Conference Slides]

Hwang, G. J., & Chen, N. S. (2023). Exploring the Potential of Generative Artificial Intelligence in Education: Applications, Challenges, and Future Research Directions. Educational Technology & Society, 26(2).

“Digital badges are a credible means through which learners can establish portfolios and articulate knowledge and skills both in academic and professional settings.”  (Muilenburg & Berge, 2016)

I just completed the blog critique and multimedia assignment. The following link is my video on this assignment: https://youtu.be/GlAIGCshn-0?si=Iu-ybq0NMl7_oL_l

It was my first attempt to research and utilize the digital badges. I read journals and blogs to get more knowledge about this tool to gamify online learning environments. A digital badge is a visual award system for learners to foster the motivation of online learners, recognize the online learners' achievement, and establish credibility. Digital badges have the same functions as when we award small prizes to reward our learners in the class because of their achievements or when we implement games in our face-to-face class to motivate our learners. Interestingly, there is metadata or information attached to the digital badge to verify the information about the skills or courses that learners achieved to earn the badge.

I learned a few digital badge tools e.g., Canvas Badges, Accredible, and Certifier, to create the badge graphics, attach the metadata, issue the badge to the learner, and find the strategy to integrate digital badges into the Canvas learning management system. Badge graphics can be created by using digital badge tools or other tools such as Canva, PowerPoint, or Adobe. Surprisingly, many free badge graphics on the internet can be used.

Canvas Badges, Accredible, and Certifier have a free account that I can use to create badge graphics, attach the metadata, and issue the badge to my learners.  All the tools are very easy to use and user-friendly. There is no limited number of how many badges you can issue for your learners except Accredible. You can issue up to 20 credentials either badge or certificate for Accredible. 

Unfortunately, I cannot integrate the digital badge tools into the learning management system e.g., Canvas. You must have a paid subscription. I found an alternative on how to use this gamification strategy without integrating the digital badge into our online course by creating a “hidden” module with release conditions (learners must complete the module or pass a quiz to release the badge module. The badge module contains a page with a message about the badge that the learners earn. However, the weakness of using this strategy is the instructors have to monitor the course to regularly check the learners who meet the release conditions so that instructors can issue the badge through the selected digital badge platform.

Surprisingly, I discovered that my institution’s LMS, D2L Blackboard, has an “AWARDS” feature under Assessments where we can directly create and issue digital badges and/or certificates to our courses. You can create the badge graphic, assign the metadata associated with the badge, and release conditions for learners to earn this badge. However, my institution only allows the learners who earned the digital badges to share their badges internally e.g., their profile or e-portfolio. Unfortunately, I was not able to show this “AWARDS” feature in my video due to privacy/proprietary information.

In general, I would apply digital badges to my online courses as one of the strategies to motivate and engage my online learners in their learning. I believe that my online learners will be happy when they earn digital badges and the fact that they can share their badges on their professional and/or social networks e.g., LinkedIn, Facebook, or Instagram.

This week, the MDDE663, Emerging Issues in Distance Education Technologies, is in week 3.  Honestly, after reading and listening to their experience with emerging technologies such as AI and ChatGPT in the introduction and synchronous session in the first two weeks, I was so nervous about this course because I am not sure if I have enough knowledge and practical skills in AI or ChatGPT. One of my classmates even used Bing Chat (Chat GPT powered by GPT-4) to create her technology timeline!

The CBS Saturday Morning interview with Geoffrey Hinton, the godfather of AI, has given me knowledge of how AI works, its benefits, and the AI revolution.  I learned a new word “deep learning” in this interview which is a popular approach to machine learning.

My biggest concern about AI was that this technology could be smarter than humans and do anything. Their IQ is more than Albert Einstein's, they can be smarter and smarter each year! Maybe, in the future, we will not have our own opinions anymore because they can answer everything. Will AI wipe out humanity?

Geoffrey Hinton and other reading materials about AI in week 2 agreed that AI possibly can wipe out humanity.  However, we, as humans, need to embrace and utilize AI synergy and use them to help and support our work since we cannot stop technology. In education, we can use AI and ChatGPT as our “digital assistants” to accelerate and become increasingly productive in building teaching and learning environments that are more engaging, passionate, and effective.

Gillani et al. (2023) shared all the benefits and limitations of AI in education and mentioned that one of AI’s limitations is AI has a lack of abstract reasoning and learning how to learn. Meanwhile, Humans are very good at it.

Another element that I think is important to remember is technology cannot replace the human connection. Every aspect of our lives including our jobs depends on human connection. This will never go away. Gawdat (2023), ex-Google CBO, said that your life purpose is you and your only product at the end of your life is how far you’ve come. Thus, the human connection will remain available.

One of my classmates posted that she found ChatGPT very helpful in creating a pop quiz. Yuen (2023) added that ChatGPT is incredibly fast at coming up with ideas and lessons. Wow, this is amazing.  I usually use H5P to create a pop quiz. Yuen (2023) also suggested her learners keep the window open with ChatGPT so that they can cross-check materials or ask questions as they arise. I am amazed at how many people have used this technology in their daily lives. At my work, we are still debating whether our learners and staff could use ChatGPT.  Using ChatGPT is still considered plagiarism at my institution.

Now, I realize that AI and ChatGPT can be beneficial if we embrace and use them wisely to be our digital assistants to accelerate our work. It is time for me to have practical experience in using AI and experience the benefits of it!

REFERENCES:

Gillani, N., Eynon, R., Chiabaut, C., & Finkel, K. (2023). Unpacking the “Black Box” of AI in Education. Educational Technology & Society, 26(1), 99-111.

James Laughlin. (2023, September 13). Urgent: Ex-Google CBO says AI is now IMPOSSIBLE to stop with Mo Gawdat. YouTube. https://youtu.be/fDHvUviV8nk?si=K2mMDFnS2ghyyVnR

The first collaborative work in this program and the first assignment was we were asked to write a group collaborative paper.  Our group decided to write the critical review on the following paper "Exploring a Community of Inquiry Supported by a Social Media-Based Learning Environment".

I loved my group.  Each member knew their strengths and weaknesses.  We stepped up and filled each other's weakness.  I think that was a good point of working in group.  We must be back up each other.

We created a group in Whatsapp as our communication platform and google doc for our collaborative work.  These platforms worked well for us.

The challenge was only to unify our expectations but that was not a big problem for our group.  We knew when we should step up to cover others and understand each other as well.

I was so happy that my  mark for first assignment is not that bad! And as I guessed, I have grammatical errors in my writing.  This is definitely a learning curve for me and I am sure that my writing will get better and better.