50 research outputs found
Enabling peer review in large University courses
No full textIn the Austrian university education, we are often faced with large classes of 600+ students. While this situation cannot be changed due to university policies and financing, we still aim at providing the best possible teaching and learning experience for such a setting. We highly value constructivist learning, which is driven by the need for personalised feedback throughout the individual learning process. Due to the high students numbers, we lack the resources to connect and interact with every single student, however, an alternative is to encourage a reciprocal community, in which students take charge of their own learning and support each other on their journey through peer review. However, we found that the basic peer review process, as used in scientific settings or Massive Open Online Courses, did not fully meet our expectations when introduced in a first year lecture. We found that students were hesitant in providing critical feedback about their peers work; that the extra step of writing reviews often had a negative impact on their motivation to continue their own work; that they were discontent with the extra workload and that they did not fully trust their peers perspective on their work. Hence, we saw the need to redesign and adapt the basic peer review system to better fit our students needs. This thesis reports on our process and findings from using an exploratory design approach to implement peer review in the bespoke learning platform Aurora over the span of 6 years to answer the research question: "What do we need to consider when designing peer review for large university courses to meet students needs and make the process more sustainable to organise and perpetuate?". Since we are dealing with an ill-defined problem existing within a bigger technological learning environment, we took an exploratory design approach. We introduced a basic version of peer review into the lecture, observed how it was received throughout the semester and conducted a survey with students at the end of the semester towards gaining a better understanding of the context and its issues. This pointed us towards three main challenges when using peer review in large university courses: the varying quality of reviews, conveying a feeling of fairness, and keeping up student motivation in a self-directed learning environment. Based on these findings we iteratively created design artefacts to explore possible solutions to these challenges. In this work, we give an overview of such recurring design challenges and our response in the form of influential design artefacts that impacted peer review activities. Furthermore, we expand on our research practices and reflect on challenges arising from the use of design-based research and co-creation methods in our long-term, real-life context. This work contributes to the field of technology enhanced learning and teaching in that it reflects on design aspects necessary to create a better learning environment for students in the context of peer review in large university courses, such as better guiding students through the semester, visibility of every action, transparency of the whole process, or avoiding ranking amongst students. While these are aspects drawn from work in our particular context, our findings also lead us to the formulation of five transferable design considerations, suggesting to plan for scaffolding', 'contextualisation', 'framing and mindset', 'value' and 'transparency' when designing technology for educational contexts
Peer Review als Grundlage für innovative Leistungsbeurteilung
No full textPeer Reviewing ist ein essentieller Bestandteil wissenschaftlichen Arbeitens und dient der Sicherstellung von Qualität sowie dem aktiven Diskurs zwischen WissenschafterInnen. Durch das Aufkommen von MOOCs und dadurch den vermehrten Bedarf nach innovativen Methoden zur skalierenden Leistungsbeurteilung, wurde Peer Review schnell als mögliches Mittel zum Zweck identifiziert.
Die TU Wien beschäftigt sich seit mehr als 10 Jahren mit dem Einsatz von neuen Formen der Leistungsbeurteilung, vor allem in der Massenlehre, im Rahmen derer oft mehr als 800 Studierende eine Lehrveranstaltung besuchen. In dieser Keynote werden die dabei gewonnenen Erkenntnisse anhand von Beispielen und Anekdoten dargelegt, die sowohl unsere Erfolge wie auch Misserfolge zeigen und so einen tieferen Einblick in die Entwicklung und Umsetzung einer interaktiven Lernplattform als Basis der Leistungsbeurteilung geben
Can e-learning systems profit from the integration of social software?
No full textDiese Arbeit beschäftigt sich mit der Frage, inwieweit der Einsatz von Social Software im Kontext von E-Learning Systemen positive Auswirkungen auf die Leistungen der BenutzerInnen hat. Es wird untersucht, ob Studierende, die Social Software im Bildungskontext anwenden, bessere Ergebnisse liefern als jene, die versuchen, allein zu arbeiten. Die Arbeit bietet einen Überblick über Entwicklung und Auswirkungen von Lerntheorien, E-Learning und Social Software in der heutigen Gesellschaft. Weiters werden bereits bestehende Projekte und Untersuchungen zum Thema aufgelistet. Anschließend wird ein Projekt vorgestellt, anhand dessen die Fragestellung konkret untersucht wird.Das Projekt wurde an der Technischen Universität Wien entwickelt, und besteht aus mehreren Komponenten, die alle wichtigen Bereiche einer E-Learning Applikation abdecken sollen. Die Komponenten wurden durch verschiedene Social Software Elemente erweitert und sollen einen interaktiven und regen Austausch aller TeilnehmerInnen zu den Inhalten der Vorlesungen fördern. Das Projekt findet seit einigen Semestern Einsatz in verschiedenen Lehrveranstaltungen an der Technischen Universität Wien. Die daraus resultierenden Daten werden in dieser Arbeit ausgewertet und die Ergebnisse präsentiert. Abschließend werden Möglichkeiten der Weiterentwicklung des Projekts diskutiert.This thesis focuses on the question of whether or not using social software in an e-learning context has a positive effect on the users' achievements. It analyzes the performance of students who are involved in social software interactions in comparison to those students who only use the e-learning possibilities offered. Furthermore, an overview of the development and effects of learning theories, e-learning and social software in modern society is given. Current projects and surveys on the topics are presented to provide models of good practice.The posed question is analyzed using a project which was created at the Technical University of Vienna. The project consists of various e-learning components which are enhanced with social software elements.The goal of the project is to provide a means for an interactive and productive online communication on the topics of the course. The project has been in use in various lectures at the Technical University of Vienna over the last six semesters. The resulting data will be examined for the purpose of finding answers to the posed question and the results will be presented in the thesis. Finally, possibilities of future work will be discussed
Badges as a component of reputation management in university education
No full textIn dieser Arbeit wird exploriert, wie sich, im Internet etablierte Reputationsmechanismen, sowie Gamificationkonzepte, in einen Universitätskontext der TU Wien einbetten lassen. Um so etwa die Lehre zu verbessern, eine granulärere Beurteilung der Leistungen von Studierenden zu ermöglichen oder etwa einen zusätzlichen Anreiz zu außerordentlichen Leistungen für Studierende zu bieten. Ein Gamificationkonzept, das primär betrachtet wird, sind sogenannte Badges. Diese sind digitale Abzeichen, welche für die Erfüllung gewisser Aufgaben oder das Erreichen bestimmter Ziele vergeben werden. Es wird auch diskutiert, inwieweit bereits vorhandene Reputation von externen Quellen eingebracht werden kann und wo Reputationsmechanismen in bestehenden Systemen wie z.B. TUWEL oder TISS eingesetzt werden können. Mit Hilfe von Literaturrecherche und Interviews wird der Status Quo in der Forschung und im Einsatz befindliche Methoden sowie Ansprüche an ein Reputationssystem erhoben. Darauf basierend entstanden ein Prototyp für ein Badgevergabesystem und ein konzeptuelles zentrales System für die TU Wien um Reputation zu erfassen und präsentieren.This paper explores how reputation methods established in the internet as well as gamification concepts could be used in the university context of the TU Vienna in order to improve teaching, enable a more granular rating of student‘s performance or motivate students in general to show exceptional results. A gamification concept this work will focus on are badges. These are digital awards that can be earned by completing certain tasks. In this paper will be discussed as well how already existing reputation information could be aggregated and how already existing systems like TUWEL or TISS could be augmented with reputation methods. In order to define current reputation methods and requirements to a reputation system literature research and interviews were conducted. Based on the findings a prototype for a platform to award and present badges as well as a conceptual TU Vienna centric reputation aggregation system were designed
Using non-digital games to enhance learning in interdisciplinary approaches to informatics
No full textGame-based Learning (GBL) ist ein interdisziplinäres Forschungsfeld mit wachsender Popularität. An der TU Wien wird es in der Lehrveranstaltung "Denkweisen der Informatik" im Bachelor-Studiengang Informatik eingesetzt. Dort werden Brett- und Kartenspiele, die Themen der Lehrveranstaltung adressieren, von erstsemestrigen Studierenden gespielt, die danach über die GBL-Aktivität, als Teil der Kursarbeit, reflektieren. Frühere Anwendungen von GBL in diesem Rahmen waren überwiegend erfolgreich und haben gleichzeitig Verbesserungsmöglichkeiten aufgezeigt. In dieser Arbeit werden die Spiele analysiert, die in der nächsten Iteration von GBL im Kurs verwendet werden sollen, und es werden für jedes Spiel spezifische Reflection Prompts erstellt, die das Lernen durch Spiele verbessern sollen, indem der Reflexionsprozess der Studierenden dadurch unterstützt wird. Darüber hinaus wird in dieser Arbeit untersucht, wie Studierende von DWI den Lerneffekt der Spiele, der GBL-Aktivität und GBL als Methode wahrnehmen. Für die formale Analyse der Spiele werden zwei etablierte Frameworks für Lernspiele verwendet und mit Design-Artefakten kombiniert, die von den Designer:innen der Spiele zur Verfügung gestellt wurden. Prä- und Post-Workshop-Fragebögen werden designed und die Ergebnisse daraus analysiert, um die Antworten der Studierenden zu diskutieren.Die Ergebnisse dieser Arbeit können als Designüberlegungen für zukünftige Lernspiele und zukünftige GBL-Aktivitäten dienen, um das Lernen in GBL-Aktivitäten allgemein, und ebenso im Informatikstudium im Besonderen, zu verbessern.Game-based learning (GBL) is an interdisciplinary field of study that has been growing in popularity. At TU Wien, it is used in the course "Denkweisen der Informatik" (eng.: ways of thinking in informatics) in the bachelor's program of Informatics. There, non-digital games that address topics of the course are played by first-year students. These students then reflect on the GBL activity as part of the course work. Past applications of GBL in this setting have been mainly successful, and at the same time revealed potential areas for improvement. This thesis analyzes the games to be used in the next iteration of GBL in the course, and creates reflection prompts specific to each game, which should enhance learning through games by supporting the students in the reflection process. Additionally, the thesis investigates how students of DWI receive the learning impact of the games, the GBL activity, and GBL as a method. Two established assessment frameworks for educational games are used to analyze the games formally, combined with design artifacts provided by the designers of the games. Pre- and post-workshop surveys are designed and the results analyzed to discuss students' responses.The results of the thesis can serve as design considerations for future games with a learning purpose, and future GBL activities, enhancing learning in GBL activities overall, as well as in Informatics education specifically
Exploring the use of peer review in large university courses
No full textDouble blind peer review is a standard practice in the scientific community. It acts as a means of validating work as well as of getting feedback to improve it. As such, it seems prudent to also use it as a learning tool in large lectures to provide students with personalized feedback on their work. The general process can be directly adopted for the lecture context, but details need to be modified and adapted to create a better learning experience. The structure of a large lecture has been adjusted to provide the context for a double blind peer review process. Not only has the evaluation of activities during the semester changed to fit in with the double blind peer review, but also the organization of said activities was adapted to accompany the evaluation change. The first semester yielded promising results, but also pointed towards some issues with the current state of the system. We devised a list of design implications for future revisions of the double blind peer review system, based on feedback and experiences during the semester as well as on a survey among students at the end of the semester. These implications will be implemented to improve and refine the new system for upcoming semesters
Layout Considered Harmful: On the Influence of Information Architecture on Dialogue
No full textDiscussionsareanimportanttoolforstudentstoengagewith new content, bring up new ideas and generate knowledge. This paper fo- cuses on the representation of asynchronous online discussion forums in an e-learning context and how it influences the outcome of discus- sions. We compare the results of a traditional discussion visualisation - a vertically threaded comment system - to the two dimensional system Discourse, in which every new statement of the discussion opens a new subthread. We draw our conclusions from a qualitative analysis of pairs of discussions on the same topic conducted in both systems. Our findings suggest that discussions in Discourse are more focused and goal-oriented than in traditional threaded system
Exploring the Use of Peer Review in Large University Courses
No full textDouble blind peer review is a standard practice in the scientific community. It acts as a means of validating work as well as of getting feedback to improve it. As such, it seems prudent to also use it as a learning tool in large lectures to provide students with personalized feedback on their work. The general process can be directly adopted for the lecture context, but details need to be modified and adapted to create a better learning experience. The structure of a large lecture has been adjusted to provide the context for a double blind peer review process. Not only has the evaluation of activities during the semester changed to fit in with the double blind peer review, but also the organization of said activities was adapted to accompany the evaluation change. The first semester yielded promising results, but also pointed towards some issues with the current state of the system. We devised a list of design implications for future revisions of the double blind peer review system, based on feedback and experiences during the semester as well as on a survey among students at the end of the semester. These implications will be implemented to improve and refine the new system for upcoming semesters
Aurora - Exploring Social Online Learning Tools Through Design
No full textTeaching is an integral part of our work at university and we strive to achieve a high quality in teaching our students. Yet, we face classes with up to 800 students per semester and do not have the resources to ensure a close rapport with each and every one of them. For the last 6 years we have been working on an online solution for this problem, with the aim of letting students take responsibility of a large part of their own learning process. The e-learning system Aurora envelops a number of components ranging from organizational and informational tools to discussion systems and student portfolios. Students are invited to participate actively in this online learning environment and are presented with a variety of options they can choose from in order to accumulate enough credits to pass their courses. Over the years of developing Aurora, issues and inadequacies of the system became apparent and lead us to change our system design iteratively, learning from its shortcomings
Explorative Design as an Approach to Understanding Social Online Learning Tools
No full textThe everyday availability and use of technology has changed education as much as it has changed everything else. For 8 years now, we have used technological interventions to change a setting where we teach up to 800 participants per semester in a class, in order to make it more interactive, engaging, and interesting for the students. We document a snapshot from this ongoing process. Aurora is an online system that has been developed from simple experiments with existing tools and software to bring interaction to the crowd of learners. Over the years, it has turned into a solid and extensive collection of tools for online teaching, learning, and communication. This article traces the development of Aurora over two consecutive years. We document the structure of the system we developed, the insights from an academic year of using it, changes designed and implemented, and first evaluations of the use of the revised version
