by Moreno-León, Jesús & Robles, Gregorio Universidad Rey Juan Carlos. Madrid, Spain.
There are many technologies that have been created aiming to assist students in the development of computational thinking skills, such as Logo [1], Alice [2] and, especially, Scratch [3], which has become the standard programming language to teach computing in schools. However there is a lack of tools that support both teachers and learners in the assessment of this competence.
Dr. Scratch is a free/libre/open source web tool that analyses Scratch projects to offer feedback to educators and learners by assigning a computational thinking score to the projects. This computational thinking score, which ranges from 0 to 21 points, is based on the degree of development of different dimensions of the computational thinking competence, specifically abstraction and problem decomposition, logical thinking, synchronization, parallelism, algorithmic notions of flow control, user interactivity and data representation, which are evaluated by inspecting the source code of the analysed project [4]. Dr. Scratch also detects certain bad habits of programming or potential errors that are commonly found in the projects of the Scratch repository [5, 6], such as non-significant sprite names, repetition of code, code that is never executed and the incorrect initialization of object attributes, as shown in Figure 1.
Figure 1 – Dr. Scratch feedback report assigns a computational thinking score and detects some bad programming habits
For each of the bad programming habits detected in the code and for each of the computational thinking dimensions where there is room for improvement, the tool provides links to information that can be used to improve the projects. For example, if a project includes repeated code, Dr. Scratch provides a link to sample source code and an explanation of why this situation should be avoided and how users could improve their project by developing their own blocks (see Figure 2).
Figure 2 – Ideas and tips provided by Dr. Scratch to avoid code repetition by creating new blocks
Aiming not to overwhelm novice programmers, the feedback report provided by Dr. Scratch in the results page depends on the computational thinking score. Thus, if the score is low the tool will only show basic information of the most important improvements to perform in the code. As the score increases, Dr. Scratch will provide more information of the analysed projects.
By offering a partially gamified interface [7], with scores and levels, Dr. Scratch tries to encourage learners to improve their programming skills. In a previous investigation [8], over 100 learners in the range from 10 to 14 years participated in a series of workshops where they tried to improve their Scratch projects using the advice offered by the tool. The results show that participating students increased their score and, in consequence, enhanced their computational thinking skills.
Dr. Scratch is being used by teachers and organizations from all over the world as a support tool in the evaluation tasks [9]. For instance, the tool can be used to detect students that do not use certain instructions in their projects, such as logic operations or clones, so teachers can prepare specific tasks to help them understand their importance.
“Basically, I really like the opportunity to give the students a totally subjective (and external to their teacher) perspective on their coding and where it does and doesn’t quite match up. I also appreciate the reduction in my time in analysis and marking, including trying to find where their code might be in error if it is not doing what it should.
I normally still need to mark to see if they have met the criteria I have set, but Dr. Scratch is a huge help still!”
Paul Herring, St Peters Lutheran College, Australia.
“Students often believe that if their project “works”, then their project cannot get any better. I use Dr. Scratch to show weaknesses in the code that can be improved bringing in the foreground all the computer science concepts students are learning actively but silently. Dr. Scratch helps my students to improve their coding skills in a way that can be transferred to professional programming languages. I often get feedback from my students that when they use Dr. Scratch their work seems more relevant to the computer course and that it enhances their feeling of achievement.”
Dimitris Nikolos, Model Experimental Junior High School of University of Patras, Greece.
There are some shortcomings that we plan to address in the near future. So, the analysis of a single project by a learner does not provide a complete picture of his/her computational thinking development. The reason is that there are wonderful simple projects that do not require to be modified in order to include more complex structures (those that give a higher computational thinking score). In the near future the development team of Dr. Scratch will include a new feature to allow the creation of user accounts. In consequence, the analysis of the portfolio of projects of the users will provide a richer picture, as the aggregate scores will allow detecting gaps of knowledge more accurately.
It is worth mentioning that although Dr. Scratch is successfully supporting teachers in the assessment tasks, the tool should not be understood as a replacement of evaluators or mentors, as there are key computational thinking skills, such as debugging, that are not evaluated. Moreover, functionality, originality or creativity, key aspects of programming projects, are not either taken into account in the score provided by Dr. Scratch.
Acknowledgments
The work of both authors has been funded in part by the Region of Madrid under project “eMadrid – Investigación y Desarrollo de tecnologías para el e-learning en la Comunidad de Madrid” (S2013/ICE-2715). The work of Gregorio Robles has been funded in part by the Spanish Government under project SobreSale (TIN2011- 28110). We would like to thank as well Eva Hu and Mari Luz Aguado for their technical support with Dr.Scratch. References
[1] S. Papert (1980). Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc.
[2] Cooper, S., Dann, W., & Pausch, R. (2000, April). Alice: a 3-D tool for introductory programming concepts. In Journal of Computing Sciences in Colleges (Vol. 15, No. 5, pp. 107-116). Consortium for Computing Sciences in Colleges.
[3] Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., … & Kafai, Y. (2009). Scratch: programming for all. Communications of the ACM, 52(11), 60-67.
[4] Moreno-León, J., & Robles, G. (2015). Analyze your Scratch projects with Dr. Scratch and assess your computational thinking skills. Proceedings of the Scratch Conference 2015, (pp. 48–53). Amsterdam, The Netherlands. [5] Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (2011, June). Habits of programming in scratch. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education (pp. 168-172). ACM.
[6] Moreno, J., & Robles, G. (2014). Automatic detection of bad programming habits in scratch: A preliminary study. Proceedings of the Frontiers in Education Conference (FIE), 2014 IEEE (pp. 1–4). IEEE.
[7] Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps. O’Reilly Media, Inc.
[8] Moreno-León, J., Robles, G., & Román-González, M. (2015). Dr. Scratch: Automatic analysis of Scratch projects to assess and foster computational thinking. Revista de Educación a Distancia, (46).
[9] Moreno-León, J., & Robles, G. (2015). Dr. Scratch: a Web Tool to Automatically Evaluate Scratch Projects. In Proceedings of the Workshop in Primary and
