Assessing Learning Skills and Work Habits: What Do Report Card Data Tell Us?
DOI:
https://doi.org/10.53967/cje-rce.v45i3.5191Keywords:
learning skills, report cards, classroom assessment , self-regulation , 21st century skillsAbstract
Many school systems ask teachers to assess and report upon aspects of student performance beyond academic achievement. In Ontario, K–12 teachers assess a common set of six Learning Skills and Work Habits. How well teachers are able to undertake these assessments is not well studied. This study examines Grade 9 and 12 report card data from two districts in Ontario, Canada to determine to what extent different learning skills are assessed independently of each other, and to what extent they are associated with teacher-awarded academic achievement and achievement on a standardized Grade 9 mathematics examination delivered by Ontario’s Educational Quality and Accountability Office (EQAO). Results indicate that the set of six Learning Skills and Work Habits are assessed as a unitary construct. Grades on these skills have higher correlations with teacher-awarded grades than with standardized test scores. Finally, gender differences in both academic achievement and achievement on the set of skills are investigated.
Metrics
References
Allal, L. (2013). Teachers’ professional judgement in assessment: A cognitive act and a socially situated practice. Assessment in Education: Principles, Policy & Practice, 20(1), 20–34.
Almlund, M., Duckworth, A. L., Heckman, J. J., & Kautz, T. D. (2011). Personality psychology and economics (No. w16822). National Bureau of Economic Research.
Association for Supervision and Curriculum Development. (2007). The learning compact redefined: A call to action. http://www.ascd.org/ASCD/pdf/Whole%20Child/WCC%20Learning%20Compact.pdf.
Babad, E. Y., Inbar, J., & Rosenthal, R. (1982). Teachers’ judgment of students’ potential as a function of teachers’ susceptibility to biasing information. Journal of Personality and Social Psychology, 42(3), 541–547.
Bass, K. M. (2005). Reality’s limits. Measurement: Interdisciplinary Research and Perspectives, 3(2), 84–88.
Baumeister, R. F., & Vohs, K. D. (2007). Self-regulation, ego depletion, and motivation. Social and Personality Psychology Compass, 1(1), 115–128.
Bernardin, H. J. (1978). Effects of rater training on leniency and halo errors in student ratings of instructors. Journal of Applied Psychology, 63(3), 301–308.
Bol, L., Stephenson, P. L., O’connell, A. A., & Nunnery, J. A. (1998). Influence of experience, grade level, and subject area on teachers’ assessment practices. The Journal of Educational Research, 91(6), 323–330.
Borghans, L., Duckworth, A. L., Heckman, J. J., & Ter Weel, B. (2008). The economics and psychology of personality traits. Journal of Human Resources, 43(4), 972–1059.
Bowers, A. J. (2011). What’s in a grade? The multidimensional nature of what teacher-assigned grades assess in high school. Educational Research and Evaluation, 17(3), 141–159.
Briesch, A. M., Chafouleas, S. M., & Riley-Tillman, T. C. (2010). Generalizability and dependability of behavior assessment methods to estimate academic engagement: A comparison of systematic direct observation and direct behavior rating. School Psychology Review, 39(3), 408–421.
Brochu, P., Deussing, M. A., Houme, K., & Chuy, M. (2013). Measuring up: Canadian results of the OECD PISA study. Council of Ministers of Education.
Brookhart, S. M. (1993). Teachers’ grading practices: Meaning and values. Journal of Educational Measurement, 30(2), 123–142.
Brown, G. T. (2011). Teachers’ conceptions of assessment: Comparing primary and secondary teachers in New Zealand. Assessment Matters, 3, 45–70. https://link.gale.com/apps/doc/A307413786/AONE?u=queensulaw&sid=bookmark-AONE&xid=f122ed65
Casner-Lotto, J. & Barrington, L. (2006). Are they really ready to work? Employers’ perspectives on the basic knowledge and applied skills of new entrants to the 21st century US workforce. Partnership for 21st Century Skills.
Cizek, G. J. (1996). Grades: The final frontier in assessment reform. NASSP Bulletin, 80(584), 103–10.
Conference Board of Canada. (2015). Employability Skills 2000+. http://www.conferenceboard.ca/topics/education/learning-tools/employability-skills.aspx
Cross, L. H., & Frary, R. B. (1999). Hodgepodge grading: Endorsed by students and teachers alike. Applied Measurement in Education, 12(1), 53–72.
DeLuca, C., & Bellara, A. (2013). The current state of assessment education aligning policy, standards, and teacher education curriculum. Journal of Teacher Education, 64(4), 356–372.
Diamond, A. (2013). Executive functions. Annual Review of Psychology, 64, 135–168.
Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20, 391–409.
Duckworth, A., & Gross, J. J. (2014). Self-control and grit: Related but separable determinants of success. Current Directions in Psychological Science, 23(5), 319–325.
Duckworth, A., & Seligman, M. (2006). Self-discipline gives girls the edge: Gender in self-discipline, grades and achievement test scores. Journal of Educational Psychology, 98(1), 198–208.
Duckworth, A. L., & Schulze, R. (2009). Jingle jangle: A meta-analysis of convergent validity evidence for self-control measures [Manuscript]. University of Pennsylvania, Department of Psychology.
Duckworth, A. L., & Yeager, D. S. (2015). Measurement matters: Assessing personal qualities other than cognitive ability for educational purposes. Educational Researcher, 44(4), 237–251.
Duncan, C. R., & Noonan, B. (2007). Factors affecting teachers’ grading and assessment practices. Alberta Journal of Educational Research, 53(1), 1–21.
Falch, T., & Naper, L. R. (2013). Educational evaluation schemes and gender gaps in student achievement. Economics of Education Review, 36, 12–25. https://doi.org/10.1016/j.econedurev.2013.05.002
Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. University of Chicago Consortium on Chicago School Research.
Ferrito, J. (2015). An investigation of the relationship between behavioral feedback on social-emotional skills on report cards and academic achievement [Unpublished doctoral dissertation]. Rutgers, The State University of New Jersey.
Fieldwork Education (n.d.). International Primary Curriculum. https://fieldworkeducation.com/curriculums/primary-years
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911.
Galla, B. M., Plummer, B. D., White, R. E., Meketon, D., D’Mello, S. K., & Duckworth, A. L. (2014). The Academic Diligence Task (ADT): Assessing individual differences in effort on tedious but important schoolwork. Contemporary Educational Psychology, 39(4), 314–325. https://doi.org/10.1016/j.cedpsych.2014.08.001
Greene, J. A., Robertson, J., & Costa, L. J. C. (2011). Assessing self-regulated learning using think-aloud methods. In B. Zimmerman & D. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 313–328). Routledge.
Guskey, T. R., Swan, G. M., & Jung, L. A. (2011). Grades that mean something: Kentucky develops standards-based report cards. Phi Delta Kappan, 93(2), 52–57.
Hadwin, A., & Winne, P. (2012). Promoting learning skills in undergraduate students. In J. R. Kirby & M. J. Lawson (Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 315–338). Cambridge University Press.
Howley, A., Kusimo, P. S., & Parrott, L. (2000). Grading and the ethos of effort. Learning Environments Research, 3(3), 229–246.
Hunter, D., Mayenga, C., & Gambell, T. (2006). Classroom assessment tools and uses: Canadian English teachers’ practices for writing. Assessing Writing, 11(1), 42–65.
International Baccalaureate Organization. (2009). IB learner profile booklet. http://www.ibo.org/programmes/documents/learner_profile_en.pdf
Ivancevich, J. M. (1979). Longitudinal study of the effects of rater training on psychometric error in ratings. Journal of Applied Psychology, 64(5), 502–508.
Ivcevic, Z., & Brackett, M. (2014). Predicting school success: Comparing conscientiousness, grit, and emotion regulation ability. Journal of Research in Personality, 52, 29–36.
Jacob, B. A. (2002). Where the boys aren’t: Non-cognitive skills, returns to school and the gender gap in higher education. Economics of Education Review, 21(6), 589–598.
Keeley, J. W., English, T., Irons, J., & Henslee, A. M. (2013). Investigating halo and ceiling effects in student evaluations of instruction. Educational and Psychological Measurement, 73(3), 440–457.
Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural equation modeling, 10(3), 333–351.
Kleitman, S., & Costa, D. S. (2014). The role of a novel formative assessment tool (Stats-mIQ) and individual differences in real-life academic performance. Learning and Individual Differences, 29, 150–161.
Lai, E. R. (2011). Metacognition: A literature review. images.pearsonassessments.com/images/tmrs/Metacognition_Literature_Review_Final.Pdf
Lekholm, A. K., & Cliffordson, C. (2008). Discrepancies between school grades and test scores at individual and school level: Effects of gender and family background. Educational Research and Evaluation, 14(2), 181–199. https://gupea.ub.gu.se/bitstream/handle/2077/18673/gupea_2077_18673_2.pdf;jsessionid=BD7AB2B4C5A13D83177AD4FD95AADA66?sequence=2
Levin, H. (2012). More than just test scores. Prospects, 42(3), 269–284.
Linn, R., & Miller, M. (2005). Measurement and assessment in teaching. Pearson Prentice Hall.
McGill, D. A., Van der Vleuten, C. P. M., & Clarke, M. J. (2011). Supervisor assessment of clinical and professional competence of medical trainees: A reliability study using workplace data and a focused analytical literature review. Advances in Health Sciences Education, 16, 405–425.
McMillan, J. H. (2001). Secondary teachers’ classroom assessment and grading practices. Educational Measurement: Issues and Practice, 20(1), 20–32.
McMillan, J. H., & Workman, D. J. (1998). Classroom assessment and grading practices: A review of the literature. Metropolitan Educational Research (ERIC Document Reproduction Service No. ED453-63).
Merchant, S. (2016, September 29–October 1). Assessment and reporting of learning skills and work habits: Capturing the complexity [Conference presentation]. The Annual Conference of the Consortium for Research on Educational Assessment and Teaching Effectiveness, Louisville, KY, United States.
Merchant, S., Klinger, D. A., & Love, A. (2018). Assessment and reporting of non-cognitive skills: A cross-Canada survey. Canadian Journal of Educational Administration and Policy, 187, 2–17.
Miller, T., Klinger, D., & Shulha, L. (2006). Behaviour assessment in Ontario mathematics classrooms. Educational Research and Reviews, 1(1), 1–6.
Muenks, K., Wigfield, A., Yang, J. S., & O’Neal, C. R. (2017). How true is grit? Assessing its relations to high school and college students’ personality characteristics, self-regulation, engagement, and achievement. Journal of Educational Psychology, 109(5), 599–620.
National Education Association. (2012). Preparing 21st century students for a global society: An educator’s guide to the “four Cs.” National Education Association.
Northern Ireland Ministry of Education. (2007). Thinking skills and personal capabilities for key stage 3. Council for the Curriculum, Examinations & Assessment. https://ccea.org.uk/key-stage-3/curriculum/thinking-skills-personal-capabilities
Ontario Ministry of Education. (2005). The Ontario curriculum grade 9 and 10 mathematics. http://www.edu.gov.on.ca/eng/curriculum/secondary/math910curr.pdf
Ontario Ministry of Education. (2010). Growing success: Assessment, evaluation and reporting in Ontario schools. http://www.edu.gov.on.ca/eng/policyfunding/growsuccess.pdf
Ontario Ministry of Education. (2016). 21st century competencies: Foundation document for discussion. http://www.edugains.ca/resources21CL/21stCenturyLearning/21CL_21stCenturyCompetencies.pdf
Pellegrino, J., & Hilton, M. (2013). Education for life and work: Developing transferable knowledge and skills in the 21st century. National Academies Press.
Pilcher, J. K. (1994). The value-driven meaning of grades. Educational Assessment, 2(1), 69–88.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40.
Poth, C. A. (2012). What assessment knowledge and skills do initial teacher education programs address? A Western Canadian perspective. Alberta Journal of Educational Research, 58(4), 634–656.
Protivínský, T., & Münich, D. (2018). Gender bias in teachers’ grading: What is in the grade? Studies in Educational Evaluation, 59, 141–149.
Russell, J. A., & Austin, J. R. (2010). Assessment practices of secondary music teachers. Journal of Research in Music Education, 58(1), 37–54.
Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159–179.
Schunk, D. H. (2008). Metacognition, self-regulation, and self-regulated learning: Research recommendations. Educational Psychology Review, 20, 463–467.
Silva, E. (2009). Measuring skills for 21st-century learning. Phi Delta Kappan, 90(9), 630–634.
Singapore Ministry of Education. (2021). Character and citizenship education syllabus (CCE) – Secondary. https://www.moe.gov.sg/-/media/files/secondary/syllabuses/cce/2021-character-and-citizenship-education-syllabus-secondary.pdf
Smith, R., Ager, J., & Williams, D. (1992). Suppressor variables in multiple Regression/Correlation. Educational and Psychological Measurement, 52(1), 17–29.
Sparks, S. (2014). New character report cards rate students on ‘grit.’ Education Week. http://www.edweek.org/ew/articles/2014/06/05/34measuring-motivation-b4.h33.html
Stecher, B. M., & Hamilton, L. S. (2014). Measuring hard-to-measure student competencies: A research and development plan. RAND.
Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2), 245–251.
Stetser, M. C., & Stillwell, R. (2014). Public high school four-year on-time graduation rates and event dropout rates: School years 2010-11 and 2011-12. First Look. NCES 2014-391. National Center for Education Statistics.
Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4(1), 25–29.
Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140(4), 1174–1204.
Wang, M. T., & Fredricks, J. A. (2014). The reciprocal links between school engagement, youth problem behaviors, and school dropout during adolescence. Child Development, 85(2), 722–737.
Watson, D. L., & Emery, C. (2010). From rhetoric to reality: The problematic nature and assessment of children and young people’s social and emotional learning. British Educational Research Journal, 36(5), 767–786.
Wigelsworth, M., Humphrey, N., Lendrum, A. & Kalambouka, A. (2010). A review of key issues in the measurement of children’s social and emotional skills. Educational Psychology in Practice, 26(2), 173–186.
Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeider (Eds.), Handbook of self-regulation (pp. 531–566). Academic Press.
Woehr, D. J., & Huffcutt, A. I. (1994). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67, 189–205.
Wyatt-Smith, C., & Klenowski, V. (2013). Explicit, latent and meta-criteria: Types of criteria at play in professional judgement practice. Assessment in Education: Principles, Policy & Practice, 20(1), 35–52.
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3–17.
Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeider (Eds.), Handbook of self-regulation (pp. 13–39). Academic Press.
Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: A social cognitive career path. Educational Psychologist, 48(3), 135–147.
Zimmerman, B. J., & Kitsantas, A. (2014). Comparing students’ self-discipline and self-regulation measures and their prediction of academic achievement. Contemporary Educational Psychology, 39(2), 145–155.
Zimmerman, B. J., & Martinez-Pons, M. (1988). Construct validation of a strategy model of student self-regulated learning. Journal of Educational Psychology, 80(3), 284–290.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Canadian Society for the Study of Education
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The Canadian Journal of Education follows Creative Commons Licencing CC BY-NC-ND.
