EVALUATING HIGHER EDUCATION POLICY IN TURKEY : ASSESSMENT OF THE ADMISSION PROCEDURE TO ARCHITECTURE , PLANNING , AND ENGINEERING SCHOOLS

The admission procedure to higher education institutions in Turkey is based on the student’s high school grades and Central University Entrance Examination (CUEE) score, with a much greater weight on the latter. However, whether the CUEE is an appropriate measure in the admission process to universities is still a much-debated question. This study assesses the validity of the CUEE as a selection tool for design-based departments by examining the relationship between CUEE scores and success in university education in two design-based departments, architecture and city planning. The analysis is then extended to test the relationship in three engineering departments, computer engineering, civil engineering, and mechanical engineering. Based on the bivariate correlation and one sample t-test result, we report that CUEE scores and graduation grades have no relationship at all. We conclude that the current admission procedure to designbased schools based on solely a central examination score is not preferable.


Introduction
The admission procedure to higher education institutions in Turkey is based on the student' s high school grades and the Central University Entrance Examination (CUEE) score, with a much greater weight on the latter.The CUEE is administered by the Student Selection and Placement Center (OSYM) affiliated with the Council of Higher Education (YOK); a high school diploma is the only requirement to take the exam.There were 85 public universities offering programs at the undergraduate level and 30 semi-private ones in Turkey by the end of the 2007-2008 academic year.
Every year more than 1.5 million high school graduates take the exam, and less than one third of them have the chance to enroll in a 2-year or a 4-year program.The supply of higher education in Turkey has clearly failed to meet an increasing social demand since the early 1960s (Dundar and Lewis, 1999).Because of the high number of high school graduates and the limited capacity of the universities, the establishment of the Student Selection and Placement Center (OSYM) centralized the university examination in 1974 (YOK, 2007).The CUEE was the end result of an effort to give each student an equal opportunity in this fierce competition and was first administered within the same year.The exam was administered in two-tier form between the years 1981 and 1999, but this form was abandoned in 2000.Students' high school grades have been added to the CUEE scores since 1982.The number of students taking the test has increased from 466,963 in 1980 to 1,856,618 in 20051,856,618 in (YOK, 2007)), including both current high school graduates and those who were unable to enroll in a program in previous years.However, the system has brought forth some important problems.Kockar and Dincoz (2004) define CUEE as anxiety provoking and examine the sources of anxiety symptoms of students preparing for the university entrance exam.The exam is also a real economic burden.Almost all high school students and former high school students pay extra money for private classes and tutors to get higher scores and get the university education they have always dreamed of.According to the Turkish Education Society' s (TED) 2005 research report on the CUEE, over $8 billion was spent for CUEE preparation in 2004, including private classes, tutors, and test books, when 1,786,883 students took the test.That is roughly $5,000 per student, which is a significant amount for a country with $5,400 gross national income per capita in 2006.Dundar and Lewis (1999) state that the parents' income levels have a significant effect on the students' chances to obtain higher education.OSYM statistics reveal that students from higher income groups are more likely to get higher scores from centralized tests (OSYM, 1992).
Even if students get high scores on the CUEE, they may still experience mismatches between their interests and abilities and the university education (Aycan, 2001).Whether the CUEE is an appropriate measure in the admission process to universities is still a much-debated question.As Zwick (2002, p.79) correctly states: "The validity of admissions tests as a selection tool for higher education institutions is judged largely by the degree to which test scores can predict later scores."Students are admitted to design-based university programs, including architecture and city planning, on the same basis as they are to programs in general.This overriding worldwide practice based on centralized examination scores and pre-college education grades results in admitting students with strong verbal and quantitative skills to designbased programs.This practice, however, clearly ignores the student' s present level of cognitive skills required in design-based programs (Moore, 1970).
There is evidence that visual, spatial, and perceptual abilities are desired for success in design related education (Zwick, 2002).However, one can easily say that the CUEE has no intention to measure students' visual, spatial, and perceptual abilities.Rather, the test measures how well a student is able to learn the knowledge offered in the high school curriculum.There are more than 30 architecture departments and more than 10 planning departments offering programs at the undergraduate level.There are 50 students in each department on average.Thus, approximately 2,000 students are at risk of unexpected failure in design-related university education after years of preparation with enormous financial and psychological costs.This number exceeds 10,000 when other design-based university programs are considered, such as graphic design, landscape architecture, interior architecture, and industrial design.
This research aims to assess the suitability of the CUEE-based admission procedure to architecture and planning schools in Turkey.To do that, the relationship between students' CUEE success and their success in university education is examined in two parts.First, the rank in admission and the rank in graduation for a sample of 233 students studying in architecture and city planning departments are compared.A similar test is then performed using data pertaining to three engineering departments to see whether CUEE score might be a crucial predictor of students' success in non-design based engineering departments.The remainder of the paper is organized as follows.Data collection and manipulation are described in Section 2. Empirical analysis and results are presented in Section 3. Section 4 concludes the paper.

Literature Review
Research on predicting college grades using test scores has come a long way since the early studies relating psychological test scores and grades in various subjects at Columbia and Yale Universities in the late 1800s.According to Zwick (2002), more than 3,000 studies have been conducted focusing on the association between admission test scores and subsequent grades using SAT scores in the United States alone.Studies focused on different aspects including gender, ethnicity.and study major.McCornack and McLeod (1988) aim to determine whether gender bias exists in the prediction of college course grades using aptitude test scores and high school grades.They use high school grade point average and verbal and mathematics scores from the SATs for the 1985-86 academic year.They conclude that the predictive power of high school grades and standardized test scores differ for men and women.Fleming and Garcia (1998) use data for 1,774 freshmen and seniors studying in the United States in 1984.The results indicate minor differences for the validity coefficients for GPAs: 0.33 for black students in the predominantly white schools, 0.36 for black students in predominantly white schools, and 0.34 for white students overall.Pieronek et al. (2004) conclude that SAT scores and course grades do not accurately predict which students will remain in engineering, based on a survey of students enrolling in a required first-year engineering course at the University of Notre Dame.
Studies focusing on the relationship between standardized test scores and success in design-based university education are limited.Pitcher, Olsen, and Solomon (1962) use data from 433 architecture students at 11 schools of architecture in the United States.Their results show that high school grades are highly correlated with the first-year architecture grades, but its validity as a predictor of later success declines.The correlation coefficient decreases from 0.47 to 0.19.The Architecture School Aptitude Test (ASAT), a specialized centralized test first administered in 1964 and abandoned several years later, is found to be a better predictor of first-year and later grades than the College Entrance Examination: 0.47 and 0.30 respectively.The correlation coefficient increases to 0.54 from first-year grades and to 0.43 for later-year grades when these three measures are used together.Moore (1970) claims that the Architecture School Aptitude Test measures the applicant' s present level of knowledge or skill, but not his potential.He examines the available tests of creative problem solving for their potential as predictors of success in architecture using data from 94 architecture students at the University of California in 1967.Moore (1970) concludes that three creative problem-solving tests have the potential to be a very strong predictor of success in architecture education.These three tests are (1) Sketches, which measure the ability to generate visual patterns and images that confront to simple specifications; (2) Gestalt Transformations, which measure the ability to shift the functions of an object or part of an object to use it in a new way; and (3) Figure Analogies, which measure the ability to recognize figural relationships between different forms.Rende et al. (2006) examine the factors affecting success in university education using data from 2002-05 for engineering students at a private university in Turkey.Their results reveal that that CUEE rank is a significant predictor of success in engineering departments.However, it is hard to say that the results are valid for the design-related departments.Pre-university education and design-based university education in Turkey are clearly unparallel.This contradiction often results in the failure of successful high school graduates in designbased programs, such as architecture and planning.

Data Processing and Analyses
Two tests are conducted to answer whether the CUEE score is a good predictor of success in university educa-tion.In this section these two tests are described in detail and the results are discussed.

Sample
This study uses the archival data collected by the student affairs office at Dokuz Eylul University, Izmir, Turkey, was used in this study.Dokuz Eylul University is one of the largest public universities in Turkey.Located in the thirdlargest city in the country, the student profile shows great variety in various aspects, including birthplace and parents' income level.It is thus plausible to assume that the students at Dokuz Eylul University constitute a good representative population of the students studying at public universities in Turkey in general.The data included two types of information: ( 1  shows the distribution of students by department and by CUEE year including (1) the number of students enrolled each year, (2) the students eliminated from the sample because they did not graduate within the examined time period, and (3) the number of students in the sample.Table 2 (see page 8) shows the distribution of students by department and graduation year including (1) the number of students graduated each year, (2) the students eliminated from the sample because they enrolled before 1998, and (3) the number of students in the sample.

Procedure in Organizing Data
First, for each student, the rank in admission was calculated based on the CUEE scores for each year (1998)(1999)(2000)(2001)(2002).Then, an admission percentile was calculated by dividing the student' s admission rank by the total number of students who entered the university in that year and multiplying by 100.Second, for each student, the rank in graduation was calculated by sorting the GPAs from the lowest to the highest for each year (2003)(2004)(2005)(2006).Then, a graduation percentile was calculated by dividing the student' s graduation rank by the total number of students graduated in that year and multiplying by 100.

Empirical Analysis and Results
The relationship between admission ranks and graduation GPAs was analyzed through the comparison of admission percentiles and graduation percentiles using scatter plots, applying bivariate correlation and one sample t-test.Results showed no significant relation between the two measures.Figure 1 (see page 9) shows the distribution of students by their graduation percentile versus admission percentile for the whole sample.
Clearly, no visual pattern is observed.For example, students in higher admission percentiles are in varying graduation percentiles, not only high graduation percentiles.The case for students in lower admission percentiles is similar (Figure 1, page 9).Bivariate correlation analysis supports this conclusion and shows no significant effect of relation between admissions success scores and graduation success scores (Pearson' s correlation coefficient = 0.06, n = 233, p = 0.36).
When the data for architecture and planning students were analyzed separately, the results did not differ much.Figures 2a and 2b (see pages 9, 10) show no visual pattern between admission percentiles and graduation percentiles for architecture and planning students.Again, bivariate correlation analysis shows no significant effect of relation between admission percentiles and graduation percentiles for architecture students and planning students (architecture students: Pearson' s correlation coefficient = 0.13, n = 139, p = 0.12; planning students: Pearson' s correlation coefficient = -0.04,n = 94, p = 0.71).
Finally, whether the difference between admission ranks and graduation ranks differs from zero was tested with one sample t-test.The hypothesis here is: where n is the sample size, AP i is the admission percentile, and GP i is the graduation percentile for student i.The hypothesis can be worded as follows: The mean value for the absolute difference between students' graduation and admission ranks is zero.
For the whole sample (n = 232) the mean difference was 31.72 (21.93) and this difference significantly differs from zero (t statistics = 22.08, degrees of freedom = 232, p-value = 0.000).That is to say, there is a 31.72 percent difference between students' graduation and admission percentiles on average.The hypothesis is then rejected, indicating that there is a statistically significant gap between graduation and admission percentiles.
The results remain the same when data for architecture and planning students are analyzed separately.For architecture students (n = 139) the mean difference was 29.47 percent (21.71) and this difference significantly differs from zero (t-statistics = 16.00,degrees of freedom = 138, p value = 0.000).For planning students (n = 94) the mean difference was 35.05 percent (21.95) and this difference significantly differs from zero (t-statistics = 15.49,degrees of freedom = 93, p value = 0.000) as well.

Test 2
Test 1 is revised using data pertaining to engineering students to see whether the low level of correlation between CUEE scores and GPAs extends beyond the two designbased programs considered.

Sample
In parallel to Test 1, Test 2 used the archival data collected by the student affairs office at Dokuz Eylul University, Izmir, Turkey.The data included students' (1) CUEE scores for the years 2000-2003 and (2) GPAs at graduation for the years 2004-2007 2 .The students were from three engineering departments: computer engineering, civil engineering, and mechanical engineering.The majority of classes in each department are based on analytical skills.
The data was initially collected for 992 students: 241 computer engineering students, 366 civil engineering students, and 385 mechanical engineering students.
Among the 241 computer engineering students, 199 students entered the department between the years 2000 and 2003 (42 students entered the university before 2000), and 160 students graduated between the years 2004 and 2007 (81 students are still studying).Among the 366 civil engineering students, 314 students entered the department between the years 2000 and 2003 (52 students entered the university before 2000), and 210 students graduated between the years 2004 and 2007 (156 students are still studying).Finally, among the 385 mechanical engineering students, 320 students entered the department between the years 2000 and 2003 (65 students entered the university before 2000), and 240 students graduated between the years 2004 and 2007 (145 students are still studying).The data for students who entered the university before 2000 or who are still studying were eliminated from the sample.Thus, the analyses were based on a sample of 377 students; 100 computer engineering students, 126 civil engineering students, and 151 mechanical engineering students.Table 3 (see page 10) shows the distribution of students by department and by CUEE year including (1) the number of students enrolled each year, (2) the students eliminated from the sample because they did not graduate within the examined time period, and (3) the number of students in the sample.Table 4 (see page 11) shows the distribution of students by department and by graduation year including (1) the number of students graduated each year, (2) the students eliminated from the sample because they enrolled before 2000, and (3) the number of students in the sample.

Empirical Analysis and Results
For each student an admission percentile and a graduation percentile were calculated, as described in Test 1.In parallel to Test 1, scatter plots, bivariate correlation coefficients, and one sample t-test show no significant relationship between the two measures.
Figure 3 shows no visual pattern in the distribution of students by their graduation percentile versus admission percentile for the whole sample.Students in higher admission percentiles are in varying graduation percentiles, not only in high graduation percentiles.The case for students in lower admission percentiles is similar.Bivariate correlation analysis supported this conclusion and showed no significant effect of relation between two variables (Pearson' s correlation coefficient = 0.03, n = 377, p = 0.55).
When the data for computer engineering, civil engineering and mechanical engineering students were analyzed separately, the results did not differ much.Figures 4a, 4b, and 4c (see pages 12 & 13) show no visual pattern when the admission percentiles and graduation percentiles are considered.Again, bivariate correlation analysis showed no significant effect of relation between admission percentiles and graduation percentiles (computer engineering students: Pearson' s correlation coefficient = 0.03, n = 100, p = 0.75; civil engineering students: Pearson' s correlation coefficient = 0.02, n = 126, p = 0.82 and; mechanical engineering students: Pearson' s correlation coefficient = 0.04, n = 151, p = 0.65).
Whether the difference between admission ranks and graduation ranks differs from zero was tested with one sample t-test.For the whole sample (n = 377) the mean difference was 32.04 (23.86) and this significantly differs from zero (t statistics = 26.07,degrees of freedom = 376, p-value = 0.000).There is a 32.04 percent difference between students' graduation and admission percentiles on average.The results remained the same when the data was analyzed separately for each department.For computer engineering students (n = 100) the mean difference was 31.03(23.28) and this difference significantly differed from zero (t-statistics = 13.33,degrees of freedom = 99, p value = 0.000).For civil engineering students (n = 126) the mean difference was 33.01 (24.42) and this difference significantly differed from zero (t-statistics = 15.17,degrees of freedom = 125, p value = 0.000).For mechanical engineering students (n = 151) the mean difference was 31.89(23.90) and this difference significantly differed from zero (t-statistics = 16.39,degrees of freedom = 150, p value = 0.000).

Overview of Findings
The validity of a centrally administered test as a selection tool is judged largely by the degree to which test scores can predict later scores (Zwick, 2002).The validity of CUEE can then be well assessed by comparing the test scores and the later grades.This study compared the CUEE scores and GPAs for a sample of architecture and city planning students who studied at Dokuz Eylul University between the years 1998 and 2006, and extended the analysis to three engineering departments: computer engineering, civil engineering, and mechanical engineering.
Based on the bivariate correlation and one sample ttest results, we can clearly state that there is no relation between the students' CUEE success and their success in university education when the students who were admitted to the university are considered.On average, there is a 31.72 percent difference between students' graduation and admission percentiles for students enrolled in these two design-based programs.This rate is 29.47 percent for architecture students and 35.05 percent for city planning students, indicating that CUEE is less relevant as a criterion in admission procedure for planning students than for architecture students.These results are also valid for the three engineering departments considered.There is a 32.04 percent difference between students' graduation and admission percentiles on average, and these results remains the same when the data is analyzed separately for each engineering department.

Policy Implications
Pre-college education in Turkey is reported to be far from preparing students to a design-based college education, but high school students today know more about college education through the use of the Internet.Potur and Barkul (2007) survey 98 freshmen in an architecture program in Istanbul, Turkey at the time of enrollment.Their findings show that freshmen in design-based departments are aware of the fact that creative thinking is a key component in design education.However, the student selection procedure to these departments is in question because the student' s high school grades and CUEE score are the two determinants.
The results of the current study have shown that CUEE is not an appropriate way of selecting students for design-based education.But what is the correct way-or is there a correct way-to select students for designbased higher education?Goldschmidt et al. (2001) review the criteria for admission into schools of architecture in 21 countries.In 70 schools of architecture surveyed, the admission criteria include: high school records, scores in general scholastic ability tests, results of special aptitude tests for architecture, portfolios, interviews, letters of recommendation, statements of intent, and written essays.As clear in Goldschmidt et al. (2001), design-based schools have always been selective as to whom they admit.Historically, mostly technical presentation skills were an overriding measure of adequacy.Today, creativity and reasoning power play a much more important role in determining who will be allowed to acquire architectural education (Goldschmidt et al., 2001).However, in countries where a centrally adminis-tered test is the only criteria, failure (in varying rates) is guaranteed.
The admission procedure to design-based programs should undoubtedly include tests based on students' present levels of cognitive skills and their potential to develop these skills.Tests aiming to measure the level of creative problem solving abilities may also be promising because design is the process of developing functional solutions to well-defined problems.It is apparent that different design-based programs require different cognitive and creative skills.Clearly, developing a standard test for student selection to design-based programs is a long way off, and will by no means be a smooth ride.We conclude that the CUEE is not an appropriate way of selecting students for design-based programs, and developing a standard test for design-based programs is clearly beyond the scope of this paper.

Caveats
Note that results obtained from this study can hardly be generalized because the data utilized pertains to a single school of architecture for a given time period.Also note that results are valid for two design-based departments, architecture and city planning.Results for other designbased departments, such as graphic design, textile design, or interior architecture and for other non-design departments, may vary.Also, as Rothstein (2004) suggests, other variables may affect a student' s success in university, such as students' demographic information and background.The parents' income levels have a significant effect on the student' s chances to obtain higher education as well (Dundar and Lewis, 1999).Other factors that may be related to educational success include gender, ethnicity, and study major.None of these variables are controlled in the analyses.The results should then be referred to cautiously.
A second issue is about the samples used in the analyses.Following past empirical research, including Pitcher, Olsen, and Solomon (1962), Moore (1970), Fleming andGarcia (1998), andPieronek et al. (2004), we used data for students who were admitted to the university.Extending the tests to the students who would like to be admitted but left out would be a wonderful extension to the present analyses.However, obtaining such a dataset is practically impossible.Nonetheless, the present results confirm that the current admission procedure to design-based schools based on solely a central examination score is not preferable.The debate for selection criteria is yet to start.

Notes
1.An earlier version of this paper was presented at the DesignTrain Congress: Guidance in/for Design, May 10-12, 2007, Amsterdam, The Netherlands.
2. The years studied in Test 2 are different than the years studied in Test 1 because of the availability of the data.For the engineering departments at Dokuz Eylul University the CUEE scores were not available for the years earlier than 2000.Considering the fact that a student can graduate in a minimum of four years, the GPA scores were collected for years after 2004.
) Central University Entrance Examination (CUEE) scores for students who entered the university between the years 1998 and 2002, and (2) grade point averages (GPAs) at graduation for students who graduated between the years 2003 and 2006.Data used in Test 1 pertains to students from two design-based departments, city planning and architecture.The data was initially collected for 509 students: 323 architecture students and 186 planning students.Among the 323 architecture students, 274 students entered the department between the years 1998 and 2002 (49 students entered the university before 1998), and 188 students graduated between the years 2003 and 2006 (135 students are still studying or graduated before 2003).Among the 186 planning students, 175 students entered the department between the years 1998 and 2002 (11 students entered the university before 1998), and 105 students graduated between the years 2003 and 2006 (81 students are still studying or graduated before 2003).The data of students who entered the university before 1998 or who are still studying in the department were eliminated from the sample.Thus, the analyses are based on a sample of 233 students (139 architecture students and 94 planning students) who entered the university between the years 1998 and 2002 and graduated between the years 2003 and 2006.

Figure 1 .
Figure 1.Scatter plot for admission percentiles versus graduation percentiles for all students in design-based departments (n = 233)

Table 1 .
The distribution of students by department and CUEE year for architecture and city planning departments

Table 2 .
The distribution of students by department and graduation year

Table 4 .
The distribution of engineering students by department and graduation year