Topic: Integration of Dees chapters 1–2 with the Module/Week 4 (Satterlee Chs. 7 & 8) and Module/Week 5 Reading & Study material and the student's research. Discuss the 3 most important concepts

Thinking Skills and Creativity 21 (2016) 41–49 Contents lists available at ScienceDirect Thinking Skills and Creativity j ourna l h o mepa ge: h t tp://www.elsevier.com/locate/tsc How might language affect critical thinking performance? Emmanuel Manalo a,∗, Chris Sheppard b aGraduate School of Education, Kyoto University, Kyoto, Japan bCenter for English Language Education, Faculty of Science and Engineering, Waseda University, Tokyo, Japan a r t i c l e i n f o Article history:

Received 27 July 2015 Received in revised form 29 March 2016 Accepted 11 May 2016 Available online 12 May 2016 Keywords:

Critical thinking skills instruction Critical evaluation Cognitive cost Language proﬁciency Language structure a b s t r a c t This study examined whether language structure or language proﬁciency might affect students’ critical thinking performance. Previous research has claimed that many non- Western students struggle with the demands of demonstrating critical thought. Two language-related causes have been suggested: one concerning structural limitations in the non-Western students’ ﬁrst language, and the other concerning their second language pro- ﬁciency. In Study 1 described here, reports written by 110 Japanese second year university students, who had received instruction in academic discourse for critical evaluation (which is one aspect of critical thinking), were analyzed for use of evaluative statements. No dis- advantage was found for use of the Japanese language, which is considered as having a more indirect structure that may make critical evaluation more difﬁcult. Measurements of language proﬁciency in English and Japanese, however, were found to correlate with pro- duction of evaluative statements in those respective languages suggesting that language proﬁciency could affect critical evaluation use. In Study 2, the same task was given to 43 ﬁrst year students who had not yet received the same instruction. Analysis revealed similar patterns in their written work but at a lower level, suggesting that the second year students had beneﬁtted from the skills instruction. Furthermore, unlike the second year students, the ﬁrst year students evidenced no correlations between their language proﬁciency scores and their production of evaluative statements, suggesting that proﬁciency on its own is inadequate: students need instruction on the speciﬁc language forms and structures to use to demonstrate critical thinking in their written work.

© 2016 Elsevier Ltd. All rights reserved. 1. Introduction In modern societies, the ability to evaluate the credibility of information that one encounters is an essential skill, especially considering the proliferation of unvetted information through the Internet and other forms of mass media (e.g., Glassner, Weinstock, & Neuman, 2005 ; Thomm & Bromme, 2011 ). However, educational development of critical thinking skills, which includes critical evaluation, is not a straightforward matter. There are, for example, disagreements about appropriate meth- ods for critical thinking skills instruction and assessment, and various individual and situational factors are believed to inﬂuence the development and use of such skills (e.g., Davies, 2006; Halpern, 1998; Ku, 2009 ; Manalo, Kusumi, Koyasu, Michita, & Tanaka, 2013 , 2015 ; Ten Dam & Volman, 2004 ). One of the main areas of contention is whether cultural background contributes to differences in critical thinking performance (e.g., Ennis, 1998; Manalo et al., 2013, 2015 ). Some authors, for example, have portrayed Asian students as being deﬁcient in critical thinking compared to Western students (e.g., Atkinson, ∗ Corresponding author at: Graduate School of Education, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan.

E-mail address: [email protected] (E. Manalo). http://dx.doi.org/10.1016/j.tsc.2016.05.0051871-1871/© 2016 Elsevier Ltd. All rights reserved. 42 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 1997; Fox, 1994 ), and it has been found that many instructors at the tertiary level subscribe to such a view (e.g., Lee & Carrasquillo, 2006 ; Robertson, Line, Jones, & Thomas, 2000 ).

Two language-related explanations have been proposed for the apparent differences in critical thinking performance manifested by students from different cultural backgrounds. One of those explanations concerns the structure of the student’s native language (L1). This explanation posits that, due to their structure, some languages may present constraints in the ease with which certain thinking skills can be undertaken or expressed. This explanation is sometimes referred to as the “Sapir- Whorf hypothesis” or the “linguistic relativity hypothesis” (see, e.g., Au, 1983; Hill & Mannheim, 1992; Hockett, 1954; Whorf, 1956 ). An example of a claim of this kind is Bloom’s (1981) proposal that counterfactual thinking (i.e., thinking about what might have been, contrary to facts) may be more difﬁcult in Chinese compared to English.

Some observations of linguistic differences, such as “indirectness” being a feature more prevalent in some languages, particularly Asian languages (e.g., Kong, 2005 ), would appear to support the idea that language structure could affect the ease with which certain modes of thinking could be undertaken or expressed. In a study by Itakura and Tsui (2011) , for example, evidence was found that book reviewers use different strategies to convey critical evaluation when writing in Japanese compared to English: in Japanese, criticism is usually indirectly conveyed and is frequently preceded by an apology. Previous studies, however, have failed to provide convincing evidence that language structures could actually impose constraints in what users of the language can communicate. Although the earlier-mentioned study by Bloom (1981) claimed to have found evidence for such constraints where counterfactual thinking in the Chinese language is concerned, subsequent investigations failed to replicate or support Bloom’s results (Au, 1983; Liu, 1985; Takano, 1989; Vorster & Schuring, 1989 ). Even where the weaker version of the linguistic relativity hypothesis is concerned (i.e., that language can merely inﬂuence people’s thought patterns, rather than constrain their thought processes), the ﬁndings of the few studies that claim to have found supporting evidence (e.g., Boroditsky, 2001 ) have not been replicated in subsequent independent investigations (e.g., Chen, 2007; January & Kako, 2007; Tse & Altarriba, 2008 ).

The other language-related explanation for the apparent culture-based differences in critical thinking performance con- cerns students’ second language proﬁciency (e.g., Floyd, 2011 ; Lun, Fischer, & Ward, 2010 ; Paton, 2005 ). This explanation is based on the observation that the students who are usually reported as having lower levels of critical thinking competence are international students in English speaking countries. It suggests that, because most of those students have to use a sec- ond language (L2, which is usually English) in their host environment, and they may lack adequate proﬁciency in that L2, they would likely manifest lower competence when performing tasks that require use of that L2. Tasks that are likely to get affected include cognitive tasks like critical thinking.

One way of understanding this possible inﬂuence of language proﬁciency on critical thinking performance is in terms of cognitive cost (i.e., the mental resources cost associated with task performance). Language processing entails the use of cognitive resources in working memory (Baddeley, 1986, 1998 ), and lower proﬁciency in a language would require the use of more resources. Thinking critically would likewise require the use of working memory resources. There are, however, limited resources available in working memory (Baddeley, 1986, 1998 ) and, if a considerable amount of those resources has already been expended on utilizing a language in which proﬁciency is low, there may not be adequate resources remaining for the satisfactory execution of critical thinking.

The negative impact of the higher cognitive cost entailed in using a language in which proﬁciency is low, on the execution of other cognitive tasks, has been demonstrated in previous research. Takano and Noda (1993, 1995) showed that the use of a foreign language detrimentally affects performance in concurrently undertaken non-linguistic tasks like arithmetic calculation and mental imagery, and Manalo and Uesaka (2012, 2014) reported evidence indicating that students’ lower proﬁciency in an L2 limits their ability to use diagrams when explaining information in that L2. Where critical thinking is concerned, both Lun et al. (2010) and Floyd (2011) reported indications that lower proﬁciency levels in English could detrimentally affect Asian students’ performance in critical thinking tests administered in English. However, neither of those studies used appropriate, objective measures of L2 proﬁciency to reliably conﬁrm the connection between L2 proﬁciency and critical thinking skills performance. 1.1. Overview of the present study The present study examined whether there might be evidence to support either or both (i) the language structure expla- nation, and/or (ii) the language proﬁciency explanation, in students’ manifestation of critical thinking in their written work.

The study was not intended to be a comprehensive test of the language structure hypothesis: it examined only whether, in the written work of Japanese university students, there might be observable differences in the presence of critical thinking qualities depending on the language used, Japanese or English. Critical thinking was operationalized as students’ use of evaluative statements. This decision was based on the fact that use of evaluative statements comprises a salient expression of critical evaluation, which in turn is central to the notion of critical thinking application (e.g., Fisher & Scriven’s, 1997, p.

21 , deﬁnition of critical thinking as “skilled and active interpretation and evaluation of observations and communications, information and argumentation” – italics added).

In the present study, Japanese was deemed an appropriate language to examine because, like a number of other Asian languages, it employs patterns of expression that make it more indirect and inductive compared to English (e.g., Itakura & Tsui, 2011; Scollon & Wong-Scollon, 1991 ). Evaluation, however, requires precision and directness in conveying judgments about the quality or value of the subject being referred to. Thus, structural features of the Japanese language could make E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 43 the production of evaluative language relatively more difﬁcult. If so, it should be possible to detect lower rates of evaluative language use in the students’ written work in Japanese compared to English.

It was equally important to consider whether using an L2 might detrimentally affect students’ critical evaluation perfor- mance. If this was the case, one would expect that critical evaluation performance in the L1 would exceed that in the L2.

One would also expect to ﬁnd a relationship between the critical evaluation performance and L2 proﬁciency. Thus, possible relationships between students’ L2 proﬁciency and their production of evaluative statements were investigated. The ques- tion was whether L2 proﬁciency would manifest as a limiting factor because lower proﬁciency entails higher cognitive cost when using the L2, leaving insufﬁcient resources in working memory for critical evaluation. If this explanation is supported, a relationship should be found between the students’ L2 proﬁciency and their evaluative statements production in the L2, but not in the L1. A relationship in both the L1 and the L2 would suggest that general language or intellectual abilities – rather than L2 proﬁciency – affect critical evaluation performance. The reason is that language abilities, and intellectual abilities and performance, are generally considered as being related (e.g., Ackerman, 1986; Neisser et al., 1996 ). Thus, a student with higher language and intellectual abilities could be expected to score higher in measurements of L2 proﬁciency, and evidence better performance in tasks like critical evaluation – in both their L1 and L2.

Furthermore, it is conceivable that this kind of relationship between language proﬁciency and critical evaluation also exists in the L1. In other words, that the more proﬁcient or skilled a student might be in using his or her native language, the less cognitive resources such use might demand, and the more resources might be made available for critical evaluation – which could then be better executed. Thus, in the present study, the possible existence of this relationship in the L1 was also investigated.

When considering the development of critical evaluation skills through instruction, if the language structure explanation holds, one would expect that skills instruction provided in the L2 would not result in any discernible skills improvement in the L1 (i.e., since the L1 has inherent structural constraints on the use or expression of those skills). In contrast, if the language proﬁciency explanation holds, one could expect that skills instruction provided in the L2 – where proﬁciency is lower – would result in at least some discernable transfer to the L1, where proﬁciency is higher and more cognitive processing resources are likely to be available for the application of those skills. Critical evaluation skills are thinking skills and its acquisition should not be bound to the language of instruction. The important question is whether constraints or limitations in the language to use could detrimentally affect critical evaluation performance.

The research reported here comprised two related studies. In Study 1, evaluative statements that second year Japanese university students produced in Japanese (their L1) and in English (their L2) were examined. These students had received instruction on academic discourse. Thus, they were not naïve as to the requirements of expressing evaluative language, and any differences in the writing they produced in L1 and L2 could be attributed to either the inherent structure of the language they were using or their proﬁciency in using that language. In Study 2, the same writing task was given to ﬁrst year students who had received little instruction on academic discourse, and nothing explicit on the production of evaluative language. The purpose of this second study was to ﬁnd out if the characteristics of L1 and L2 written work produced by the ﬁrst year students, compared to their second year counterparts, differed. Hence, the aim was to ﬁnd out whether the additional instruction that had been received by the more advanced second year students might have made a difference. 2. Study 1 Study 1 tested two hypotheses. The ﬁrst was that students’ production of evaluative statements in Japanese and in English would differ. The second was that the students’ proﬁciency in a particular language would be related to the amount of evaluative statements they produce in that language. 2.1. Method 2.1.1. Participants The participants were 110 Japanese university students in their second year of study in science and engineering disciplines.

For these students, Japanese is L1 and English is L2. They were taking a compulsory English communication skills development course that covers oral and written academic discourse using a task-based learning approach. The students came from four different classes in that course.

2.1.2. Materials and procedure As part of the communication skills course, the students were provided class instruction, textbook explanations and examples (Anthony, Rose, & Sheppard, 2010 ), and practice in the use of language appropriate for critical evaluation, including ranking and debating different reasons and other forms of alternatives (e.g., clearly stating the premises, and then drawing conclusions). These materials and the instruction were all provided in English.

For the purposes of the present investigation, the students were additionally provided with a single page Japanese translation of the part of the textbook dealing with how to make valid arguments. They were also supplied brief (one page) written examples (one in English and one in Japanese) of how alternative reasons could be ranked according to judgments about their relative importance. The example texts conveyed someone’s opinion about the most important reason for learning the English language, among four possible reasons. The texts provided examples of evaluative statements and provision of 44 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 support for claims, although those were not labeled or overtly identiﬁed in any way in the texts. The equivalence and appropriate use of language in the English and Japanese versions were checked by several bilingual teachers of the course.

Although all materials provided in the course are usually in English, the Japanese versions were supplied in this case to avoid possible disadvantage to the students’ production of evaluative language in Japanese (i.e., without the Japanese versions, it could be argued that the students might have simply been unfamiliar with the equivalent Japanese expressions for critical evaluation).

During two consecutive 90-min class sessions of the course, the students were introduced to the Titanic and Space Shuttle Challenger disasters, including four basic causes that have been proposed for the occurrence of each of those disasters. During the class sessions, the students participated in guided exercises to explore and discuss the disasters and their corresponding possible causes.

For homework, the students were asked to write two brief reports to explain what they considered to be the most important cause of each of the disasters. To avoid any possible misunderstandings about the requirements of the homework task, written instructions were provided in Japanese. The students were randomly assigned to write one report in English and the other in Japanese (i.e., if they were asked to write the Titanic report in English, they had to write the Challenger report in Japanese, and vice versa).

2.1.3. Analyses The total number of sentences [Total], and the number of evaluative sentences (i.e., sentences where some evaluation of the relative value of the topic is made) [Evaluative], were counted and scored in the analysis of the students’ written work.

Operational criteria were drawn up for determining what data counted as “evaluative sentences”. Examples of such requirements included: the sentence must explicitly say something about the worth or value of the subject, and that worth or value must be in comparison to something else. Conditional statements that explicitly convey a relative evaluation of the subject were counted. The following examples, in contrast, did not count: the use of simple adjectives or adverbs to describe something, prescriptive statements not explicitly expressing a relative evaluation or judgment, and conditional statements in general. Inter-rater reliability was checked by asking an independent coder to score a randomly selected sample of 25% of the data. Reliability coefﬁcients obtained (Cronbach’s alphas) were deemed to be satisfactory: they were 0.922 in English, and 0.960 in Japanese.

Language proﬁciency was measured (i) by using the students’ TOEIC-IP (Test of English for International Communication – Institution Program) scores, and (ii) in terms of the complexity of the sentences the students produced in English and in Japanese (explained below). TOEIC is a norm-referenced test of English listening comprehension and reading skills, widely used as a measure of students’ English language proﬁciency levels in Japan (http://www.ets.org/toeic ). Although there is a speaking and writing test version, it is not as widely used as the listening and reading test version, which is what the TOEIC-IP contains. Students in the faculty where the students came from are required to sit the TOEIC-IP test at regular intervals during their period of enrolment, and their scores on that test are made available to their teachers. The TOEIC test has been used in numerous previous studies as a measure of L2 proﬁciency (e.g., Dornyei & Katona, 1992 ; Eggly, Musial, & Smulowitz, 1999 ), and various studies have reported that it has at least adequate reliability and validity (e.g., In’nami & Koizumi, 2011 ; Perkins, 1987 ; Powers, Kim, & Weng, 2008 ). The TOEIC is also considered a good measure of non-native English speakers’ ability to use English in real life situations rather than just in strictly academic contexts (Eggly et al., 1999 ).

For these reasons, the authors of the present study decided to use the students’ TOEIC-IP scores as one measurement of their English proﬁciency.

The other measure of proﬁciency used – complexity – was selected because, along with accuracy and ﬂuency, it is generally considered as one of the key dimensions of proﬁciency in language production – both oral and written (e.g., Ellis, 2009; Ishikawa, 1995; Ortega, 2003; Skehan & Foster, 1999 ; Wolfe-Quintero, Inagaki, & Kim, 1998 ). However, unlike accuracy and ﬂuency, for which it is difﬁcult to come up with meaningful and equivalent/comparable measurements for English and Japanese, the authors of the present study were able to determine a viable way to measure complexity.

Usually, analysis of production complexity in English entails calculating the ratio of words over sentences or terminable units (T-units; e.g., Ishikawa, 1995; Ortega, 2003 ; Richards, Platt, & Platt, 1992 ). However, what counts as a word in Japanese is not exactly the same as in English, and thus, such a method for calculating complexity was deemed inap- propriate. Instead, the number of verbs that the students used were counted and divided by their total number of sentences to generate a measure of complexity. Verbs were chosen as they are comparable in English and Japanese, and the use of more verbs can be considered a good indicator of more complex sentence structures (i.e., simple sen- tences usually contain only one verb, and inclusion of more verbs within one sentence usually requires the use of a more complex structure). It should also be noted that even though this measurement of complexity and the measure- ment of the students’ use of critical evaluation were derived from the same source (i.e., the writing that the students produced), these measurements were not dependent on each other. In other words, students can produce complex writing without being critically evaluative, and vice versa: they can write evaluative sentences that are low in complex- ity.

Analyses of variance were conducted to compare the students’ production of sentences (Total, and Evaluative) in English and in Japanese. Correlational analyses were carried out to examine possible relationships with the students’ most recent TOEIC test scores and their complexity scores in English and in Japanese. E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 45 Table 1 Means (with standard deviations in brackets) of total and evaluative sentences that the second year students produced in English and in Japanese. Total Evaluative English 20.31 (5.59) 3.46 (1.76) Japanese 18.68 (5.92) 3.75 (1.80) Table 2 Correlation coefﬁcients between second year students’ TOEIC and writing complexity scores, and numbers and proportions of evaluative sentences they produced, according to the language used (effect sizes, R2shown in brackets). Scores English Evaluative Sentences Japanese Evaluative Sentences Number Proportion Number Proportion TOEIC 0.23 *(0.053) 0.14 (0.020) 0.18 (0.032) −0.06 (0.004) English Complexity 0.19 *(0.036) 0.24 *(0.058) 0.15 (0.023) 0.17 (0.029) Japanese Complexity 0.02 (0.000) 0.09 (0.008) −0.07 (0.005) 0.33 ***(0.109) *p < 0.05. **p < 0.01. ***p < 0.001. 2.2. Results Table 1 shows the means, and standard deviations (in brackets), obtained for Total and Evaluative categories of sentences in the students’ written work in English and in Japanese.

Repeated measures 2 × 2 ANOVAs demonstrated that there were no signiﬁcant effects due to the task (i.e., the Titanic compared to the Challenger reports), hence demonstrating task equivalence. The analysis however revealed signiﬁcant effects due to language in both the total number of sentences written [Total], F(1, 108) = 11.47, p = 0.001, p2= 0.020; and the number of evaluative sentences [Evaluative], F(1, 108) = 4.81, p = 0.030, p2= 0.006.

These results indicate that the students wrote signiﬁcantly more sentences in English compared to Japanese, but produced more of the target evaluative language in Japanese. The ANOVA results also revealed that the students wrote more verbs in Japanese (mean = 48.52, SD = 14.01) than in English (mean = 38.77, SD = 12.54), F(1, 109) = 84.56, p < 0.001, p2= 0.119. The language produced was also signiﬁcantly more complex in Japanese as measured by the number of verbs per sentence (F(1, 109) = 100.8, p < 0.001, p2= 0.269), indicating that the reason for the production of fewer sentences in Japanese was because the students’ greater proﬁciency in that language enabled them to produce more complex sentences.

Because the total number of sentences that the students produced differed considerably between the two languages, the proportions of evaluative sentences (i.e., Evaluative as a proportion of Total) were also calculated and compared according to the language used. The comparison revealed a signiﬁcant difference: F(1, 109) = 20.32, p < 0.001, p2= 0.039. This result indicates that the proportion of evaluative sentences was higher in the reports that the students wrote in Japanese compared to those they wrote in English.

For the correlational analysis, the correlations between the students’ TOEIC scores and their writing complexity scores in English and in Japanese, and the evaluative sentences they produced in the respective languages (actual and proportions of total), were examined. The results are shown in Table 2. The students’ TOEIC scores correlated signiﬁcantly with the num- ber of evaluative sentences they produced in English. The correlation between the students’ TOEIC scores and the number of evaluative sentences they produced in Japanese was not signiﬁcant. Where the complexity scores were concerned, the complexity of the students’ writing in English signiﬁcantly correlated with both the number and proportion of evaluative sentences they produced in English. The complexity of their writing in Japanese signiﬁcantly correlated with the proportion of evaluative sentences they produced in Japanese. All cross language correlations between writing complexity and num- bers/proportions of evaluative sentences were not signiﬁcant. Taken together, these results lend support to the argument that language proﬁciency inﬂuences students’ ability to evidence critical evaluation in their writing.

A signiﬁcant correlation was also found between the proportions of evaluative sentences the students produced in English and in Japanese, r = 0.72, p < 0.001, R2(effect size) = 0.518. This result suggests that students who were more critically evaluative in one language also demonstrated greater critical evaluation in the other language. 2.3. Discussion Differences were found in both actual numbers and proportions of evaluative sentences that the students produced in English and in Japanese. The direction of the differences, however, was opposite to the language structure-based prediction:

higher proportions of evaluative sentences were found in Japanese instead of English. This result suggests that the students were better at producing evaluative language in their L1.

The signiﬁcant relationship found between the students’ language proﬁciency and their critical evaluation performance lends support to the notion that language proﬁciency could explain apparent cultural differences in students’ critical think- ing performance. When using an L2 (such as English) in which they may lack adequate proﬁciency (e.g., less automated 46 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 processing, inadequate linguistic knowledge), Asian students in Western countries may not be able to demonstrate critical thinking to the same degree as their native English-speaker counterparts because of the higher cognitive processing resource requirements of that L2, which depletes resources available in working memory for critical thinking. The important ﬁnd- ing in the present study is that proﬁciency in both languages (English and Japanese) signiﬁcantly correlated with critical evaluation performance in the respective language.

There still remains the possibility that the signiﬁcant correlations between the two languages in the proportions of the target language produced, are a result of other factors, which are correlated to both ﬁrst and second language proﬁciency, and which might affect the production of evaluative language – irrespective of the language being used. In other words, in addition to language proﬁciency, people’s critical evaluation performance may also be inﬂuenced by some other individual factors – such as general intelligence, working memory capacity, or communication skills – that are independent of the language being used for expression. It would be useful to identify and examine those other factors in future research.

As noted earlier, the student participants in this ﬁrst study had already received instruction in academic discourse that included the use of evaluative language. Therefore, an important next question to address was, “To what extent had that instruction affected the relative production of evaluative language in English and in Japanese?” – which was pursued in Study 2. 3. Study 2 Study 2 tested three hypotheses. The ﬁrst was that ﬁrst year students would manifest lower use of the target evalua- tive language compared to the second year students from whom data were gathered in Study 1. The second was that the differences in evaluative language use would be consistent across English and Japanese. In other words, the prediction was that, like the second year students, the ﬁrst year students would evidence greater use of evaluative statements in their L1 (Japanese) compared to their L2 (English). The third hypothesis was that the ﬁrst year students’ proﬁciency in both languages would not be related to the proportions of evaluative language they produce. This hypothesis was based on the assumption that, without receiving instruction about how to undertake and convey critical evaluation, students’ language proﬁciency would not make a difference to their production of the target evaluative language as they would have very limited knowledge about the structures necessary for producing the required language. 3.1. Method 3.1.1. Participants Forty-three Japanese university students who were in their ﬁrst year of studies in the same science and engineering faculty as the students in Study 1 were added to the participants of Study 1, for a total of 153 participants. The additional students came from two classes of a compulsory ﬁrst year English communication skills course which deals with various aspects of oral and written academic discourse, but nothing explicit about evaluative language (which is not covered until the second year course).

3.1.2. Materials, procedure, and analysis For one of their homework assignments, the students were given brief reading materials (in English and in Japanese) about the Titanic and Space Shuttle Challenger disasters, including the proposed causes of those disasters. These materials were drawn from the textbook used in the second year course. The Japanese translations were provided to these ﬁrst year participants to ensure that their subsequent writing performance would not have been compromised by possible difﬁculties in understanding the English versions. The contents of those materials were not covered in class.

The homework task that the students had to do was the same as that given to the second year students: to produce two brief reports to explain what they considered to be the most important cause of each of the disasters, after reading the materials provided. Like the second year students, they were randomly assigned to write one report in English and the other in Japanese. Also, like the second year students, they were provided with the one-page examples (one in English and one in Japanese) of how alternative reasons (for learning the English language) could be ranked according to judgments about their relative importance. The crucial difference was that the ﬁrst year students were not provided class instruction and exercises on the use of academic discourse speciﬁcally pertaining to evaluative language.

The written reports that the students produced were analyzed and scored in the same manner described in the ﬁrst study.

The ﬁrst and second year students’ data were then compared using repeated measures ANOVAs. Correlational analyses were carried out to examine possible relationships between the students’ language proﬁciency scores and the proportions of evaluative language they produced. 3.2. Results Table 3 shows the means, and standard deviations (in brackets), obtained for the Total and Evaluative categories of sentences in the ﬁrst year students’ written work in English and in Japanese.

Analyses of variance revealed signiﬁcant effects due to year of enrolment (ﬁrst year compared to second year) in the stu- dents’ scores for the total number of sentences [Total], F(1, 152) = 22.90, p < 0.001, p2= 0.113; and the number of evaluative E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 47 Table 3 Means (with standard deviations in brackets) of total and evaluative sentences that the ﬁrst year students produced in English and in Japanese. Total Evaluative English 15.70 (8.20) 2.09 (1.71) Japanese 12.84 (6.28) 2.26 (1.24) Table 4 Correlation coefﬁcients between ﬁrst year students’ TOEIC and writing complexity scores, and proportions of evaluative sentences they produced, according to the language used (effect sizes, R2shown in brackets). Scores Proportions of Evaluative Sentences English Japanese TOEIC 0.03 (0.000) −0.02 (0.000) English Complexity 0.16 (0.026)−0.18 (0.032) Japanese Complexity 0.01 (0.000) 0.14 (0.020) sentences [Evaluative], F(1, 152) = 27.20, p < 0.001, p2= 0.130. A signiﬁcant language effects were also found for Total, F(1, 152) = 26.67, p < 0.001, p2= 0.026. No signiﬁcant interaction effects between language and year were found; nor were any signiﬁcant effects found due to the task (Titanic versus Challenger).

These results indicate that, compared to the second year students, the ﬁrst year students wrote fewer sentences in total for their reports, and fewer evaluative statements. Like the second year students, the ﬁrst year students wrote signiﬁcantly more sentences in English than in Japanese. Analysis of the complexity of the sentences they produced revealed that the total number of verbs they used was likewise higher in Japanese compared to English, F(1, 43) = 14.80, p < 0.001, p2= 0.032; and the index of complexity (total number of verbs divided by total number of sentences) was higher for what they wrote in Japanese compared to what they wrote in English, F(1, 43) = 75.58, p < 0.001, p2= 0.409. This ﬁnding indicates that the ﬁrst year students, like their second year counterparts, were producing more complex sentences in Japanese compared to English.

The results of the correlational analysis are shown in Table 4. None of the correlations were signiﬁcant – suggesting that, without instruction, the students’ proﬁciency in both languages generally did not make a difference to their production of the target evaluative language. 3.3. Discussion The results of Study 2 showed that the second year students wrote more sentences in their reports, and produced more of the target evaluative language, compared to the ﬁrst year students. This ﬁnding suggests that instruction on appropriate language to use – which had been provided to the second year students – can improve students’ abilities in manifesting critical evaluation in their written work. Although as noted the instruction was provided almost entirely in English, the signiﬁcant language effects found were all in favor of the Japanese language, which suggests that there is transfer across the languages in skills acquisition. In other words, skills taught and learned in English also produce improvements in the production of evaluative language in Japanese.

The results of the correlational analysis underline the importance of providing instruction to cultivate student competence in critical evaluation. No signiﬁcant correlations were found between the proportions of evaluative language the students produced and their proﬁciency scores in English and in Japanese. This therefore suggests that general proﬁciency on its own is not adequate: for students to evidence critical evaluation in their written work, it is necessary to also provide them with instruction on the speciﬁc language forms and structures to use. 4. General discussion The ﬁndings of this study provide evidence that, at least for Japanese students, using the Japanese language (their L1) presents no disadvantage compared to English (their L2) in the production of evaluative language (i.e., the Japanese language structure is not a limiting factor). How Japanese students’ evaluative language use might compare to that of students whose ﬁrst language is structured differently (e.g., native English speakers responding to the same tasks), or students who are fully bilingual in Japanese and English, would need to be examined in future research. However, in the present study, there appeared to be no obvious deﬁcits in evaluative language production in Japanese among the second year students who had received instruction in the necessary academic discourse.

There is evidence in the present study, however, that language proﬁciency can inﬂuence the production of evaluative language. Although the conceptualization of critical thinking is likely to be independent of the language being used, the use of a language that one is not so proﬁcient in requires greater cognitive processing resources and thus limits the remaining resources that could be utilized for the expression of critical thinking. In this study, the signiﬁcant correlations between the students’ English proﬁciency scores and the amount of evaluative language they evidenced in their English writing indicate that performance varied with L2 proﬁciency. This provides useful evidence to corroborate previously made claims (e.g., Floyd, 48 E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 2011; Lun et al., 2010; Paton, 2005 ) that some of the shortcomings in critical thinking skills manifested by international students can be attributed to their having to use an L2 in which they may not be as proﬁcient compared to their native speaker counterparts. Although those signiﬁcant correlations were not high, results were also obtained in the present study showing that the students’ Japanese writing complexity signiﬁcantly correlated with the proportion of evaluative language they produced in Japanese. This ﬁnding provides further support for the argument that language proﬁciency can inﬂuence the production of evaluative language, this time in the L1. 4.1. Conclusion The ﬁnding that L2 proﬁciency is a potential limiting factor in students’ use of critical thinking skills suggests that educational strategies for improving non-native speaker students proﬁciencies in English (or the language used in the host country) would be beneﬁcial. Successful implementation of such strategies would likely reduce the perceived deﬁciencies in Asian and other foreign students’ critical thinking competencies (cf. Lee & Carrasquillo, 2006; Robertson et al., 2000 ).

The ﬁndings of this study also indicate that appropriate classroom instruction is essential for facilitating the development of students’ abilities to demonstrate critical thinking competencies in the work they produce. The second year students evidenced similar writing proﬁles to those of ﬁrst year students; however, having received instructions in academic discourse relevant to critical evaluation, they also produced more of the target evaluative language. They did this in both languages, L1 and L2, even though academic discourse instruction was primarily provided in the L2 – suggesting some transfer of skills across languages. Furthermore, the lack of signiﬁcant correlations between the ﬁrst year students’ L1 and L2 proﬁciency and their use of the target evaluative language underline the necessity of providing critical evaluation instruction to students.

Without receiving explicit instruction, most students − irrespective of their language proﬁciency – would not likely know how to effectively demonstrate critical evaluation in the work they produce. List of funder and grant number This research was supported by a grant-in-aid (23243071) received from the Japan Society for the Promotion of Science. Conﬂict of interest There is no conﬂict of interest known to the authors.

Acknowledgements The authors would like to thank Kyoko Watanabe and Rongting Shen for their invaluable assistance during various stages of this study.

References Ackerman, P. L. (1986). Individual differences in information processing: an investigation of intellectual abilities and task performance during practice.

Intelligence, 10, 101–139. Anthony, L., Rose, R., & Sheppard, C. (2010). Concept building and discussion: foundations. Tokyo: DTP Publishing. Atkinson, D. (1997). A critical approach to critical thinking in TESOL. TESOL Quarterly, 31, 9–37. Au, T. K. F. (1983). Chinese and English counterfactuals: the Sapir-Whorf hypothesis revisited. Cognition, 15, 155–187. Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press. Baddeley, A. (1998). Human memory. Boston: Allyn & Bacon. Bloom, A. (1981). The linguistic shaping of thought. Hillsdale, NJ: Erlbaum. Boroditsky, L. (2001). Does language shape thought?: Mandarin and English speakers’ conceptions of time. Cognitive Psychology, 43, 1–22. Chen, J. Y. (2007). Do Chinese and English speakers think about time differently? Failure of replicating Boroditsky (2001). Cognition, 104, 427–436. Davies, W. M. (2006). An ‘infusion’ approach to critical thinking: Moore on the critical thinking debate. Higher Education Research & Development, 25, 179–193. Dornyei, Z., & Katona, L. (1992). Validation of the C-test amongst Hungarian EFL learners. Language Testing, 9, 187–206. Eggly, S., Musial, J., & Smulowitz, J. (1999). The relationship between English language proﬁciency and success as a medical resident. English for Speciﬁc Purposes, 18, 201–208. Ellis, R. (2009). The differential effects of three types of task planning on the ﬂuency, complexity, and accuracy in L2 oral production. Applied Linguistics, 30, 474–509. Ennis, R. H. (1998). Is critical thinking culturally biased? Teaching Philosophy, 21, 15–33. Fisher, A., & Scriven, M. (1997). Critical thinking: its deﬁnition and assessment. Norwich, UK: Centre for Research in Critical Thinking. Floyd, C. B. (2011). Critical thinking in a second language. Higher Education Research and Development, 30, 289–302. Fox, H. (1994). Listening to the world. Urbana, IL: National Council of Teachers of English. Glassner, A., Weinstock, M., & Neuman, Y. (2005). Pupils’ evaluation and generation of evidence and explanation in argumentation. British Journal of Educational Psychology, 75, 105–118. Halpern, D. F. (1998). Teaching critical thinking for transfer across domains. American Psychologist, 53, 449–455. Hill, J. H., & Mannheim, B. (1992). Language and world view. Annual Review of Anthropology, 21, 381–406. Hockett, C. E. (1954). Chinese versus English: an exploration of the Whorﬁan thesis. In H. Hoijer (Ed.), Language in culture (pp. 106–123). Chicago:

University of Chicago Press. In’nami, Y., & Koizumi, R. (2011). Factor structure of the revised TOEIC test: a multiple-sample analysis. Language Testing, 29, 131–152. Ishikawa, S. (1995). Objective measurement of low-proﬁciency EFL narrative writing. Journal of Second Language Writing, 4, 51–69. E. Manalo, C. Sheppard / Thinking Skills and Creativity 21 (2016) 41–49 49 Itakura, H., & Tsui, A. B. M. (2011). Evaluation in academic discourse: managing criticism in Japanese and English book reviews. Journal of Pragmatics, 43, 1366–1379. January, D., & Kako, E. (2007). Re-evaluating evidence for the linguistic relativity hypothesis: response to Boroditsky (2001). Cognition, 104, 417–426. Kong, K. C. C. (2005). Linguistic resources as evaluators in English and Chinese research articles. Multilingua, 24, 275–308. Ku, K. Y. L. (2009). Assessing students’ critical thinking performance: urging for measurements using multi-response format. Thinking Skills and Creativity, 4, 70–76. Lee, K. S., & Carrasquillo, A. (2006). Korean college students in United States: perceptions of professors and students. College Student Journal, 40, 442–456. Liu, L. G. (1985). Reasoning counterfactually in Chinese: are there any obstacles? Cognition, 21, 239–270. Lun, V. M. C., Fischer, R., & Ward, C. (2010). Exploring cultural differences in critical thinking: is it about my thinking style or the language I speak?

Learning and Individual Differences, 20, 604–616. Manalo, E., & Uesaka, Y. (2012). Elucidating the mechanism of spontaneous diagram use in explanations: how cognitive processing of text and diagrammatic representations is inﬂuenced by individual and task-related factors. In P. Cox, P. Rodgers, & B. Plimmer (Eds.), Diagrammatic representation and inference. Lecture notes in artiﬁcial intelligence (LNAI) 7352 (pp. 35–50). Berlin, Heidelberg: Springer-Verlag. Manalo, E., & Uesaka, Y. (2014). Students’ spontaneous use of diagrams in written communication: understanding variations according to purpose and cognitive cost entailed. In T. Dwyer, H. Purchase, & A. Delaney (Eds.), Diagrammatic representation and inference. Lecture notes in artiﬁcial intelligence (LNAI) 8578 (pp. 78–92). Berlin, Heidelberg: Springer-Verlag. Manalo, E., Kusumi, T., Koyasu, M., Michita, Y., & Tanaka, Y. (2013). To what extent do culture-related factors inﬂuence university students’ critical thinking use. Thinking Skills and Creativity, 10, 121–132. Manalo, E., Kusumi, T., Koyasu, M., Michita, Y., & Tanaka, Y. (2015). Do students from different cultures think differently about critical and other thinking skills? In M. Davies, & R. Barnett (Eds.), Palgrave handbook of critical thinking in higher education (pp. 299–316). New York: Palgrave Macmillan. Neisser, U., Boodoo, G., Bouchard, T. J., Jr., Boykin, A. W., Brody, N., Ceci, S. J., & Urbina, S. (1996). Intelligence: knowns and unknowns. American Psychologist, 51, 77–101. Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proﬁciency: a research synthesis of college-level L2 writing. Applied Linguistics, 24, 492–518. Paton, M. (2005). Is critical analysis foreign to Chinese students? In E. Manalo, & G. Wong-Toi (Eds.), Communication skills in university education: the international dimension (pp. 1–11). Auckland: Pearson Education. Perkins, K. (1987). Review of the ‘test of English for international communication’. In J. C. Alderson, K. J. Krahnke, & C. W. Stansﬁeld (Eds.), Reviews of English language proﬁciency tests (pp. 81–83). Washington, DC: TESOL. Powers, D. E., Kim, H.-J., & Weng, V. Z. (2008). The redesigned TOEIC (listening and reading) test: relations to test-taker perceptions of proﬁciency in English (ETS Research Rep. No. RR-08-56). New Jersey: ETS; Princeton. Richards, J., Platt, J., & Platt, H. (1992). Longman dictionary of language teaching and applied linguistics. Harlow, UK: Longman. Robertson, M., Line, M., Jones, S., & Thomas, S. (2000). International students, learning environments and perceptions: a case study using the Delphi technique. Higher Education Research & Development, 19, 89–102. Scollon, R., & Wong-Scollon, S. (1991). Topic confusion in English-Asian discourse. World Englishes, 10, 113–125. Skehan, P., & Foster, P. (1999). The inﬂuence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93–120. Takano, Y., & Noda, A. (1993). A temporary decline of thinking ability during foreign language processing. Journal of Cross-Cultural Psychology, 24, 445–462. Takano, Y., & Noda, A. (1995). Interlanguage dissimilarity enhances the decline of thinking ability during foreign language processing. Language Learning, 45, 657–681. Takano, Y. (1989). Methodological problems in cross-cultural studies of linguistic relativity. Cognition, 31, 141–162. Ten Dam, G., & Volman, M. (2004). Critical thinking as a citizenship competence: teaching strategies. Learning and Instruction, 14, 359–379. Thomm, E., & Bromme, R. (2011). It should at least seem scientiﬁc! Textual features of scientiﬁcness and their impact on lay assessments of online information. Science Education, 96, 187–211. Tse, C.-S., & Altarriba, J. (2008). Evidence against linguistic relativity in Chinese and English: a case study of spatial and temporal metaphors. Journal of Cognition and Culture, 8, 335–357. Vorster, J., & Schuring, G. (1989). Language and thought: developmental perspectives on counterfactual conditionals. South African Journal of Psychology, 19, 34–38. Whorf, B. L. (1956). The relation of habitual thought and behavior to language. In J. B. Carroll (Ed.), Language, thought, and reality: selected writings of Benjamin Lee Whorf (pp. 134–159). Cambridge, MA: MIT Press. Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: measures of ﬂuency, accuracy, and complexity. Technical report no. 17. Honolulu, H awai’i: University of Hawai’i, Second Language Teaching and Curriculum Center.