Scholarly Journal Article Critique (SJAC)
Running Head: SJAC 1
Opportunities for Learning, a Reform Method Revisited
Kenneth E. Matthews
EDUC 543—Educational Testing and Measures
09 February 2014
Instructor Name: Dr. Jeni McRay
Abstract
Opportunities to Learn provide teachers, administrators, policy makers, and others the ability to assess ways to make content new and learnable for students across ability level lines. The article reviewed provides a window into the convoluted and often confusing world of education reform/testing as it apples to the concept of opportunities to learn. With the continued calls for education reform, it becomes readily apparent that those interested in moving forward have to possess a good grasp of what has already been tried in the past. The authors present compelling evidence that through close analysis of research and studying what has actually been tried as a result of said research, perhaps between the two, society can find a way forward to real, effective, and durable education reform.
Opportunities for Learning, a Reform Method Revisited
Due to the current practice of curriculum differentiation, below average students are not offered course content that emphasizes preparation for Algebra and writing later in their academic careers. Research has indicated that not providing the same prerequisite course content to lower ability students preempts them from achieving a level educational playing field moving forward. The authors indicate that by exposing such students to the same content as students of higher ability levels such a measure would improve their academic achievement, if provided under the right circumstances.
Thesis Statement
The Common Core Curriculum standards currently being implemented present significant implementation challenges as they apply to students of all ability levels, yet by analyzing the pitfalls of the standards and correcting noted deficiencies using the “concept of opportunities”, it is, nonetheless possible, that such standards can have beneficial impacts for all students.
Main Points
In a breathtakingly simple analysis (Based upon the concept of “opportunities to learn” (McPartland, J., & Schneider, B., 1996, p. 1), research has indicated that lower-level ability students who have not been taught the same content as other students do not score as well on standardized tests. This assertion is undergirded by the disparity between what is included on the standardized test in question as compared to what was taught to such students in class. As one might expect, students of advanced ability scored higher on the tests, not merely because they are more advanced academically, but also because they were afforded more opportunities to learn.
The authors refer to several studies that had the effect of exposing the deficiency in the way that teachers were teaching course content to students. In one study conducted by the International Education Association (IEA), the mathematical achievement of 132, 775 students were analyzed by measuring the opportunities to learn afforded to students (McPartland, J., & Schneider, B., 1996, p. 2). This measurement was based upon what percentage of students teachers reported they had taught tested materials to (McPartland, J., & Schneider, B., 1996, p. 2). Other measures included in this massive study included 26 total hypothesis’, consisting of a multitude of other variables, including school organization, curriculum, and instructional methods.
In a revealing result, the study indicated a high degree of correlation between what teachers had reported were their student’s respective opportunities to learn and how students had actually scored on the specific test items. Also of particular interest, on a more international scale, was the fact that nations who tended not to differentiate showed a substantially reduced level of correlation as students of all ability levels test outcomes tended to be more streamlined. An intriguing outcome to this study was that mathematical achievement did not seem to improve when there was an increased investment of time in the instruction. This data seems to suggest that what is taught is more important than how long it is taught (McPartland, J., & Schneider, B., 1996, p. 3).
The IEA has also conducted on-going studies of elementary school opportunities to learn, thereby improving and expanding subsequent research designs and techniques. One significant result of such further study indicates that students may be receiving a significant amount of their math and science education from outside of school. The unfortunate takeaway indicates that accurately measuring what students are learning in school in these subjects may be much more difficult than originally thought. Since a good deal of what students learn comes from outside of school, the challenge becomes determining what was learned in school. Currently, there do not appear to be any substantive solutions to this issue.
Another study undergirding the concept of opportunities to learn was the Instructional Dimensions Study published in 1980 by Cooley and Leinhardt (McPartland, J., & Schneider, B., 1996, p. 3). Perhaps the most important result of this study was that regardless of the type of whole-class technique or structure employed by a teacher (Individual, small group, or whole class), the most beneficial form of instruction for students of lower ability levels in math and reading was individualized instruction. This finding undergirds the notion that the occurrence of instruction is more important than the modality.
Through the aforementioned studies and a variety of others as well, the authors articulate a construct aimed at making the Common Core Standards accessible and achievable for students from the lower echelons of ability levels. The authors additionally strike a cautionary note as it relates to the notion of making the assumption that merely increasing academic rigor in the basic core curriculum disciplines would produce the desired result of increases in lower achieving students’ academic outcomes. This is based upon the assumption that students who present as underachieving in basic curriculum areas are likely underachieving due to being poorly prepared from previous academic experiences. If such were the case, such students would likely not improve in achievement as they would still lack the prerequisite skills necessary to thrive under more rigorous academic conditions.
The authors also call attention to the problem of weakened and in some cases deliberately undermined attempts at desired changes in curriculum and instructional policies. This practice manifested itself in various states, such that, as opposed to increasing curriculum requirements as has been indicated to be desirous by contemporary research, many states merely increased the number of credits required for graduation. In a study conducted in 1989 (Clune, White, and Patterson) (McPartland, J., & Schneider, B., 1996, p. 6), the study indicated that many states offered an expanded selection of courses with the sole intent of helping students pass the states minimum competency standards. As a result, such measures completely undermined the intent to improve academic rigor.
In other states, seemingly well-intended changes in curriculum did little to change the status-quo as it related to both student achievement and graduation outcomes (For example, whether students went on to attend college or went straight into the work force). Such was the case in Maryland which employs fairly stringent tracking practices (McPartland, J., & Schneider, B., 1996, pp. 6-7). A similar outcome occurs in New York state schools, the only difference being that as opposed to a defined system of tracking, New York employs localized “Regents examinations” designed to exclude all but a small minority of students from either taking a curriculum with higher academic rigor, or receiving either a high school certificate, as opposed to a more desirable diploma.
Other measures of evasion and subterfuge are employed by many states in order to manipulate the percentages of students that “appear” to be failing tests. This has been accomplished by implementing “accommodations” that the authors characterize as “political gestures” (McPartland, J., & Schneider, B., 1996, p. 8). Such practices include: establishing minimalist standards and artificially low cut-off scores, providing exemptions to certain low achieving students, and allowing multiple attempts to pass tests, and the like the study by Ellwein, Glass, and Smith, 1988 (McPartland, J., & Schneider, B., 1996, p. 8). The authors suggest that measures designed to mitigate such practices would include strict security measures for testing, and administration of tests by officials who have no stake in testing outcomes. Yet these measures would only go part of the way toward enforcing testing as further inquiry demonstrates.
Other activities conducted by school systems and states to alter testing outcomes are breathtaking in scope. Additional activities such as manipulating the sample of students being tested and reporting chronically absent students as dropouts tends to skew testing results in the favor of the school and/or school system (McPartland, J., & Schneider, B., 1996, p. 8). The authors recommend making misrepresentation of information less enticing for schools and systems to alter their data and demographics by requiring “…major changes in the social organization of the learning environment...” (McPartland, J., & Schneider, B., 1996, p. 9). The authors indicate that measures designed to reduce incentives for misrepresenting facts would mitigate such practices. Giving teachers and their schools a decent chance of preparing their students to meet requirements without feeling the need to cheat appears to be the most compelling.
The authors also provide extensive information on addressing student diversity. In particular, they address the lack of reforms in the area of the social organization of schools as lagging behind, this inhibits students of diverse backgrounds less equitable opportunities meet higher standards. Some of the issues associated with making achievement more equitable for diverse student populations are; enhancing resources in low wealth districts and allowing for greater flexibility in the employment of resources for maximum effect (McPartland, J., & Schneider, B., 1996, p. 9)
In particular, it is indicated that significant increases in financial allocations for low wealth districts is warranted. This mandate seems to be married to the proposal by Darling-Hammond that a standards of excellence be established which would require equitable access to school funding (McPartland, J., & Schneider, B., 1996, p. 10). This proposal seems to mirror previous proposals aimed at closing the funding gaps between high wealth and low wealth districts.
On-going debate continues over which metric should be employed as a key indicator for improving the dimensions of learning opportunities. Most contentious are the issues of per-pupil expenditures versus the amount of time allotted during the school year to addressing the diversity disparity. The parallel concern with these two issues is over how increases in revenue and resources will be effectively implemented (McPartland, J., & Schneider, B., 1996, pp. 12). The herculean task that lies before researchers is evaluating instructional delivery systems to help all students meet the higher standards.
Other relevant concerns voiced by the authors pertinent to the issue of diversity are: the extent of the need for additional resources and funding (McPartland, J., & Schneider, B., 1996, p. 12), mitigating the challenges posed by students failing courses, and the feasibility of after school coaching for failing students. Measures aimed at positively impacting the challenge of diversity are: reconciling the shortages of staffing with tutoring provided by teachers (and students), along with remediation services, doubling courses, and extended school day/year. The authors mention recent changes in distribution of federal aid aimed at expanding opportunities to learn as a method of improving funding streams to meet the proposed challenges.
In addressing the criteria for academic success, the authors particularly make note of differences in initial ability levels of students as a metric that needs to be included when assessing proficiency of both teachers and students. Additionally, the potential benefits to be garnered from a program that recognizes growth from the aforementioned initial ability levels is posited as a way to energize both positive expectations and recognition for school and districts that implement the common core curriculum without attempting to subvert the process with misleading tactics (McPartland, J., & Schneider, B., 1996, p.12)
Three studies ranging in time from the late 1970’s to the early 1980’s revealed that the concept of “incentives for improvement” had significant positive impact on student achievement and interest in particular courses. Individual progress would be measured based upon growth over a school year, while graduation was earned through periodic assessments. Such a system would deliver a legitimate opportunity for all students to achieve academic success by eliminating tracking, providing performance incentives for all students based on rewards that would be based on the progress of individual students as opposed to overall outcome based grades (McPartland, J., & Schneider, B., 1996, p. 13)
Based upon the research of Lee, Bryk, and Smith 1993 and McPartland 1990, the authors also render a cautionary message concerning the threat of undermining teacher’s positive, caring relationships with their students (McPartland, J., & Schneider, B., 1996, p. 12). Two concerns are presented that could derail these relationships. The first threat involves movement away from supportive one-on-one relationships between students and teachers toward more rigorous academic requirements will create regimented and fragmented relationships between students and teachers. The second problem is generated by teachers who are required to enforce more rigorous academic standards must then do battle with low achieving students who will attempt to force teachers into “classroom treaties” such that teachers would be coerced into softening standards for such students. The challenge then becomes, how can teachers maintain high academic standards in the face of such obstacles?
To mitigate the just mentioned issues, the authors point to the value of interdisciplinary teams. Such teams provide the benefits of sustaining curriculum reforms; and have been attributed to both higher student achievement and better student teacher relations (McPartland, J., & Schneider, B., 1996, p. 13). Additionally, having external standards of academic achievement, as opposed to those formulated specifically by the teachers, prevents students from attempting to circumvent the existing standard with the understanding that the teacher cannot alter them (McPartland, J., & Schneider, B., 1996, p. 13).
Sociologists have pointed to sociological forces as the reason why past curriculum reforms have failed. According to the McPartland and Schneider, current reforms are also threatened by these same forces. The on-going obstacle behind all this is the students that do not possess adequate prerequisite skills. Opportunities to learn reforms that provide enhanced resources and more flexible scheduling provide the most promise for being able to successfully implement the new core curriculum standards.
Critical Assessment
In my opinion, although this article was published in 1996, the authors present a compelling, yet mixed bag of information as it applies to the field of education. Some of the topics they discussed continue to have relevance today (Some 18 years later), while others reflect the problems associated by data analysis as discussed by Dr. Daniel Koretz Measuring Up, What Educational Testing Really Tells Us. In particular, I found intriguing the author’s usage and comparisons of global test scores as presented in the IEA data from a study in 1964. Koretz argues that samplings from different groups tested and at different times would show variances due to changes in the composition of students over time. Such data comparisons present “compositional effects” (Koretz, 2009, pp-87-88).
The particularly disturbing inaccuracy of the IEA study (McPartland, J., & Schneider, B., 1996, p. 2) was its reliance on a multitude of testing systems from many different countries all utilizing what must be a variety of divergent testing methods, comprising a multitude of mixes in ability groups, and differences in both content and test designs. As Koretz argues against the accuracy of measurement in terms of the SAT scores in the US during the era of “A Nation at Risk” (Koretz, 2009, p- 58), one must also argue against making carte blanche comparisons between and among data drawn from often confusing and ill-fitted global assemblages of testing outcomes as well.
One must, however, balance criticism with praise for some of the other analysis presented by the authors of the article in question, The Instructional Dimensions Study in 1980 (McPartland, J., & Schneider, B., 1996, p. 3), presents a thought provoking comparison of 100 different elementary schools by measuring such constructs as classroom management and small group versus whole-class instruction. The study utilizes multiple regression analysis to isolate variations in scores in order to expose the aspects of such instruction that showed the greatest gains. Unsurprising was the finding that pulling underachieving students out of class for individualized instruction helped such students, regardless of the method of instruction utilized.
Also compelling were the forthright and honest assessments of the impact of tertiary forces upon educational effectiveness within the field of education. Citations of such impacts ran the gamut from purposefully broadening curriculum offerings while “dumbing down” content (McPartland, J., & Schneider, B., 1996, p. 7) in order to give the appearance of complying with expanded curricular offerings, to purposefully denying certain groups of students the opportunity to take more rigorous curriculum based upon what seems to be arbitrary and exclusionary criteria (McPartland, J., & Schneider, B., 1996, p. 7). As mentioned previously, concise examples of the states of Maryland and New York were offered as examples of states giving token homage to increasing course offerings and strengthening content. In both examples, the reader is left with the impression that the authors were making a legitimate effort at being fair and impartial in their assessment of progress as it applies to the “tricks” used to circumvent curriculum reform (Purposeful or otherwise) in the US, at that time.
Other examples of keen analysis are presented from the authors in relation to the access to resources and flexibility in curriculum implementation. Particularly telling was the acknowledgement of the massive need for funding increases in low wealth school districts to fund expanded curriculum initiatives, while acknowledging the need to prepare such students to be able to achieve the higher academic standards common core proposes (McPartland, J., & Schneider, B., 1996, p. 10). The authors allude to variances in financial resources between districts (McPartland, J., & Schneider, B., 1996, p. 10) which is still a raging bone of contention for those advocating for more financial equity in education. Ground zero will undoubtedly be, as it has been in the past, wealthy versus poor districts within each state.
Particularly interesting also was the acknowledgement by the authors as to the use of tutoring to improve student achievement in elementary schools. In a program titled “Success For All” tutoring is provided for the neediest students, the results of which were “impressive gains” in student achievement (McPartland, J., & Schneider, B., 1996, p. 11). One is left to surmise that by employing the phrases achievement and impressive gains, the authors are likely referring to standardized testing outcomes. If such is the case, then what we are probably talking about here is test score “inflation” (Koretz, 2009, p-34) brought about by intensive review (as opposed to real content remediation as a “curriculum” add-on) as is likely the public selling point for this program.
The authors discuss changes (At the time of the article) to procedures for receiving federal aid (Title I funds), which they indicated would make additional funds available for low-wealth schools. One must acknowledge, though, the realities of such measures as to the way and manner in which states and school districts are implementing them. From an on-line source is derived this quote:
The data reveal that more than 40 percent of schools that receive federal Title I
money to serve disadvantaged students spent less state and local money on
teachers and other personnel than schools that don't receive Title I money at the
same grade level in the same district. ("More than 40%," 2011)
If this is the case, then what is the purpose behind shortchanging low wealth schools? The answer, sadly, is quite simple. In the age of budget cutbacks and deficits, state and local governments are willing to take extraordinary (Albeit shocking) measures to try to save money any way they can. When one closely examines this behavior one surmises that this is really nothing short of “back door” misappropriation. Obviously, there has to be a better way to balance budgets than to take much needed money from schools that demonstrate such an obvious need.
There can certainly be no denying that redefining what is characterized as academic success in terms of student achievement is a step in the right direction. Couple this with what the authors explain as multiple levels of “absolute proficiency” and one sees promise. The whole notion behind such a redefining of proficiency is creating standards that are realistically achievable for all students. Daniel Koretz weighs in on this issue when he states:
“…most standards-based systems have three or four ranges or categories for reporting performance. Information about differences among students within any one of those ranges does not register. And those unnoted differences can be very large.” (Koretz, 2009, p-194)
Clearly there is something to be gained by moving more toward a “slide rule” in terms of student performance as opposed to a one size fits all approach, wherein every student must achieve at the same level to “achieve” a certain proficiency level. The absurdity is accepting the premise that all assessed students approach a standardized test with the same prerequisite skills (We know they do not). It does not take a genius to ascertain that there is something fundamentally flawed about such a system.
Enough cannot be said concerning the need to discard the practice of tracking as well. Segregating students in separate classes based upon perceived ability levels establishes an artificial wall of separation between students that need not be. McPartland, and Schneider carry this construct further as it relates to academic awards when they state “Criteria that separately recognize both progress and proficiency could generate incentives for students of all levels of achievement.” (McPartland, J., & Schneider, B., 1996, p. 13). Clearly this is the path to motivating underachieving students to strive to do their best on what they would surely perceive to be a more level playing field.
At the core of teaching is the fundamental capacity of teachers to have empathy and a caring attitude toward the challenges their students face. Without such relationships between student and teacher, many more students would likely wither academically and eventually drop out of school. The authors identify the threat to this solemn relationship between students and teachers as being the “departmentalized staff”. The mental picture one derives from such a characterization is the educational assembly line where students are daily passed from one class to another to receive their daily educational component. Students in this arrangement are subjected a sterile environment, long on content exposure and short on humanity. In such an arrangement, students with a strong streak of independence, who are high functioning, will thrive. Their underachieving counterparts, unfortunately, will fall by the wayside.
The solutions that the authors posit to these issues are in a word; compelling. The concept of interdisciplinary teams of teachers, coupled with external standards to mitigate the prospect of classroom treaties as previously discussed are creative and workable. Yet, one cannot diminish the importance of a highly competent teacher. In an excerpt from an on-line master’s thesis, one finds this quote:
"...teachers, competent in their subject knowledge, elicit a positive response with approximately 70% of classroom time devoted to academic tasks (Little & Akin-Little, 2003) while lessening the tendencies/opportunities for disruptive behavior (Sutherland & Wehby, 2001)...Highly effective teachers are expected to create classroom management strategies that will make the students see the need for learning (Brophy, 1998; Cowley, 2003; Jones, 1996)." (Nasey, 2012, p. 29)
Having previously taught high school history in a variety of settings with a highly culturally diversified cross-section of students, I feel somewhat qualified to say that a teacher’s ability to manage students has as much or more impact on the capacity of so motivated students to attempt to broker a classroom treaty as any other factor. Teachers who are competent and who have a high degree of subject matter knowledge are highly effective at identifying such behaviors and preempting them before they fester into such a destructive outcome. External standards aside, one must surmise that a less than effective teacher is much more likely to foster a classroom environment prone to a classroom treaty than any other factor.
In their concluding section on policy and practice, the authors make the argument that for a common core curriculum to succeed, it is imperative that higher standards be undergirded by “opportunities to learn reforms” (McPartland, J., & Schneider, B., 1996, p. 15) and a classroom environment that gives “extra help and more time to the most needy students…” (McPartland, J., & Schneider, B., 1996, p. 15). Perhaps, if we want to learn more about the efficacy of opportunities to learn, we should study where it seems to be working—in China. Facing many of the same challenges that present themselves to their American counterparts, the Chinese have achieved some stunning testing results. The following excerpt from an on-line article demonstrates this point:
Shanghai in China displayed stunning results on the PISA 2009 assessment. In
mathematics, more than a quarter of Shanghai's 15-year-olds can conceptualize, generalize, and creatively use information based on their own investigations and modeling of complex problem situations. They can apply insight and understanding and develop new approaches and strategies when addressing novel situations. In the United States it is 2.5 percent and, on average in the industrialized world, it is still below 4 percent. (Schleicher, 2012)
If we are to believe that such testing outcomes are occurring in China, then the paramount question must be—why?
The answer according to the quoted author is considerably more simplistic than you might imagine. The following quote provides clarity to the matter:
“This reform has not just been about updating and repackaging educational
content, but aims at helping students find out who they are, where they want to go
in life, and how they will get there, in a rapidly changing and increasingly
uncertain world.” (Schleicher, 2012)
To draw distinctions between how education reform in China is occurring as opposed to the US, the Chinese take a much longer view. The author indicates that the biggest weakness that the US suffers from in terms of education reform is the specter of political expediency. Where US education policy is driven by suspense dates, deadlines, and political gamesmanship, the Chinese look upon reform as a longitudinal imperative that transcends petty political agendas and competing philosophies.
In closing, how do we achieve the educational reform climate that the Chinese enjoy? The answer is seemingly simple, as a society, we must decide that the education of our children is more important than political agendas, more important than the budgetary considerations that authorize cutting educational funding to low wealth schools and districts, and to training, cultivating, and compensating teachers whose primary commitment is to providing the most compassionate, beneficial, constructive, and academically challenging classroom environment that they can possibly deliver. Unfortunately, in the meantime, we’ll likely stand by to see what movement will usher in the next round of “flavor of the day” reforms in the US.
Works Cited
Koretz, Daniel. Measuring Up: What Educational Testing Really Tells Us . Cambridge,
MA: Harvard University Press, 2009. Print.
McPartland, J., & Schneider, B. (1996). Opportunities to learn and student diversity:
Prospects and pitfalls of a common core curriculum. American Sociological
\
Association, Retrieved from
http://search.proquest.com.ezproxy.sckans.edu/education/docview/216356256/14
346A8D01570777476/11?accountid=13979
More than 40% of low-income schools don't get a fair share of state and local funds,
department of education research finds. (2011, November 30). Retrieved from http://www.ed.gov/news/press-releases/more-40-low-income-schools-dont-get-fair-share-state-and-local-funds-department-The imperative for educational reform. (1983, April). Retrieved from
http://datacenter.spps.org/uploads/SOTW_A_Nation_at_Risk_1983.pdf
Nasey, C. (2012). Teachers' use of classroom-based management strategies: A survey of
new zealand teachers. (Unpublished master's thesis) Retrieved from
http://mro.massey.ac.nz/bitstream/handle/10179/3483/02_whole.pdf?sequence=1
Schleicher, A. (2012, December 19). What we can learn from educational reform in
china. Retrieved from http://www.huffingtonpost.com/andreas-schleicher/educational-reform-in-china_b_2327908.html
