Tuesday, April 1, 2014

Age Is Not the Problem

Edward Frenkel recently resurrected an old complaint in his Los Angeles Times op-ed, “How our 1,000-year–old math curriculum cheats America’s kids.” He observes that no one would exclude an appreciation for the beauty of art or music from the need to build technique. Why do we do that in mathematics? As I said, this is an old complaint. Possibly no one has voiced it more eloquently than Paul Lockhart in A Mathematician’s Lament, the theme of Keith Devlin’s 2008 MAA column, “Lockhart’s Lament.” Enough time has passed that it is worth my while to bring this lament back to the attention of the readers of MAA columns. I also want to respond to Frenkel’s post. I have two problems with what he writes.

The first is the suggestion that we spend too much time on “old” mathematics and not enough on what is “new.” I share Frenkel’s disappointment that too few have any appreciation of mathematics as a fresh, creative, and self-renewing field of study. Frenkel himself has made a significant contribution toward correcting this. In his recent book, Love and Math, he has opened a window for the educated layperson to glimpse the fascination of the Langland’s program. But I disagree with Frenkel’s solution of devoting “just 20% of class time [to] opening students’ eyes to the power and exquisite harmony of modern math.” There is power and exquisite harmony in everything from early Babylonian and Egyptian discoveries through Euclid’s Elements to the Arithmetica of Diophantus and the development of trigonometry in the astronomical centers of Alexandria and India, all of which were accomplished more than a millennium ago and are still capable of inspiring awe. 

In fact, I believe that one of the worst things we could do is to create a dichotomy in students’ minds between beautiful modern math and ugly old math. We must communicate the timeless beauty of all real mathematics. The challenge of the educator is to engage students in rediscovering this beauty for themselves, not outside of the standard curriculum, but embedded within it. The question of how to accomplish this leads to my second problem with Frenkel.

Frenkel makes the implicit assumption that what we need is a wake-up call, that it is time to recognize that mathematics education must do more than create procedural facility. In fact, the need to combine the development of technical ability with an appreciation for the ideas that motivate and justify the mathematics that we teach goes back at least a century to Felix Klein and his Elementary Mathematics from an Advanced Standpoint. It is front and center in the Practice Standards of the Common Core State Standards in Mathematics. It was a driving concern of Paul Sally at the University of Chicago, who we so recently and unfortunately lost. It continues to motivate Al Cuoco and his staff engaged in the development of the materials of the Mathematical Practice Institute. It lies at the root of Richard Rusczyk’s creation of the Art of Problem Solving. It permeates the efforts of literally thousands of us who are struggling to enable each of our students to encounter the thrill of mathematical exploration and discovery.

As we know, it takes more than good curricular materials and good intentions to accomplish this. It requires educators who understand mathematics both broadly and deeply and can bring this expertise to their teaching. Many are working to spread this knowledge among all who would teach mathematics to our children. This is the inspiration behind the reports of the Conference Board of the Mathematical Sciences on The Mathematical Education of Teachers. It is a goal of the Math Circles, in particular the Math Teachers’ Circles that reach those who too often are unaware of the exciting opportunities for exploration and discovery within the curricula they teach.

The mathematician’s lament is still all too relevant, but it is neither unheard nor unheeded. I am encouraged by the many talented and dedicated individuals and organizations working to meet its challenge.

Saturday, March 1, 2014

Collective Action by STEM Disciplinary Societies

At the end of January, it was my great pleasure to be part of the leadership for a meeting at the MAA Carriage House of representatives of a collection of STEM disciplinary societies [1] and concerned educational associations [2] to consider ways that these societies can coordinate efforts to increase their collective impact on undergraduate education. Across academia, but especially at research universities, most faculty identify first with their discipline and department and only secondarily with their university. Disciplinary societies therefore have the potential to impact how faculty think about their teaching and how willing they are to reach outside their own department in seeking ideas and support for improving undergraduate education.

Many disciplinary societies are actively promoting effective methods for engaging students to improve both what they learn and their desire to persist. The American Physical Society and the American Association of Physics Teachers have been particularly effective in this regard. See, for example, the Physics Education Research User’s Guide, perusersguide.org, described in my column “Learning from the Physicists,” July, 2012. Over the past several years, the life sciences community, scattered over some 147 disciplinary societies, has come together to produce a joint report, Vision and Change in Undergraduate Biology Education: A Call to Action [3]. Recognizing that it is not sufficient to issue a report, Vision and Change continues to seek ways to implement the changes it champions. One outgrowth has been PULSE, the Partnership for Undergraduate Life Sciences Education, which is building communities that share experiences of department-level implementation of the Vision and Change recommendations. Inspired by the example of PULSE, the mathematics community began last summer to build a comparable effort, INGenIOuS, Investing in the Next Generation through Innovative and Outstanding Strategies.

We have much to learn from each other. Beyond just sharing information, an ability to offer comparable statements of vision and comparable programs to promulgate effective practices would increase their collective impact. This would be especially true if the disciplinary societies were to establish and promote linkages that enable individuals to connect with others at their university who are working toward the same ends but within other departments.

With these goals in mind, 28 representatives of disciplinary societies and educational associations met at the MAA Carriage House in Washington, DC on January 30–31 for an NSF-sponsored workshop [4] entitled ISSUES, Integration of Strategies that Support Undergraduate Education in STEM, to look for opportunities to work collectively. As preparation, most of the societies provided a summary of their current activities directed toward faculty development and the improvement of undergraduate education. These Profiles can be found within the ISSUES website at serc.carleton.edu/issues. A summary of the workshop is available at serc.carleton.edu/issues/workshop14.

The workshop identified five concrete areas in which disciplinary societies could increase their effectiveness by sharing and coordinating their efforts:
  1. Supporting Early Career Faculty. Within the disciplinary societies, the task is to develop workshops for and build communities of early career faculty, as well as partnering with the Discipline-Based Educational Research community to assess the long-term effectiveness of this work. On individual campuses, the task is to work with deans and chairs to build cross-disciplinary networks of faculty who have been through these experiences, supported by networks of mentors both from the individual’s profession and from within the individual’s home institution.
  2. Strengthening Departments. There is a need to increase the value placed on the department chair and to provide support for the chair by supplying tools for departmental self-assessment of teaching effectiveness together with practical suggestions that chairs and departmental leaders can implement to improve teaching effectiveness.
  3. Communicating Career Pathways. We need to increase the diversity of students within our disciplines by increasing student awareness of the variety of pathways that are available to them, actively recruiting students to these pathways, preparing them for a variety of careers, and introducing them to a network of potential employers.
  4. Shifting Cultural Norms. Disciplinary societies should strive to move their members toward embracing teaching practices that align with what educational research has shown to be most effective and toward a mindset of continual efforts to improve undergraduate teaching and learning. This can be accomplished through policy statements, rubrics for assessing effective educational processes, and active promotion of these practices. Part of our collective goal should be the adoption of consistent language that reinforces this message across disciplinary boundaries.
  5. Measuring the Impact of Our Own Programs for Improving Undergraduate Education. The disciplinary societies can benefit from developing common rubrics for assessing the effectiveness of their own programs and using these to help frame discussion and dialog across the societies.

On point 1, we are already working with the Association of American Universities (AAU) to put together a pilot project on AAU campuses that will build local networks of faculty from multiple disciplines who have each been through an early career professional development program run by their disciplinary society. On point 5, we are beginning the task of gathering information from the disciplinary societies about their experiences with assessment of their own programs. Within the next months, we hope to see progress on all of these agendas.

 


Footnotes and References:

[1] The disciplinary societies that were represented were the American Association of Physics Teachers, American Chemical Society, American Geophysical Union, American Institute of Biological Sciences, American Institute of Physics, American Mathematical Society, American Physical Society, American Psychological Association, American Society for Engineering Education, American Society for Microbiology, American Statistical Association, Mathematical Association of America, National Association of Biology Teachers, National Association of Geoscience Teachers, and the Society for Industrial and Applied Mathematics.

[2] The educational associations that were represented included the American Association for the Advancement of Science, Association of American Universities, Association of Public Land-Grant Universities, Howard Hughes Medical Institute, National Academy of Sciences, National Science Foundation, and Project Kaleidoscope of the Association of American Colleges and Universities.

[3] Brewer, C.A., and Smith, D. (eds.). 2011. Vision and Change in Undergraduate Biology Education: A Call to Action. Washington, DC: American Association for the Advancement of Science. Available at visionandchange.org/files/2013/11/aaas-VISchange-web1113.pdf

[4] The workshop was made possible by a grant from the National Science Foundation, #1344418. The opinions expressed here do not necessarily reflect those of NSF.

Saturday, February 1, 2014

Mathematics for the Biological Sciences

MAA has just published a Notes volume, Undergraduate Mathematics for the Life Sciences: Models, Processes, and Directions [1] that provides examples and advice for mathematics departments that want to reach out to the growing population of biological science majors.

Biological science majors have replaced prospective engineers as the largest group of students taking regular Calculus I. From the MAA’s Calculus Survey [2], just over 28% of all students in mainstream Calculus I intend to pursue a major in the biological sciences, the largest single group of majors in this course. It is larger than engineers (just under 28%) or the combined physical science (7%), computer science (7%), and mathematical science majors (1%). For women in mainstream Calculus I, 42% intend a biological science major. For Black or Hispanic students, 34% are going into biological sciences. This dominance is certain to only increase. As the graph in Figure 1 illustrates, the growth in science, engineering, and mathematical sciences majors is occurring almost exclusively in the biological sciences.

Figure 1: Number of full-time entering freshmen who identified a STEM field as their most likely major. Data from The American Freshman surveys. [2]
Mathematics has done well by encouraging students who have to study mathematics to continue its study. Mathematics departments actually graduate more majors than the number of students who enter with the intention of pursuing a math major. In 2012, 18,842 students graduated with a Bachelor’s degree in mathematics.[3] Four years earlier, in 2008, only 11,583 entered a full-time program with the intention of majoring in mathematics.[4] Even after subtracting the roughly 5,000 students per year who are heading into K-12 mathematics teaching and who get a degree in mathematics but identify education as their intended field when they enter, we see that mathematics—uniquely among the major STEM disciplines—still has a net gain in majors. If we are to maintain this happy state of affairs, then we need to convince our audience that mathematics is relevant to its interests.

Mathematics departments are recognizing this fact. The 2010 CBMS report revealed that 41% of those at research universities had added interdisciplinary courses in mathematics and biology within the past five years. As the new MAA Notes volume illustrates, there is a tremendous amount of experimentation under way.

This volume begins with descriptions of thirteen programs that range from calculus for biology majors, to programs that draw calculus and statistics together into a year-long course, to bioinformatics, to research programs for biology majors that incorporate significant quantitative analysis. The institutions include large universities: Illinois at Urbana-Champaign, Ohio State, and the Universities of Minnesota, Nebraska-Lincoln, and Utah. There also are smaller places: Benedictine University, Macalester College, University of Richmond, Chicago State, Sweet Briar College, University of Wisconsin-Stout, and East Tennessee State.

The volume continues with a collection of essays on “Processes.” These are nine accounts of the trials and tribulations of getting such a program started and keeping it going. This is particularly useful because these essays describe both programs that have survived, moving beyond the small group of individuals who initiated them, and programs that have failed or are failing, the ones that have not managed to establish themselves as a permanent feature of the local curriculum. The final four essays, labeled “Directions,” speak to opportunities and needs.

Unfortunately, this book is only available as a pdf file ($25) or as Print-on-Demand ($43), so you probably will not see a display copy at MAA meetings. But it is well worth checking out.

References

[1] Ledder, G., J.P. Carpenter, and T.D. Comar (Eds.) (2013). Undergraduate Mathematics for the Life Sciences: Models, Processes, and Directions. MAA Notes #81. Washington, DC: Mathematical Association of America. www.maa.org/publications/ebooks/undergraduate-mathematics-for-the-life-sciences

[2] Higher Education Research Institute. (Multiple years). The American Freshman. www.heri.ucla.edu/tfsPublications.php

[3] NCES. (2013). Digest of Education Statistics. Table 322.10. nces.ed.gov/programs/digest/d13/tables/dt13_322.10.asp

[4] Higher Education Research Institute. (2008). The American Freshman: National norms for fall 2008. www.heri.ucla.edu/tfsPublications.php

Wednesday, January 1, 2014

MAA Calculus Study: Seven Characteristics of Successful Calculus Programs

By David Bressoud and Chris Rasmussen

In these days of tight budgets and pressure to improve retention rates for science and engineering majors, many mathematics departments want to know what works, what are the most effective means of improving the effectiveness of calculus instruction. This was the impetus behind the study of Characteristics of Successful Programs in College Calculus undertaken by the MAA. The study consisted of a national survey in fall 2010, followed by case study visits to 17 institutions that were identified as “successful” because of their success in retention and the maintenance of “productive disposition,” defined in [NRC 2001] as “habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one’s own efficacy.”

Our survey revealed that Calculus I, as taught in our colleges and universities, is extremely efficient at lowering student confidence, enjoyment of mathematics, and desire to continue in a field that requires further mathematics. The institutions we selected bucked this trend. This report draws on our experiences at all 17 colleges and universities, but focuses on the insights drawn from those universities that offer a Ph.D. in mathematics, the universities that both produce the largest numbers of science and engineering majors and that often struggle with how to balance the maintenance of high quality research with attention to undergraduate education.

Case studies were conducted in the fall of 2012 at five of these universities: two large public research universities, one large private research university, one public technical institute, and one private technical institute. We shall refer to these as

  • ETI: Eastern Technological Institute. Private university. Data from nine sections of calculus with an average enrollment of 33.
  • MTI: Midwestern Technological Institute. Public university. Data from seven sections with an average enrollment of 38 and one with an enrollment of 110.
  • MPU: Large Midwestern Public University. Data from 41 sections with an average enrollment of 27.
  • WPU: Large Western Public University. Data from four sections with an average enrollment of 200.
  • WPR: Large Western Private University. Data from three sections with an average enrollment of 196 and one section with 32 students.
In addition to productive disposition and improved retention rates, the five also had noticeably higher grades (see Figure 1), cutting the DFW rate from 25% across all doctoral universities to only 15% at the case study sites. The difference was in B’s and C’s. The five case study universities actually gave out a slightly lower percentage of A’s than the overall average.


We identified seven characteristics of the calculus programs at these five universities, characteristics that, as applicable, were also found at the other twelve:

  1. Regular use of local data to guide curricular and structural modifications. In his description of the MAA study of Models that Work [Tucker, 1995], Alan Tucker wrote, “No matter how successful their current programs are, faculty members in the visited departments are not yet satisfied with the programs. Experimentation is continuous.” [Tucker, 1996] We found that not only was this true of the successful programs we studied, these universities used the annual gathering and sharing of data on retention and grade distributions to guide this continuous experimentation. A bad semester was not dismissed as an anomaly, but was viewed as an opportunity to understand what went wrong and what could be done to avoid a similar occurrence.
  2. Attention to the effectiveness of placement procedures. Though this could be considered part of the first characteristic of successful programs, it received so much attention from all of the universities that we have elevated it to the level of a separate point. These universities evaluate and adjust their placement procedures on an annual basis. We also found a great deal of attention paid to those students near the cut-off, paying particular attention to programs in support of those allowed into Calculus I but most at risk and working with those who did not quite make the cut so that they were placed in programs that addressed their actual needs.
  3. Coordination of instruction, including the building of communities of practice. As Tucker reported in 1996, “There is a great diversity of instructional and curricular approaches, varying from one visited department to another, and even varying within a single department.” We found this, but we also found that those teaching Calculus were in regular communication with the other instructors of this class. Of course, where classes were taught by graduate teaching assistants, there was much tighter coordination of instruction. In all cases, we found that common exams were used. The simple act of creation of such exams fostered communication among those teaching the course. In some cases, communication about teaching was much more intentional, sharing innovative pedagogies, assignments, and approaches to particular aspects of the curriculum. In all cases there was also a course coordinator, a position that was not rotating but a more or less permanent position with commensurate reduction in teaching load.
  4. Construction of challenging and engaging courses. This is reflected in an observation that Tucker made in 1996: “Faculty members communicate explicitly and implicitly that the material studied by their students is important and that they expect their students to be successful in mathematical studies.” It also is the first example of effective educational practice in Student Success in College [Kuh et al, 2010, p 11]: “Challenging intellectual and creative work is central to student learning and collegiate quality.” None of the successful programs we studied believed that one could improve retention by making the course easier. Instructors used textbooks and selected problems that required students to delve into concepts and to work on modeling-type problems, or even problems involving proofs. Interviews with students—most of whom had taken calculus in high school—revealed that they felt academically challenged in ways that went far beyond their high school courses. 
  5. Use of student-centered pedagogies and active-learning strategies. This is the second example of effective educational practice in [Kuh et al, 2010]: “Students learn more when they are intensely involved in their education and have opportunities to think about and apply what they are learning in different settings.” As the first author learned twenty years ago when he surveyed Calculus I students at Penn State [Bressoud, 1994], few students know how to study or what it means to engage the mathematics, and most take a very passive role when attending a lecture. Active-learning strategies force students to engage the mathematical ideas and confront their own misconceptions. The successful programs we studied made much greater use of group projects and student presentations. 
  6. Effective training of graduate teaching assistants. Graduate students play an important role in calculus instruction at all universities with doctoral programs, whether as teaching assistants in the breakout sections for large lectures or as the instructors of their own classes. The most successful universities have developed extensive programs for training, monitoring, and supporting these instructors. Running a successful training program is not a task that can be handed off to a single person. While there is always one coordinator, their effectiveness requires a core of faculty who are willing to participate in the graduate students’ training that takes place before the start of the fall term and to assist in visiting classes and providing feedback.
  7. Proactive student support services, including the fostering of student academic and social integration. This is a broad category that ranges from the building of a student-faculty community within the mathematics department to the specifics of support mechanisms for at-risk students. These are addressed in three of the effective practices identified in [Kuh et al, 2010]: “Student Interactions with Faculty Members,” “Enriching Educational Experiences,” and “Supportive Campus Environment.” The first is mentioned in [Tucker, 1996]: “Extensive student-faculty interaction characterizes both the teaching and learning of mathematics, both inside and outside of the classroom.” The universities we visited had rich programs of extra-curricular activities within the Mathematics Department. They also had a variety of responses to supporting at-risk students. These included stretching Calculus I over two terms to allow for supplemental instruction in precalculus topics, providing “fallback” courses for students who discovered after the first exam that they were in trouble in Calculus, and working with student support services to ensure that students who were struggling got the help they needed. There also were heavily utilized learning centers that attracted all students as places to gather, work on assignments, and get help as needed. Often, these were centers dedicated solely to helping students in Calculus. What was common among all of the successful calculus programs was attention to the support of all students and a willingness to monitor and adjust the programs designed to help them.
There were some dramatic differences between instruction at the doctoral universities that were selected for the case study visits and instruction at all doctoral universities (see Table 1). Where the section size facilitated this—at ETI, MTI, MPU, and one section of WPR—instructors made much less use of lecture and much more use of students working together, holding discussions, and making presentations. Three of the five have almost universal use of online homework, and a fourth uses it for half of the sections. Graphing calculators were allowed on exams in two of the five universities, though use was not consistent across sections. The most striking difference between these five universities and the overall survey was the number of instructors who ask students to explain their thinking.

Instructors at the case study sites still consider themselves to be fairly traditional (see Figure 2), though slightly less so than the national average. They also tend to agree with the statement, “Calculus students learn best from lectures, provided they are clear and well organized” (see Figure 3). Interestingly, not a single instructor at any of the case study sites strongly agreed with this statement. On the other hand, the instructors at the case study sites were slightly less likely to disagree with it. They clumped heavily toward mild agreement, suggesting an attitude of keeping an open mind and a willingness to try an approach that might be more productive.




References

Bressoud, D. 1994. Student attitudes in first semester calculus. MAA Focus, vol 14, pages 6–7. http://www.macalester.edu/~bressoud/pub/StudentAttitudes/StudentAttitudes.pdf

Kuh, G.D.,  J. Kinzie, J.H. Schuh, E.J. Whitt. 2010. Student Success in College: Creating Conditions that Matter. Jossey-Bass.

National Research Council. 2001. Adding It Up: Helping Children Learn Mathematics. Kilpatrick, Swafford, and Findell (Eds.). National Academy Press. http://www.nap.edu/catalog.php?record_id=9822

Tucker, A. 1995. Models that Work: Case Studies in Effective Undergraduate Mathematics Programs. MAA Notes #38. Mathematical Association of America.

Tucker, A. 1996. Models that Work: Case Studies in Effective Undergraduate Mathematics Programs. Notices of the AMS, vol 43, pages 1356–1358. http://www.ams.org/notices/199611/comm-tucker.pdf
 



Characteristics of Successful Programs in College Calculus is supported by NSF #0910240. The opinions expressed in this article do not necessarily reflect those of the National Science Foundation.

Sunday, December 1, 2013

MAA Calculus Study: Persistence through Calculus

A successful Calculus program must do more than simply ensure that students who pass are ready for the next course. It also needs to support as many students as possible to attain this readiness. And it must encourage those students to continue on with their mathematics. As I wrote in my January 2010 column, "The Problem of Persistence," just because a student needs further mathematics for the intended career and has done well in the last mathematics course is no guarantee that he or she will decide to continue the study of mathematics. This loss between courses is a significant contributor to the disappearance from STEM fields of at least half of the students who enter college with the intention of pursuing a degree in science, technology, engineering, or mathematics. Chris Rasmussen and Jess Ellis, drawing on data from MAA’s Calculus Study, have now shed further light on this problem. This column draws on some of the results they have gleaned from our data.

For the MAA Calculus Study, students were surveyed both at the start and end of the fall term in mainstream Calculus I. A student was classified as a persister if she or he indicated at the start of the term an intention to continue on to Calculus II and still held that intention at the end of the term. A student was classified as a switcher if she or he intended at the start of the term to continue on to Calculus II, but changed his or her mind by the end of the term.

Not all students completed both the start and end of term surveys. While 50% of all Calculus I students received an A or B in the course, A or B students accounted for 80% of those who completed both surveys. Almost all of the remainder received a C. This implies that our data reflect what happened to the students who were doing well in the class. Of the students who started the term with the intention of taking Calculus II (74% of the students who answered both surveys), 15% turned out to be switchers. Less than 2% of all Calculus I students started with the expectation that they would not continue on to Calculus II but changed their minds by the end of the course.

The rates of switchers varied considerably. Women were far more likely to switch (20%) than men (11%). Those at large research universities were also more likely to switch (16%), particularly if they were taught by a graduate teaching assistant (19%). Rates varied by intended major, from a low of 6% switchers for those headed into engineering to 23% for pre-med majors and 27% for business majors taking mainstream calculus.

Classroom instruction had a significant effect on switcher rates (see Figure 1). "Good Teaching" reflects the collection of highly correlated observations described in this column in March 2013, "MAA Calculus Study: Good Teaching." "Progressive Teaching" refers to those practices described in the following column from April, "MAA Calculus Study: Progressive Teaching." Good Teaching is most important. In combination, Good and Progressive Teaching can significantly lower switcher rates.

Figure 1.

Our study offered students who had chosen to switch out a variety of reasons from which they could select any with which they agreed. Just over half reported that they had changed their major to a field that did not require Calculus II. A third of these students, as well as a third of all switchers, identified their experience in Calculus I as responsible for their decision. It also was a third of all switchers who reported that the reason for switching was that they found calculus to require too much time and effort.

This observation was supported by other data from our study that showed that switchers visit their instructors and tutors more often than persisters and spend more time studying calculus. As stated before, these are students who are doing well, but have decided that continuing would require more effort than they can afford.

I am concerned by these good students who find calculus simply too hard. As I documented in my column from May 2011, "The Calculus I Student," these students experienced success in high school, and an overwhelming majority had studied calculus in high school. They entered college with high levels of confidence and strong motivation. Their experience of Calculus I in college has had a profound effect on both confidence and motivation.

The solution should not be to make college calculus easier. However, we do need to find ways of mitigating the shock that hits so many students when they transition from high school to college. We need to do a better job of preparing students for the demands of college, working on both sides of the transition to equip them with the skills they need to make effective use of their time and effort.

Twenty years ago, I surveyed Calculus I students at Penn State and learned that most had no idea what it means to study mathematics. Their efforts seldom extended beyond trying to match the problems at the back of the section to the templates in the book or the examples that had been explained that day. The result was that studying mathematics had been reduced to the memorization of a large body of specific and seemingly unrelated techniques for solving a vast assortment of problems. No wonder students found it so difficult. I fear that this has not changed.

Friday, November 1, 2013

An International Comparison of Adult Numeracy

This past October, the Organization for Economic Cooperation and Development (OECD) released the first results from its survey of adult skills, OECD Skills Outlook 2013 [1]. It presents more evidence that the United States is lagging behind other economically developed nations in building a quantitatively literate workforce. A rich source of data, the report is unusual in its focus on the numerical skills of adults, covering ages 16 through 65, and on its parallel investigations of literacy and "problem solving in technology-rich environments." Intriguingly, its data suggest that—although their numerical skills rank near the bottom—U.S. workers consider the numerical demands of their work and their ability to handle those demands to be greater than do workers in most other developed countries.

The OECD measured numerical proficiency at five levels:
1.      Able to perform basic calculations in common, concrete situations.
2.      Can identify and act on mathematical information in a common context.
3.      Can identify and act on mathematical information in an unfamiliar or complex context.
4.      Can perform multi-step tasks and work with a broad range of mathematical information in unfamiliar or complex contexts.
5.      Can understand complex mathematical or statistical ideas and integrate multiple types of mathematical information where interpretation is required.

As an illustration of a task at level 3 (from the Reader’s Companion to the report [2, p. 30]): In 2005, the Swedish government closed its Barseb├Ąck nuclear power plant, which was generating 3,572 GWh (Gigawatt hours) of power per year. Given that a wind power station generates about 6,000 MWh (Megawatt hours) of power per year, that 1 MWh = 1,000,000 Wh (Watt hours), and 1 GWh = 1,000,000,000 Wh, how many wind power stations would be needed to replace the Barseb├Ąck plant?

Now the discouraging news. Only just over a third, 34.4%, of U.S. adults were capable of solving such a problem. In many OECD countries, over half the working age population was numerate at level 3 or above, including Austria (50.8%), the Czech Republic (51.9%), Finland (57.8%), Japan (62.5%), Norway (54.8%), the Slovak Republic (53.7%), and Sweden (56.6%). Germany came in just under at 49.1%. South Korea, at 41.4%, suffered from the fact that many of its older workers, especially those over 45, have skills that are far below those of younger Koreans. Other countries in which less than 40% of the population reached level 3 include Poland (38.9%), France (37.3%), and Ireland (36.4%). Only Italy (28.9%) and Spain (28.6%) came in lower than the United States. [1, Table A2.5, p. 262]

While the top 5% of U.S. adults are capable of working at level 4, the scores at the 95th percentile in the United States were well below those in most other OECD countries. The exceptions were France, Ireland, Italy, South Korea (again the unequal opportunity effect for older workers), Poland, and Spain. Only Finland had more than 2% of the adult population capable of working at level 5. In the United States, 0.7% of the adult population was capable of answering questions at level 5. [1, Table A2.8, p. 266]

The OECD data also reveal that the weakness of U.S. adults is not a recent phenomenon. The report separates numeracy skill levels by age decade: 16–24, 25–34, 35–44, 45–54, and 55–65. The United States is near the bottom of every age cohort, though it stayed above Italy and Spain and managed to climb above France and Ireland for adults 45 and older and above Poland and South Korea for adults 55 and older. [1, Table A3.2 (N), p. 272]

Given the low marks on numerical ability, it is interesting that when U.S. workers were asked whether they need to use their numeracy skills at work, the percentages were near the top of the OECD list. All of the following comparisons are for workers in the top 25% in terms of numeracy level. In the United States, 28.8% of these workers said that they need to use their numeracy skills frequently, as opposed to 28.0% in Finland, 26.7% in Germany, and only 17.7% in Japan. Only the Czech Republic at 30.0% and the Slovak Republic at 29.4% reported higher rates of frequent use of numerical skills. [1, Table A4.3, p.  303]

In addition, U.S. workers are more inclined to consider their numeracy skills to over qualify them for the requirements of their job. In the United States, 9.4% of workers considered their numeracy skills greater than the requirements of their job. In Italy, it was 12.6%; in Spain, 15.8%. In contrast, only 7.9% of the workers in Japan and 7.0% in Finland considered their numeracy skills to be greater than the demands of their job. [1, Table A4.25, p. 358] Across the OECD countries, there is a strong negative correlation between numerical ability and the perception of how well one has mastered the numerical skills required for one’s work.

That should be the most troubling aspect of this study.

 


[1] OECD (2013), OECD Skills Outlook 2013: First Results from the Survey of Adult Skills, OECD Publishing. http://dx.doi.org/10.1787/9789264204256-en

[2] OECD (2013), The Survey of Adult Skills: Reader’s Companion, OECD Publishing.


[3] The OECD countries in the survey were Australia, Austria, Czech Republic, Denmark, Estonia, Finland, France, Germany, Ireland, Italy, Japan, Korea, Netherlands, Norway, Poland, Slovak Republic, Spain, Sweden, United States, and three subnational entities: Flanders (Belgium), England (UK), and Northern Ireland (UK). Some data are also presented for Cyprus and the Russian Federation.

Tuesday, October 1, 2013

Evidence of Improved Teaching

Last December I discussed the NRC report, Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. One of its themes is the importance of the adoption of “evidence-based teaching strategies.” It is hard to find carefully collected quantitative evidence that certain instructional strategies for undergraduate mathematics really are better. I was pleased to see two articles over the past month that present such evidence for active learning strategies.

One of the articles is the long-anticipated piece by Jerry Epstein, "The Calculus Concept Inventory—Measurement of the Effect of Teaching Methodology in Mathematics" which appeared in the September 2013 Notices of the AMS [1]. Because this article is so readily available to all mathematicians, I will not say much about it. Epstein’s Calculus Concept Inventory (CCI) represents a notable advancement in our ability to assess the effectiveness of different pedagogical approaches to basic calculus instruction. He presents strong evidence for the benefits of Interactive Engagement (IE) over more traditional approaches. As with the older Force Concept Inventory developed by Hestenes et al. [2], CCI has a great deal of surface validity. It measures the kinds of understandings we implicitly assume our students pick up in studying the first semester of calculus, and it clarifies how little basic conceptual understanding is absorbed under traditional pedagogical approaches. Epstein claims statistically significant improvements in conceptual understanding from the use of Interactive Engagement, stronger gains than those seen from other types of interventions including plugging the best instructors into a traditional lecture format. Because CCI is so easily implemented and scored, it should spur greater study of what is most effective in improving undergraduate learning of calculus.

The second paper is "Assessing Long-Term Effects of Inquiry-Based Learning: A Case Study from College Mathematics" by Marina Kogan and Sandra Laursen [3]. This was a carefully controlled study of the effects of Inquiry-Based Learning (IBL) on persistence in mathematics courses and performance in subsequent courses. They were able to compare IBL and non-IBL sections taught at the same universities during the same terms.

IE and IBL describe comparable pedagogical approaches. Richard Hake defined IE as
“… those [methods] designed at least in part to promote conceptual understanding through interactive engagement of students in heads-on (always) and hands-on (usually) activities which yield immediate feedback through discussion with peers and/or instructors.” [4]
IBL does this and also is expected to incorporate a structured curriculum that builds toward the big ideas, a component that may or may not be present in IE. For the Kogan and Laursen study, IBL was a label that the universities chose to apply to certain sections. The trained observers in the Kogan and Laursen study found significant differences between IBL and non-IBL sections. They rated IBL sections “higher for creating a supportive classroom atmosphere, eliciting student intellectual input, and providing feedback to students on their work” than non-IBL sections. IBL sections spent an average of 60% of the time on student-centered activities; in non-IBL sections the instructor talked at least 85% of the time.

Kogan and Laursen compared IBL and non-IBL sections for three courses:
  • G1, the first term of a three-term sequence covering multivariable calculus, linear algebra, and differential equations, taken either in the freshman or sophomore year;
  • L1, a sophomore/junior-level introduction to proof course; and
  • L2, an advanced junior/senior-level mathematics course with an emphasis on proofs.

For L1 and L2, students did not know in advance whether they were enrolling in IBL or non-IBL sections. The IBL section of G1 was labeled as such. In all cases, the authors took care to control for discrepancies in student preparation and ability.

IBL had the least impact on the students in the advanced course, L2. IBL students had slightly higher grades in subsequent mathematics courses (2.6 for non-IBL, 2.8 for IBL) and took slightly fewer subsequent mathematics courses (1.5 for non-IBL, 1.4 for IBL).

For the introduction to proof course, L1, IBL students again had slightly higher grades in the following term (2.8 for non-IBL, 3.0 for IBL). There were statistically significant gains (p < 0.05) from IBL in the number of subsequent courses that students took and that were required for a mathematics major, both for the overall population (0.5 for non-IBL, 0.6 for IBL) and, especially, for women (0.6 for non-IBL, 0.8 for IBL).

For L1, the sample size was large enough (1077 non-IBL, 204 IBL over seven years) to investigate persistence and subsequent performance broken down by student overall GPA, recorded as low (< 2.5), medium (2.5 to 3.4), or high (> 3.4). For the non-IBL students, differences in overall GPA were reflected in dramatic differences in their grades in subsequent mathematics courses required for the major, all statistically significant at p < 0.001. Low GPA students averaged 1.96, medium GPA students averaged 2.58, and high GPA students averaged 3.36. All three categories of IBL students performed better in subsequent required courses, but the greatest improvement was seen with the weakest students. Taking this course as IBL wiped out much of the difference between low GPA students and medium GPA students. It also decreased the difference between medium and high GPA students in subsequent required courses. For IBL students, low GPA students averaged 2.43, medium GPA students averaged 2.75, and high GPA students averaged 3.38 in subsequent required courses. See Figure 1.
Figure 1: Average grade in subsequent courses required for the major following introduction to proof class taught either as non-IBL or IBL.
While the number of subsequent courses satisfying the requirements for a mathematics major was higher for all students taking the IBL section of L1, here the greatest gain was among those with the highest GPA. For low GPA students, the number of courses was 0.50 for non-IBL and 0.51 for IBL; for medium GPA the number was 0.53 for non-IBL, 0.62 for IBL; and for high GPA the number was 0.49 for non-IBL, 0.65 for IBL. See Figure 2.
Figure 2: Average number of subsequent courses taken and required for the major following introduction to proof class taught either as non-IBL or IBL.
For the first course in the sophomore sequence, G1, IBL did have a statistically significant effect on grades in the next course in the sequence (p < 0.05). The average grade in the second course was 3.0 for non-IBL students, 3.4 for IBL students. There also was a modest gain in the number of subsequent mathematics courses that students took and that were required for the students’ majors: 1.96 courses for non-IBL students, 2.09 for IBL students.

These have been the highlights of the Kogan and Laursen paper. Most striking is the very clear evidence that IBL does no harm, despite the fact that spending more time on interactive activities inevitably cuts into the amount of material that can be “covered.” In fact, it was the course with the densest required syllabus, G1, where IBL showed the clearest gains in terms of preparation of students for the next course.

IBL is often viewed as a luxury in which we might indulge our best students. In fact, as this study demonstrates, it can have its greatest impact on those students who are most at risk.
 


[1] J. Epstein. 2013. The Calculus Concept Inventory—Measurement of the Effect of Teaching Methodology in Mathematics. Notices of the AMS 60 (8), 1018–1026. http://www.ams.org/notices/201308/rnoti-p1018.pdf

[2] D. Hestenes, M. Wells, and G. Swackhamer. 1992. Force concept inventory. Physics Teacher 30, 141–158. http://modelinginstruction.org/wp-content/uploads/2012/08/FCI-TPT.pdf

[3] M. Kogan and S. Laursen. 2013. Assessing Long-Term Effects of Inquiry-Based Learning: A Case Study from College Mathematics. Innovative Higher Education 39 (3). http://link.springer.com/article/10.1007/s10755-013-9269-9

[4] R.R. Hake. 1998. Interactive engagement versus traditional methods: A six-thousand student survey of mechanics test data for physics courses. American J. Physics 66 (1), 64–74. http://www.physics.indiana.edu/~sdi/ajpv3i.pdf