1. OVERVIEW

Although software has long been available for doing statistical analysis, the role of technology in teaching and learning statistics is still evolving. Computers and calculators reduce the computational burden and allow more extensive exploration of statistical concepts. The availability of powerful computing tools has also led to newer methods of analyzing data and graphically exploring data. During the past 20 years, since personal computers became available in homes and schools, the developments in educational technology have progressed at an accelerated pace (Ben-Zvi, 2000).

The types of technology that are now being used in statistics instruction fall into one or more of the following categories:- statistical packages and spreadsheets for analyzing data and constructing
visual representations of data;

- multimedia materials to teach, tutor, and/or test students’ statistical
knowledge and skills;

- Web or computer-based tools, including simulations, to demonstrate and
visualize statistical concepts;

- graphing calculators for computation, graphing or simulation;

- programming languages that students can use to set up more complicated simulations or numerical analyses.

- Students' active construction of knowledge, by 'doing' and 'seeing'
statistics.

- Opportunities for students to reflect on observed phenomena.

- The development of students’ metacognitive capabilities, that is, knowledge about their own thought processes, self-regulation, and control.

2. THE INTRODUCTORY NON-CALCULUS STATISTICS COURSE

Like the calculus reform movement in college mathematics programmes, a statistics reform movement has called for changes in the introductory statistics course. Changes are recommended in course content (e.g., more emphasis on real data, less emphasis on formal probability and mathematical formulae), pedagogy (use of active learning, cooperative groups, and real data) and student assessment (use of alternative approaches including student projects, reports, and writing assignments). In addition, instructors are urged to incorporate more technology in their courses, to represent, analyze, and explore data, as well as to illustrate abstract concepts (Cobb, 1992).

The 1990 Mathematics Undergraduate Program Survey (CBMS) showed that the enrolment of students in statistics courses taught in departments of mathematics in four-year colleges and universities and in two-year college mathematics programmes, more than doubled from 1970 to 1990 (from 76,000 to 179,000 students). Recently, many institutions have added the requirement of a data-oriented, or quantitative-literacy course in their core curriculum, increasing the statistics enrolment, so that now more sections of statistics courses are offered than calculus courses in colleges in the USA. The 1995 CBMS study reported 3530 sections of elementary statistics taught in mathematics departments, 820 in statistics departments, and 2566 sections taught in two-year colleges. Numerous elementary statistics courses are also taught in other disciplines, such as psychology, business, education, sociology, and economics.

In a recent survey of the introductory statistics course, Garfield (2000) found that most students in these courses are required to use some type of technology, and most of the faculty surveyed anticipated increasing or changing the use of technology in their courses. The type of technology is often different depending on the type of institution. Students in two-year college courses are more likely to use graphing calculators (for computations using small data sets). Students in four-year colleges or universities are more likely to use a statistical software or spreadsheet program.

*2.1 Statistical Software Packages *

A variety of different software packages are used in introductory statistics courses, several of which are reviewed by Lock (1993). The most widely used statistical software program in introductory statistics courses is Minitab (Joiner and Ryan, 2000), which is perceived as easy to learn and use, and is available on both Mac and PC platforms. Data are entered in a data window and pull-down menus allow students to easily graph or analyze data. In addition, ‘macros’ are small programmes that instructors or students can create to generate data or run simulations. Minitab is incorporated into over 250 textbooks directly or as a special supplement and has been used in over 2000 colleges and universities around the world (Shaughnessy, Garfield, and Greer, 1997).

Data Desk (Velleman, 1998) offers a strong, visual approach to data exploration and analysis, integrating dynamic graphics with tools to model and display data. The intent of this program is to allow students to explore data with an open mind, no preconceptions, and no formal hypotheses, in the spirit of detective work. Users are asked to consider the question: "Can I learn something useful?" Data Desk provides many unique tools that allow students to look for patterns, ask more detailed questions about the data, and ‘talk’ with the program about a particular set of data. Despite the many unique features of this software, it has not been widely used in introductory courses. Data Desk is sometimes difficult for students who are accustomed to a spreadsheet system for data entry, and some instructors do not like the non-standard method of manipulating data.

A newer software package, developed to help students explore and learn statistics, is Fathom. Originally called ‘Dataspace’, Fathom is described as a dynamic computer learning environment for data analysis and statistics based on dragging, visualization, simulation, and networked collaboration (Finzer and Erickson, 1998). Although originally aimed at middle school and high school level students, Fathom is beginning to be used in college level statistics classes as well.Although not actually a statistics program, the Excel software, produced by Microsoft, is widely available in most computing labs and on many personal computers. This spreadsheet software has add-ons that allow the software to perform some statistical analyses. However, concerns have been expressed about using this software instead of more authentic statistics software (e.g., McCullogh & Wilson, 1999).

In addition to graphing calculators and statistical or spreadsheet software, technology resources available on the World Wide Web are increasing daily. Several Web sites have been established that make links to other sites to assist instructors of introductory statistics courses as well as their students. For example, Robin Lock’s Web site organizes and links Web resources by categories and awards medals to the ones he finds most useful or of the highest quality. His categories include:

- On-line Course Materials (e.g., course syllabi and materials for
particular instructors);

- On-line Textbooks (some are free and some are available for a
fee);

- JAVA applets (a platform-independent Web programming language which is
used to develop interactive demonstrations that can be accessed via any
JAVA-capable browser). Examples include randomly generated scatterplots from
which students are asked to guess correlations, dynamic regression-lines that
change when points are added or removed, graphical visualizations illustrating
the power of a statistical test when different parameters are varied, and
histograms where students manipulate the bin width to see the effect on the
overall pattern;

- Data sets or links to repositories of data (such as the Data and Story
Library that contains data sets and accompanying stories along with analyses,
and government agencies such as the Census bureau, that collect and store
data);

- Miscellaneous sites (such as the electronic
*Journal of Statistics Education*, sites that consist of links to sample test and quiz questions, or collections of links and resources for teaching an introductory level course).

The development of the World Wide Web has produced unprecedented means for instructors to easily share their ideas on ways to improve the teaching of statistics (Lock, in press). The links on Robin Lock’s Web site represent just a sample of resources that are currently available. He encourages faculty not to be daunted by the volume of on-line materials but to search and try out resources that may greatly enhance students’ learning.

An example of a statistics course that is based exclusively on the World Wide Web is The Visualizing Statistics project which has developed an on-line introductory statistics course,Velleman and Moore (1996) define multimedia as a computer-based system that combines several components such as text, sound, video, animation, and graphics. Since their article appeared, which described the "promises and pitfalls" of using multimedia in statistics courses, several types of multimedia materials have been developed and are being used in various ways in statistics courses.

*ActivStats* (Velleman, 2000), is a multimedia resource for students to learn
basic statistics. It can be used by itself or along with a textbook. This CD ROM
which runs on either Mac or PC platforms, contains videos of uses of statistics
in the real world, mini-lectures accompanied by animation, tools similar to Java
applets to illustrate concepts interactively, and a student version of Data Desk. Lessons on the CD ROM instruct
students in how to use the software, and many homework exercises are included
that have data sets formatted for analysis with Data Desk. Other versions of
*ActivStats* may be used with software programmes such as Excel or SPSS.

Cumming and Thomason (1998) describe
*StatPlay* as multimedia designed to help students develop a sound
conceptual understanding of statistics and to overcome misconceptions. These
materials consist of dynamically linked simulations and microworlds, 'play
grounds' and estimation games, along with recorded mini-lectures or directions
on analyzing data or using the software tools.

Another unique CD ROM is the *Electronic
Companion to Statistics*
(Cobb and Cryer, 1997). In contrast to *ActivStats* and
*StatPlay*, this is a study guide for students that provides interactive
illustrations and exercises to present examples of the different topics covered.
It allows students to explore the relationships between statistical concepts
using concept maps, and includes a variety of self assessment items and
animations to help students review or better understand important
concepts.

Many other multimedia resources are currently being developed around the world, several of which were described at the Fifth International Conference on Teaching Statistics (Pereira-Mendoza, 1998).

*2.4. Stand-alone Simulation Software*

Some concepts in statistics are particularly challenging for students to learn, such as the idea of a sampling distribution and the Central Limit Theorem. The Tools for Teaching and Assessing Statistical Inference project (Garfield, delMas and Chance, 2001), has developed some simulation programmes that are easy for students to use, and are accompanied by structured lab activities and assessment instruments. Although there are other simulation tools on the Internet or on CD ROM (such as ActivStats), this software is unique in that it provides more detail and flexibility. It allows students to manipulate a variety of populations (based on discrete or continuous data), and draw samples from these populations for different sample sizes. Templates are provided to compare the sampling distribution to the population and to a normal distribution, as well as to calculate probabilities. The tool can be used to illustrate confidence intervals and p-values as well.

The RESAMPLING STATS Software (Simon and Bruce, 1991) is a software package that offers an easy-to-use, powerful tool to conduct repeated simulations (including the bootstrap), calculate test statistics, and analyze and view the results.

*2.5 Other technological resources*

Two other types of technological resources that are used in introductory
statistics classes are video and email. Moore (1993) discusses strengths and
weaknesses of using videos in statistics classes. The *Against All
Odds* video series produced by COMAP
is an excellent set of videos that illustrate real-world applications of
statistical topics. A shorter series of segments of these videos were produced
and distributed as *Decisions Through Data*.
Some of these videos are now included in the multimedia materials described
above. Additional videos are described in the section on Quantitative Literacy
Courses.

Email is a technological tool that is impacting all courses, including statistics courses. For example, Chance Magazine (1998) reported on the use of email in introductory statistics courses to foster outside-of-class communications between the student and instructor and between students. This type of support network appears to be especially relevant in a course such as statistics, which students often enter with much trepidation and unease. Hyde and Nicholson (1998) extend this communication to intercontinental collaboration by linking statistics classrooms in different countries, allowing students to collect and analyze data comparing themselves to their international peers.

3. THE ROLE OF TECHNOLOGY IN A PROBABILITY COURSE

As described earlier, the arrival of the computer in the classroom and the ability to analyze real data in the elementary statistics course has completely changed the way this course is taught in many colleges and universities. This has resulted also in a change in the way probability is taught in these statistics courses. The types of technology now used in these courses simulate simple experiments such as coin tossing, rolling dice, and choosing random samples from a population. Minitab has the ability to write macros to simulate any kind of chance experiment. The new statistical package Fathom can also simulate a wide range of experiments.

Statistics courses have taken advantage of this by having their students learn the basic probability ideas such as the Binomial distribution, the law of large numbers, and the central limit theorem, through simulations. This means that the statistics teachers do not have to spend time on the mathematics of probability associated with the study of combinatorics, sample spaces and formal properties of probability measures. A separate probability course still develops the mathematics of probability but instructors in these courses also take advantage of simulation to help the students better understand theoretical results and to solve problems which do not lend themselves to mathematical solutions.

Teachers of probability courses are beginning to take advantage of powerful symbolic mathematical software such as Mathematica, Maple, and Matlab. However, those who have tried using these packages have had mixed success. Simulating a chance experiment or solving a combinatorial problem to calculate a probability often requires writing a procedure and students find this difficult. They may spend so much time trying to write the procedure, that they end up feeling that they did not learn enough from this effort to justify the time they put in.

One area where the students can appreciate the value of these packages is in the study of Markov chains. The ability to raise matrices to powers and to solve equations saves the students enormous amount of work and makes Markov Chains come to life. Of course the teacher can use these packages to write procedures to illustrate basic probability results. Elliot Tanis shows on his Web site how this can be done for the Central Limit Theorem.

Traditional probability courses rarely deal with real life data in their courses. Perhaps the success of the statistics reform will convince probabilists that it is very natural and rewarding to show students how their probability models fit real data. Technology facilitates the introduction of real data into probability models, helping students become more aware of the role of variation in the real world.

*3.2 World Wide Web Resources*

The World Wide Web should play an important role in the
future of the probability course. The Web has been
particularly successful in leading to the development and dissemination of
interactive text materials. An interactive probability book by Siegrist at
his '*Virtual Laboratories in
Probability and Statistics' *has applets to go with each
topic. Several of the on-line interactive statistics books also have chapters on
probability and make good use of applets (e.g., HyperStat by David Lane at Rice
University). Individual applets that are particularly useful in teaching a
probability course are: Binomial probabilities, Normal approximation to the
binomial distribution, Central Limit Theorem, and Brownian
motion.

Probability is full of surprises, and Susan Holmes has a Web site titled "Probability by Surprises." This site includes applets related to interesting probability paradoxes and problems such as: the birthday problem, the box-top problem, the hat-check problem and others. Alexander Bogomolny also provides a discussion of a number of puzzling probability problems illustrated with applets at his Web site. His list includes: Benford’s law, the Buffon needle problem, Simpson’s paradox, and Bertrand’s paradox.

More traditional probability books are also available on the
Web. *Introduction
to Probability*
by Grinstead and Snell (1997) is an example of such a book. This book
includes applets as well as Mathematica and Maple programmes illustrating basic
probability concepts. Waner and Costenoble have an on-line probability book
‘*Calculus Applied
to Probability and Statistics for Liberal Arts and Business Majors*’
which deals exclusively with continuous probability.

Finally, Phil Pollett maintains a Web site called ' *The Probability
Web*' that provides a comprehensive resource for links to various
probability resources on the Web.

4. TECHNOLOGY IN QUANTITATIVE LITERACY COURSES

Over the past 10 years a new course is increasingly being offered in mathematics and statistics departments, often referred to as Quantitative Literacy or Statistical Literacy. This course typically attempts to help students develop an understanding of quantitative information used in the world around them, including basic concepts in statistics and probability. Gal (2000) describes what a course on statistical literacy should be and how it differs from a standard introductory statistics course. He argues that such a course should be aimed at consumers of statistics rather than producers of statistics. The basic statistical concepts taught are not different but the emphasis should be different. For example, a much broader discussion of types of experiments is essential to understanding reports in the news on medical experiments. Students need to understand different interpretations of probabilities (subjective and objective) and risk (relative and absolute) etc. than would normally be taught in a first statistics course. Gal has started a web site, "Adult Numeracy: Research and Development Exchange," which provides information on statistical literacy for adults and students.

The course called '*Chance' * at Dartmouth,
developed with several other colleges in 1992, is an example of this new type of
quantitative literacy course. This course was designed to help students
understand statistics used in the media and utilized articles from the current
news to focus class discussions each day. The Chance Web site provides resources
for teaching a Chance course or a standard course enriched with discussion of
news items that include reference to ideas of chance. The Chance project
produces a monthly electronic newsletter called Chance News that abstracts, and
provides discussion questions for, current issues in the news that use
probability or statistics concepts. This newsletter is sent out by email and
archived on the Chance Web site.

Many major newspapers have Web versions and are a good source of articles to use in a quantitative literacy course. Search engines such as Lexis-Nexis may be used to locate articles on particular topics such as DNA fingerprinting, weather forecasting, polls and surveys, and lotteries. Articles in science and medical journals are also available on Web sites where the full text of the articles may be accessed.

Another source of information on how probability and statistics is used in the real world is provided by the videos of the lectures from the Chance Lecture Series available on the Chance Web site. In this series, experts in areas of probability and statistics that are mentioned regularly in the news provide lectures on their topic. Some of these are David Moore from the Gallup Organization on problems in polling, Arnold Barnett from MIT on estimating the risk of flying, Bruce Weir on the use of DNA fingerprinting in the news, etc. The lectures are also available on a CD ROM for those with slow Internet connections.

The Web site ‘5. TECHNOLOGY IN
OTHER STATISTICS COURSES

This section focuses on more ‘advanced’ introductory statistics courses. By this we mean introductory courses with a calculus prerequisite (often serving students majoring in statistics, mathematics, science, or engineering) as well as second semester introductory courses. Similar to the statistics reform movement in the introductory course, a slower movement has addressed the content and pedagogy of these courses, with technology again playing a central role. While progress has been slower there is also tremendous potential for innovative uses of technology in these courses.

Web applets and computer packages now allow a greater emphasis on the conceptual ideas underlying the statistical methods. For example, students can use Minitab or a Java program to simulate the sampling distribution of the least squares regression slope or the chi-square statistic. This expands the types of simulations students use in other statistics courses to study more complicated techniques. Students can use Excel to graph and numerically estimate the maximum likelihood estimator, while easily changing parameter values to see how the maximum likelihood function updates. Students are therefore able to focus on the function and less on the calculus involved.

S-PLUS is another computer package that allows students to program routines and perform simulations. Similar to programming in the C language, students can be asked to write simple scripts for performing analyses. SPlus also offers exemplary graphical techniques for exploring more complicated data sets. A similar, but free, student version, R, can be downloaded from the Web. These more advanced courses have also seen an increase in the number and size of data sets that can be analyzed, due to the availability of technological resources. For example,6. RESEARCH ON THE ROLE OF TECHNOLOGY IN PROBABILITY AND STATISTICS COURSES

Although it is apparent to statistics educators that technology has had a huge impact on the content of current courses as well as the types of experiences students have in these courses, there is little research to document the actual impact of technology on student learning. There is also lack of information evaluating the effectiveness of particular types of software or activities using technology. Biehler (1997) cautions that statistics educators need a system to critically evaluate existing software from the perspective of educating students, and to produce future software more adequate both for learning and doing statistics in introductory courses.

A round table conference sponsored by the International Association for Statistics Education was convened in 1996 to examine research on the role of technology in learning statistics. Although there was little empirical research to report at that time on the impact of technology on student learning, a main outcome of this conference was to identify important issues and research questions that had not yet been explored (Garfield and Burrill, 1997). Two years later, at the Fifth International Conference on Teaching Statistics, there were 35 papers presented in seven different categories, under the topic heading "The Role of Technology in the Teaching of Statistics" (Pereira-Mendoza, 1998). Additional papers on the use of technology were in many of the other topic sessions as well. Most of these papers discussed ways technology can be used in courses, rather than offering data on learning outcomes. However, empirical studies (e.g., Finch and Cumming, 1998; Shaughnessy, 1998) provided valuable information on how technological tools can both improve student learning of particular concepts as well as raise new awareness of student misconceptions or difficulties (e.g., Batanero and Godino, 1998). While controlled experiments are usually not possible in educational settings, qualitative studies are increasingly helpful in focusing on the development of concepts or the use of skills that technology is intended to facilitate. Biehler (1998) used videos and transcripts to explore students’ thinking as they interacted with statistical software. Lee (2000) and Miller (2000) are examples of qualitative studies of how a course that integrates technological tools can support a student-centered, constructivist environment for statistics education. delMas, Garfield, and Chance (1999) provide a model of collaborative classroom research to study the impact of simulation software on students’ understanding of sampling distributions.7. SUMMARY, CONCLUSIONS, RECOMMENDATIONS

Thistead and Velleman (1992), in their summary of technology in teaching statistics, cite four obstacles that can cause difficulties when trying to incorporate technology into college statistics courses. These include equipment (e.g., adequate and updated computer labs), software (availability and costs), projection (of computer screens in classrooms), and obsolescence (of hardware, software, and projection technologies). Eight years later, we can see increased availability of computers and access to graphing calculators, updated and more widely available software, often via CD’s bundled with textbooks or on the World Wide Web, and new methods of projecting computer screens such as interactive White Boards.

The ways that these technological resources appear to have changed the teaching of probability and statistics include:- Less of a focus on computations. This frees students to spend more time
focusing on and understanding the concepts. There is also less focus on
manipulating numbers, or on exercises using only small and/or artificial data
sets.

- Improved visualization of statistical concepts and processes. Students are
better able to ‘see’ the statistical ideas, and teachers are better able to
teach to students who are predominantly visual learners.

- Dynamic representations and analyses. Discussions or activities may focus
on 'what if?' questions by changing data values or manipulating graphs and
instantly seeing the results.

- Increased use of simulations. Simulations provide an alternative to using
theoretical probability when teaching introductory statistics, better motivate
probability theory when teaching probability, and offer better ways to convey
ideas of long-run patterns.

- Empowering students as users of statistics. Students are able to solve
real problems and use powerful statistical tools that they may be able to use
in other courses or types of work. This allows them to better understand and
experience the practice of statistics.

- Facilitating discussions about more interesting problems by using
technological tools to explore interesting data sets (which may be large and
complicated), often accessed from the Internet.

- Allowing students to do more learning on their own, outside of class,
using Web-based or multimedia materials. This frees the instructor to have
fewer lectures during class and to spend more time on data analysis activities
and group discussions.

- Making the course relevant and connected to everyday life. Web resources make it easy to connect course material to real world applications and problems through data sets, media resources, and videos.

Finally, probability and statistics are specialized subjects, and many colleges may not have a faculty member whose expertise is in these areas. The methods being developed for distance learning, which incorporate many innovative uses of technology, may allow schools to share resources and make a high quality probability or statistics class at one university available to others. However, with increased distance learning courses, it is also unclear as to how much of a course can be taught exclusively using technology, what the appropriate role of an instructor should be, and how much emphasis should still be placed on students generating calculations and graphs by hand.

REFERENCES

Batanero, C. and Godino, J. D. (1998). Understanding graphical and numerical representations of statistical association in a computer environment. In L. Pereira-Mendoza (Ed.),Chance, B. (1998). Incorporating
a Listserve into Introductory Statistics Courses. *1998 Proceedings of the
Section on Statistical Education*, American Statistical Association.

Cobb, G.W. (1992). Teaching
Statistics. In L. Steen (Ed.), *Heeding the Call for Change: Suggestions for
Curricular Action*, pp. 3-43. Washington, DC: Mathematics Association of
America.

delMas, R., Garfield, J., and Chance, B. (1999). A model of
classroom research in action: Developing simulation activities to
improve students’ statistical reasoning*. Journal of Statistics Education *7(3)
(electronic journal),

Derr, J. (2000). *Statistical Consulting: A Guide to
Effective Communication*. Pacific Grove, CA: Duxbury Press.

Gal, I.
(2000). Statistical literacy: Conceptual and instructional issues.In D. Coben,
J. O'Donoghue, & G. FitzSimons, (Eds.), *Perspectives on adults learning
mathematics: Research and practice* (pp. 135-150). London:
Kluwer.

Garfield, J. (2000). A snapshot of introductory statistics. Paper presented at Beyond the Formula conference, Rochester, NY.

Garfield, J. and Burrill, G. (Eds.), (1997

Lock, R. H. (1993). A
comparison of five student versions of statistics packages*. T he American
Statistician*,* *Lock, R. H. (in press). A Sampler of WWW Resources for
Teaching Statistics. In T. Moore (Ed.), Teaching Statistics: Resources for
Undergraduate Instructors. Washington, D.C.: Mathematics Association of
America.

McCullough, B.D. and
Wilson, B. (1999). On the accuracy of statistical procedures in Microsoft Excel
97. *Computational Statistics and Data Analysis*, 31, 27-37.

Miller, J. (2000).
*The Quest for the Constructivist Statistics Classroom: Viewing Practice
Through Constructivist Theory*. Ph.D. Dissertation, Ohio State University

Moore, D. S. (1997). New
pedagogy and new content: the case of statistics*. International Statistical
Review*, 635, 123-165

Ramsey, F. and Schafer, D. (1997). *The Statistical Sleuth*. Pacific
Grove, CA: Duxbury Press.

Shaughnessy, J.M. (1998). Immersion in data handling: Using the chance-Plus
software with introductory college students.
In L. Pereira-Mendoza (Ed.), *Proceedings of the Fifth International
Conference on Teaching Statistics*, pp. 913-920. Voorburg, The Netherlands:
International Statistical Institute.

Shaughnessy, J. M., Garfield, J.B. and Greer, B. (1997). Data Handling. In A.
J. Bishop, K. Clements, C. Keitel, J. Kilpatrick and C. Laborde (Eds.),
*Intertional Handbook on Mathematics Education,* pp. 205-237. Dordrecht:
Kluwer Academic Publishers.

Velleman, P. and Moore, D.S. (1996). Multimedia for teaching statistics:
promises and pitfalls. *The American Statistician*, 50, 217-225.

*Joan Garfield*

*Beth Chance*

*California Polytechnic State University, California, USA*

bchance@calpoly.edu

*J. Laurie Snell*

*Dartmouth College *

jlsnell@dartmouth.edu