Melissa Hammann: Gender Comparison of Performance: ACT Test Score Analysis

(Ed.note: This text document was discussed in general at the July and September Evansville School Board meeting. I have selected a portion that addresses specifically the ACT portion of the comparison at Evansville High School. The report written by Melissa Hammann was part of an overall Board self examination of school performance that each board member contributed comments to. This was her analysis of the data.

I must caution readers that the data as well as what it means is still in discussion and this report is the beginning of the discussion, not the conclusion. Variation of performance can be the result of normal variation of ability between class years, curriculum "mismatch", possible effects of 4 period or 8 period day, and other factors. A special committee has been formed and a consultant has been hired to review the data and help determine what it might mean. Some data has been misaligned in translation to the web. If you seek all the data excel files, they are available from the School District. Thanks. Stay tuned to The Evansville Observer. )

-----------------------------------------------------------

ACT Trend Data, 2000-2007 (ACTTRENDS)

Please refer to Excel workbook ACTTRENDS while reading this part of the report. The raw data sheet is included for your review. Each ACT subject (Reading, English, Math and Science, plus the composite score) has been charted separately for Evansville boys and girls, with state data included for comparison. These graphs are entitled ACTBOYSREAD, ACTGIRLSREAD, etc. There are also 5 charts with all students graphed separately with color-coded trend lines included. The tabs for these graphs are labeled Ev vs State Reading Trends, Ev vs State English Trends, etc. The time domain began with the 2000-2001 school year and ends with the 2006-2007 school year.

As one surveys the historical data represented on the graphs, there is no denying that Evansville girls were on the short end of the educational stick as recently as the 2000-2001 school year, as gauged by their ACT performances. The boys outperformed the girls in all four subjects (Reading, English, Math and Science) and, of course, the composite score. This combined with the fact that our district scores were perennially lower than the state scores exacerbated the magnitude of the problem of the low girls’ ACT scores. For this reason (it is presumed), the Academic Excellence Committee was formed in 2002 to address this issue. Apparently, they designed a plan of some sort to raise the girls’ scores. And they were successful beyond anyone’s wildest dreams. However, it seems that now the plan needs to be to raise ALL of the students’ scores. A quick look through the graphs will verify the gender bias in the ACT scores in our district. If one quickly toggles through the tabs of each subject graph, one gets the distinct feeling of being on a teeter-totter: flat boys to high slope girls to flat boys to high slope girls. One gets a sense of vertigo.
Reading and English

Evansville girls overtook Evansville boys in Reading and English by 2002 and the scores just keep soaring. Evansville boy’s trend lines in Reading and English show a lackluster performance, ending up lower than state boys, state girls and Evansville girls. In terms of actual scores, they performed most poorly of the 4 groups in English and only outperformed the state boys in Reading.
Science and Math

The girls caught up with the Evansville boys in Science last year and, based on the trend line, are on the fast track to catch up with the state boys soon given a continuation of this pattern. Evansville’s girls beat Evansville’s boys in Math for the first time this year. The Math trend line predicts an even quicker overtaking of the state boys than the science trend line does. Our girls are awesome. This laudable accomplishment IS something to celebrate. But while the girls have caught up to the boys in the traditionally male dominated subjects of math and science, the boys have not done the same in the traditionally female dominated subjects of Reading and English. Worse yet, while the girls’ scores for their forte subjects have skyrocketed, the boys’ scores in their historically dominant subjects have either stagnated (math) or fallen (science). This is not acceptable.

In an egalitarian school, there is an expectation that all students are expected to improve in historically flat areas. We have empowered our girls to do just that in science and math and to soar in English and reading. Our boys’ math and science scores remain tepid (which hastened the interception by the girls’ trend lines), while their reading and English scores are at the bottom of the heap.

I reject the hypothesis that the ACT and WKCE tests do not measure the boys’ traditional learning style. They don’t measure anyone’s learning style. Theoretically, they measure what one has learned up to that point in time. If student A is hamstrung by a learning environment that suppresses his or her traditional method of processing information, that student will naturally have learned less than student B whose learning environment perfectly matches his or her learning style. By the time the ACT tests are administered, the students are on the same college-bound track, competing for the same goal: entrance to the colleges of their choice. They need to score well, regardless of their gender. The intimation that Ag students, scientists and mathematicians do not need to know how to compose a cogent sentence or critically evaluate literature is absurd. These significant ACT trends over time indicate a problem that needs to be addressed. The solution is not going to be simple. If Evansville realizes similar success in engaging our boys in their historically sluggish areas of reading and English as illustrated by a surge in the boys’ ACT scores, we will have met the needs of ALL of our students.
8 Period Day vs. Block Schedule

(Or does the Block Schedule exacerbate the differences?)

Please reference ACTREADING, ACTSCIENCE and ACTCOMPOSITE Excel files for this section of the report.

I was quoted in the paper as attributing the gender bias in Evansville’s ACT performance to the block schedule. Mr. Beedle emailed challenging that statement. The key theme in the email was “Flat boy’s performance is a nationwide problem and is present with both block schedule and the 8 period day.” I did qualify my remarks at the meeting (which wasn’t reported) that I could not unequivocally state the cause of our girls’ phenomenal improvement and our boys’ lack of same. The Block Schedule connection was just an idea that occurred to me in an AHA! moment. I generated the report at the same time the block schedule report was completed and it seemed like a plausible explanation. But Mr. Beedle’s comments challenged me to test my own hypothesis.

First, it was necessary to decide which districts to use as comparison. Often we are asked to compare our performance to like-sized districts. Querying the DPI website for districts with enrollments between 1500 and 2000 (in 2005-2006) resulted in a list of 32 similarly sized districts. This seemed a bit unwieldy, so the list was further limited to districts with enrollments between 1700 and 1900. This yielded a list of 10 schools (including Evansville), a reasonably representative and much more manageable group of schools with which to work. Each school is identified as having a block schedule (BS, N=4) or 8 period day (8PD, N=6). The schools selected were Amery(BS), Black River Falls(BS), East Troy Community (8PD), Edgerton(8PD), Ellsworth(BS), Evansville(BS), Jefferson(8PD), Osceola(8PD), Ripon (8PD) and West Salem (8PD). The ACT data for all 10 schools were tabulated by subject and separated by gender. One traditionally female dominated subject was surveyed (reading) (Excel filename ACTREADING), one traditionally male-dominated subject was assessed (science)(Excel filename ACTSCIENCE) and the composite scores were studied (Excel filename ACTCOMPOSITE) in an effort to determine how or if the block schedule contributes to gender bias in ACT performance.

Each Excel file has individual graphs comparing the boys and girls ACT performance for that subject for each school district (10 graphs) generated from a raw data sheet for that subject. These graphs are entitled Amery (BS) Boys vs. Girls, BR Falls (BS) Boys vs. Girls, and the like. In addition, there are 4 graphs, one each separating the girls’ performance in the block schedule, girls’ performance in the 8 period day, boys’ performance in the block schedule and boys’ performance in the 8PD. These graphs are called Girls Block, Girls 8 Period Day, etc. The 8PD graphs get a bit hairy looking since there are 6 trend lines for each 8PD graph, but with color coding, it’s decipherable.
Reading (ACTREADING)

The individual school graphs comparing girls to boys for each school have been grouped by schedule. First let’s address the Block Schedule data. Amery and Black River Falls trend lines indicate a traditional scenario in which girls slightly edge the boys. The slopes between the genders are similar and both schools show a decrease in ACT reading performance. Boys and girls in Ellsworth also show similar performances, but with each steadily improving with the trend lines overlapping and the boys indicating a slight advantage over the girls. Within the block schedule schools, only Evansville demonstrates the slow growth of the boys’ ACT scores and the soaring girls’ scores which started out below the boys, met then surpassed the boys. In terms of the ACT score values, most of the trend lines hover in the range of 21-22, with the notable exception of our girls data, the trend line of which ranges from 20 to 24.

In comparison, schools with the 8 period day (8PD) are less inclined to follow the traditional gender bias for Reading. Only East Troy and Ripon ACT data indicate the girls having the advantage over the boys in reading. East Troy’s trend lines are parallel and decreasing over time. The Ripon data result in a chart shape that is eerily similar to Evansville’s chart shape, but at much higher values. Ripon girls have scored at or above 23.6 on the ACT reading for the last 5 years. They are a bit of an anomaly in the overall scheme of this group of schools. Defying gender stereotypes, West Salem and Osceola boys scores are higher than the girls scores, with the Osceola girls’ trend line beginning to catch up to the boys’. The West Salem trend lines are parallel and demonstrate an increase in both male and female performance. Jefferson shows the girls starting out with the advantage, but a flat ACT performance by the girls over the years and a steady gain by their boys resulted in intersecting trend lines last year. Finally, Edgerton boys have posted a slight steady increase in ACT Reading scores while their girls’ scores have plummeted. It is of note that only the 8PD schools demonstrate the boys breaking the mold in a traditionally female dominated subject.

After charting all of the schools individually, it followed that one should compare the performance of each gender for each schedule separately. These are tabbed Girls Block, Girls 8 Period Day, etc. One gets a sense of Evansville’s performance against their peers in this way and can also toggle between the charts to see any general trends. When this was done, differences in the average ACT scores became clear between the schedules. The “Raw Data” spreadsheet has the averages and standard deviations indicated on them.

The Reading scores showed an unexpected pecking order. Highest among the averages here were the girls in the 8 Period Day schedule at 22.5, followed by the boys in the 8PD at 22.3, then the girls on the Block Schedule (21.9) and finally the boys in the block schedule (21.6). The average Evansville girls’ score over time was 22.2; the average boys’ score was 21.6.
Science (ACTSCIENCE)

Traditional gender biases are alive and well in all ten of the schools surveyed for science. There are three notable exceptions to this. Black River Falls girls science ACT scores are gradually increasing and, due to a plunge in their boys’ scores, the trend lines are set to intersect in a few years given similar circumstances continuing. The West Salem girls’ scores are increasing at about the same rate their boys’ scores are decreasing and the trend lines intersected last year. Evansville is at the other extreme of this set of trends, with the girls’ scores dramatically increasing while the boys’ scores slightly decrease over time. Our trend lines intersected in 2005.

Breaking the average scores down by schedule and gender, the average science ACT scores ranked as follows: boys 8PD(23.2), boys BS (22.6), girls 8PD(21.9) and finally girls BS(21.5). Evansville boys scored an average score of 22.4 over the years and the girls demonstrated an average of 21.5 over the years.

Composite (ACTCOMPOSITE)

The ACT composite scores for the selected school districts illustrate the stereotypical gender bias favoring boys across the board with two notable exceptions. Evansville and Ripon each demonstrate a scenario in which the boys’ scores were historically superior to the girls. The girls in both district have shown steady improvement while the boys performance has stagnated, resulting in a superior girls’ ACT composite score. The main difference between these districts is that the Ripon scores are more consistently higher than the Evansville scores. Being stagnant at 23.2 is better than being stagnant at 22. The recent surge in the Evansville girls’ scores has brought them on par with the Ripon girls’ scores! It is hoped that the girls performance can be sustained and is not a “smart class” phenomena alluded to in a previous report. We can inspire our boys to do better as well.

The ranking of averages by gender and schedule for the composite ACT scores follows: boys 8PD (22.6), girls 8PD (22.0), boys BS (21.9) and girls BS (21.5). In Evansville, the boys’ average composite ACT is 22.0 and the girls’ average composite score is 21.9.

The representative data for schedule and gender across all three ACT categories surveyed are tabulated below with rank in parentheses. Evansville data are ranked for comparison purposes.
Subject Boys 8PD Girls 8PD Boys BS Girls BS Ev Boys Ev Girls
Reading 22.3(2) 22.5 (1) 21.6 (5)

(tie)
21.9 (4) 21.6 (5)

(tie)
22.2 (3)
Science 23.2(1) 21.9 (4) 22.6(2) 21.5 (5)

(tie)
22.4 (3) 21.5 (5)

(tie)
Composite 22.6(1) 22.0(2)

tie
21.9(3)

tie
21.4(4) 22.0(2)

tie
21.9(3)

tie

Conclusions:

A survey of this small subset of Wisconsin schools illustrates that the Block Schedule has not given girls some natural advantage based on learning style. In fact, this brief study illustrates that gender bias in test results to the detriment of the girls in our state is very much alive. It is extremely disappointing to discover this via personal research. This study clarified why it is so difficult to find data regarding gender bias against boys in the literature. On the other hand, it is encouraging to discover that Evansville was not satisfied with the status quo, made and implemented a plan in 2002-2003, which is just now coming to fruition. With diligence, this will continue. Our district can also engage our boys in reading and English to help them soar.

In a surprise twist, when the ACT scores were categorized by schedule, gender and then ranked, the 8 Period Day did show a clear advantage for this randomly chosen group of districts. All three of the first ranked categories occurred in the 8 Period Day and two of the three second ranked categories occurred in the 8 Period Day. In all, 5/6 of the top average ACT scores for the subjects scrutinized here occurred in the 8 Period Day schedule. There may be many reasons for this, but the data are pretty clear. It does not seem plausible that this result is by chance. It would be interesting to survey the other 22 schools in the district size 1500-2000 originally queried at the start of the project to further address this subject. No stone should be left unturned in the quest to empower all of our students to excel.

Another conclusion gleaned from this study is that we should stop comparing ourselves to Edgerton, no matter how similar we are. Their ACT performance is abysmal and their girls’ scores are plummeting. Since our graph profiles already match the shape of Ripon’s graphs, I propose we aspire to their higher ACT scores as well.

There are bigger picture issues that follow from this data review, which our district can not, on it’s own, address. WKCE performance is a poor general predictor for ACT performance. Evansville as a district is rightly proud of our generally high WKCE performance. However, we get lulled into a false sense of security from high WKCE scores and are disappointed by low ACT performance. Ours can’t be the only district that experiences this phenomenon. That seems to indicate that there is a mismatch in curriculum, content and subject matter between Wisconsin WKCE requirements and National ACT standards. Perhaps the state needs to move the curriculum contents of the tests closer together to better capture a true picture of the educational status of Wisconsin students. Perhaps the WKCE can be made into a true educational tool instead of a hammer to beat districts into NCLB submission.
Possible Action Plan

There are some actions that Evansville as a district can take based on this overview of test scores by gender.

1. Shore up our WKCE performance to avoid NCLB penalties.

2. Address our boys’ stagnant ACT performance while remaining vigilant to maintain our girls’ awesome improvement.

3. Increase the database of ACT results and categorize them by schedule and gender to verify or disprove schedule contribution to ACT results.

Respectfully submitted,

Melissa Hammann, Evansville School Board.