Essay-Grading Software Regarded As Time-Saving Tool

Teachers are turning to essay-grading software to critique student writing, but critics point out serious flaws within the technology

Jeff Pence knows the easiest way for his 7th grade English students to boost their writing is to do more of it. But with 140 students, he would be taken by it at least a couple of weeks to grade a batch of these essays.

Therefore the Canton, Ga., middle school teacher uses an online, automated essay-scoring program which allows students to have feedback on the writing before handing in their work.

“It doesn’t let them know what you should do, however it points out where issues may exist,” said Mr. Pence, who says the a Pearson WriteToLearn program engages the students just like a game title.

A week and individualize instruction efficiently with the technology, he has been able to assign an essay. “I feel it is pretty accurate,” Mr. Pence said. “can it be perfect? No. But when I reach that 67th essay, I’m not accurate that is real either. As a team, we have been pretty good.”

With the push for students to become better writers and meet with the new Common Core State Standards, teachers are eager for new tools to assist out. Pearson, that will be located in London and new york, is one of several companies upgrading its technology in this space, also called artificial intelligence, AI, or machine-reading. New assessments to try deeper move and learning beyond multiple-choice answers are also fueling the demand for software to simply help automate the scoring of open-ended questions.

Critics contend the program does not do alot more than count words and so can’t replace readers that are human so researchers are working hard to improve the program algorithms and counter the naysayers.

Whilst the technology has been developed primarily by companies in proprietary settings, there is a focus that is new improving it through open-source platforms. New players on the market, such since the startup venture LightSide and edX, the enterprise that is nonprofit by Harvard University and the Massachusetts Institute of Technology, are openly sharing their research. This past year, the William and Flora Hewlett Foundation sponsored an competition that is open-source spur innovation in automated writing assessments that attracted commercial vendors and teams of scientists from about the planet. (The Hewlett Foundation supports coverage of “deeper learning” issues in Education Week.)

“We are seeing plenty of collaboration among competitors and individuals,” said Michelle Barrett, the director of research systems and analysis for CTB/McGraw-Hill, which produces the Writing Roadmap for usage in grades 3-12. “This unprecedented collaboration is encouraging a lot of discussion and transparency.”

Mark D. Shermis, an education professor in the University of Akron, in Ohio, who supervised the Hewlett contest, said the meeting of top public and commercial researchers, along side input from a variety of fields, could help boost performance of this technology. The recommendation from the Hewlett trials is the fact that the software that is automated used as a “second reader” to monitor the human readers’ performance or provide additional information about writing, Mr. Shermis said.

“The technology can’t do everything, and nobody is claiming it may,” he said. “But it really is a technology which has had a promising future.”

The first essay-scoring that is automated return to the early 1970s, but there clearly wasn’t much progress made before the 1990s with the advent for the buy essay Internet plus the ability to store data on hard-disk drives, Mr. Shermis said. More recently, improvements have been made in the technology’s ability to evaluate language, grammar, mechanics, and magnificence; detect plagiarism; and offer quantitative and feedback that is qualitative.

The computer programs assign grades to writing samples, sometimes on a scale of 1 to 6, in a variety of areas, from word choice to organization. The merchandise give feedback to simply help students boost their writing. Others can grade answers that are short content. To save money and time, the technology may be used in several ways on formative exercises or summative tests.

The Educational Testing Service first used its e-rater automated-scoring engine for a high-stakes exam in 1999 for the Graduate Management Admission Test, or GMAT, according to David Williamson, a senior research director for assessment innovation for the Princeton, N.J.-based company. In addition uses the technology with its Criterion Online Writing Evaluation Service for grades 4-12.

The capabilities changed substantially, evolving from simple rule-based coding to more sophisticated software systems over the years. And statistical techniques from computational linguists, natural language processing, and machine learning have helped develop better methods of identifying certain patterns in writing.

But challenges remain in coming up with a universal definition of good writing, as well as in training a computer to know nuances such as for example “voice.”

With time, with larger sets of information, more experts can identify nuanced aspects of writing and enhance the technology, said Mr. Williamson, who is encouraged by the era that is new of concerning the research.

“It really is a hot topic,” he said. “There are a lot of researchers and academia and industry looking into this, and that is a very important thing.”

High-Stakes Testing

Along with utilising the technology to improve writing in the classroom, West Virginia employs software that is automated its statewide annual reading language arts assessments for grades 3-11. The state spent some time working with CTB/McGraw-Hill to customize its product and train the engine, using thousands of papers it offers collected, to score the students’ writing according to a specific prompt.

“we have been confident the scoring is extremely accurate,” said Sandra Foster, the lead coordinator of assessment and accountability within the West Virginia education office, who acknowledged skepticism that is facing from teachers. But the majority of were won over, she said, after a comparability study showed that the precision of a teacher that is trained the scoring engine performed a lot better than two trained teachers. Training involved a few hours in how to measure the writing rubric. Plus, writing scores have gone up since implementing the technology.

Automated essay scoring is also utilized on the ACT Compass exams for community college placement, this new Pearson General Educational Development tests for a school that is high diploma, along with other summative tests. Nonetheless it have not yet been embraced because of the College Board for the SAT or even the ACT that is rival college-entrance.

The 2 consortia delivering the new assessments under the Common Core State Standards are reviewing machine-grading but have not devoted to it.

Jeffrey Nellhaus, the director of policy, research, and design for the Partnership for Assessment of Readiness for College and Careers, or PARCC, wants to determine if the technology is likely to be a good fit with its assessment, therefore the consortium is likely to be conducting a study predicated on writing from the first field test to see how the scoring engine performs.

Likewise, Tony Alpert, the principle officer that is operating the Smarter Balanced Assessment Consortium, said his consortium will evaluate the technology carefully.

Together with his new company LightSide, in Pittsburgh, owner Elijah Mayfield said his data-driven way of automated writing assessment sets itself aside from other products available on the market.

“that which we are attempting to do is build a method that instead of correcting errors, finds the strongest and weakest chapters of the writing and where you should improve,” he said. “It is acting more as a revisionist than a textbook.”

The software that is new which will be available on an open-source platform, will be piloted this spring in districts in Pennsylvania and New York.

In higher education, edX has just introduced automated software to grade open-response questions for usage by teachers and professors through its free online courses. “One associated with the challenges in past times was that the code and algorithms were not public. They were viewed as black magic,” said company President Anant Argawal, noting the technology is in an experimental stage. “With edX, we place the code into open source where you can observe how it really is done to help us improve it.”

Still, critics of essay-grading software, such as Les Perelman, want academic researchers to possess broader usage of vendors’ products to guage their merit. Now retired, the previous director for the MIT Writing Across the Curriculum program has studied a number of the devices and managed to get a score that is high one with an essay of gibberish.

“My main concern is so it doesn’t work,” he said. Whilst the technology has some use that is limited grading short answers for content, it relies a lot of on counting words and reading an essay requires a deeper level of analysis best done by a human, contended Mr. Perelman.