NPPA Decides To Re-Judge Some Best Of Photojournalism Categories

 

By Donald R. Winslow

© 2011 News Photographer magazine

 

DURHAM, NC (April 8, 2011) – At the end of a week of investigating entries while comparing submissions to the individual and group contest entry data, NPPA's leadership today agreed with Best Of Photojournalism contest committee members Terry Eiler and Stan Alost in their assessment that it is necessary to re-judge some of the entries in this year's Newspaper Picture Editing categories.

The reason for starting over comes from a conclusion they've reached today after a week of trying to figure out how many Newspaper Picture Editing entries did not get viewed for judging during the first round.

"This is the right thing to do when faced with an error of this sort," NPPA president Sean D. Elliot said today. "We’re committed to the integrity of the competition and to leave the results standing after this came to light would not be fair to any of the entrants. If you won, you want to know you won against every entry, and if you entered you need to know your work was viewed and judged fairly. This may not look great, but in the overall scheme of things going back and re-judging is the way to go."

NPPA and the contest committee members were unaware that there there was a problem until the judges were almost finished picking Picture Editing and Best Use Of Photography winners last Saturday at the contest's host site, Ohio University's School of Visual Communication (VisCom) in Athens.

Late Saturday afternoon members of the contest support team had a problem with one of the entries. It was a corrupted file that would not open for viewing. The entries are sent to OU from NPPA's office as digital material, DVDs of the pages accompanied by a data spreadsheet. The spreadsheet has the name of the entrant, numbers and categories, personal contest entry numbers, and the name of the person or group and publication.

"A file number on the spreadsheet should correspond with something we have on hand to view in the judging," VisCom director Eiler said today.

When Eiler and Alost contacted NPPA staff in Durham, they identified the corrupted file entry by number and NPPA staff electronically sent a new copy of the entry to the judges.

As judges resumed they were looking at a group entry, their last category for the contest, where they realized there were 11 groups entered – but they only had nine group files.

One of the files in the group of 11 was from a newspaper that traditionally does well in contests, and the judges began to wonder why they hadn't seen entries from that newspaper in other Best Of Photojournalism categories earlier in the judging. They could not recall seeing pages from that newspaper during judging in the past two days.

The judges for these categories are Bert Fox, Sue Morrow, and Chris Wilkins. They asked Eiler and Alost about whether they had seen entries from that newspaper, since the judges weren't sure. That's when the VisCom hosts began to investigate.

 

Contacting NPPA staff in Durham to inquire Saturday evening, it was discovered that on the NPPA contest's entry server there was 1.5GB ZIP file from the newspaper in question that had not been included in the original material provided to OU for judging. Conceivably a file that size could have contained the entire entry from one newspaper. At this point NPPA staff began investigating to see if other entries had been missed, as well as trying to determine how this could have happened.

During their investigation NPPA staff then discovered that there was a 1.1GB ZIP file from a second newspaper that also had not been included, as well as later finding an entry file from a third newspaper.

At that point Saturday evening, Elier and Alost and the judges made the decision to stop posting any more information about category winners until more facts could be determined. A VisCom photojournalism graduate student, Priscilla Thomas, was tasked with breaking down line-by-line the spreadsheet of entry data.

On Monday morning, NPPA staff did a more complete investigation of the contest's entry server and discovered that stored on the machine were five or six newspaper entries in Picture Editing categories that had not been sent for judging, and one entry that may have been placed in the wrong category.

Before the judging was halted on Saturday night, judges had named some winners in several categories. At this point the contest coordinators suspect that the errors and confusion are only in the newspaper categories, and not in the magazine categories. Magazine category winners have already been picked and posted, and these will remain unchanged at this time, Eiler said today. Their investigation has led them to conclude that all of the entries in the magazine categories were received, viewed by judges, and considered, with no missing files.

However, since the entire crop of Newspaper category entries are going to be re-judged, "The newspaper winners that had been posted are 'coming down'", Eiler said, "and will have to be re-judged."

Eiler said the categories in question will be re-judged using the same judges. How they go about it depends on the final count of how many entries need to be re-judged. If it is a handful, they may do it online and with a telephone conference call. If it is a significant number, Eiler said, they will bring the judges back to VisCom and do the entire thing again.

He acknowledged that some entrants are likely to be unhappy with the categories being re-judged, "But to be fair to everyone who entered, if we have to we'll do it all over again."

Alost said, "Our mission is to ensure fair, accurate, and thorough evaluation of all the entries. We just need to continue the process a bit longer. We thank the entrants for their patience."

 

As far as NPPA can determine at this point, the problem started with the entry process itself and the data created by people entering the contest.

"It's a bit of a jumble," Eiler said. When someone enters the contest using an online form, each time they begin an entry it creates data, and an entry number. Often the entrants don't complete their entry process before one of several things happen, including the user stopping, or losing their connection, or making an error and abandoning the entry and starting over again. There's also the confusion between individual entries and group entries from the same entrants.

The data from these multiple entry attempts and completions is what populates the spreadsheet that accompanies the DVDs to VisCom a week to ten days before the judging. The contest coordinators are then tasked with sorting out the entry data and numbers with the actual entries. In addition to entries being on DVDs, there are "hard copy" entries of actual newspaper pages to match up with entry information.

"The difficulty came with the difference between the spreadsheet and the entries," Eiler said.

"If a file was listed in the spreadsheet but not included on the DVDs, it could be a file that was never submitted, a duplicate of another file, or a dropped entry. Files that weren't submitted, or that were removed as duplicates, don't belong in the entry pool, but any file that was submitted correctly should be judged."

The problem might be easily understood in an analogy to online shopping, such as if someone abandons a "shopping cart" while shopping online at Amazon, or starts a shopping cart multiple times. The transaction isn't completed, but the process creates data without there being an actually finalization or delivery.

"We are instituting a process to avoid this in the future," Alost said tonight.

Eiler explained how the process was de-constructed during the past week and what they learned. "On Saturday we could not say with any confidence where the error was. By Wednesday it looked like there might be as many as 180 missing files. But by today, Friday, it looks more like three or four. By Wednesday we knew that there were 180 questionable entries on the spreadsheet that did not correspond with entries on the DVDs. And by today, there's not nearly as many confirmed missing files."

He also wanted to point out that this portion of the judging was not using the new technology developed for Best Of Photojournalism by PhotoShelter. "If we'd been using the new technology, which we will be using by this point next year, I don't think we'd have had this problem," Eiler said, because the technology methodology differs from the digital workflow that NPPA has used successfully for the past four years.

"This has been a challenging year for the BOP, with a computer failure at the television judging delaying those results and a delay in the judging of the television editing division still being ironed out," Elliot said today. "Add to that the complete transition of the Still Photography competition to an online judging system, and it's caused more than a little stress recently in NPPA's leadership. But we are confident we'll weather this storm, and after undertaking a complete review of all the BOP divisions we’ll move forward into the next year better prepared. Nothing that happened this year has been irreparable, and we’ll learn and grow from this."

 

 

NPPA Marketplace

Insure Your Equipment

You go where the action is….so should your insurance! Hays delivers comprehensive insurance for your gear - covering cameras, computers, editing equipment and rental.

Join the NPPA

NPPA members receive a wide range of benefits, from educational opportunities to mentoring, exclusive discounts, insurance options, business tips, and much more.