I'm referring to Bill Adair's apt analogy that when it comes to fact-checking, journalists are making a call — they are explicitly saying if something is true or not. Adair, the editor of Politifact, was speaking in a Poynter Webinar that I watched for my journalism class.
Making these "calls" unambiguous is where a rating system shines. The Truth-O-Meter's six distinct categories — True, Mostly True, Half-True, Mostly False, False and Pants On Fire — makes the fact-checker's judgement clear. Even if readers are confused by some of the intermediate steps that went into checking a claim, they will still get the overall picture.
Ultimately, I think this is a valuable service for journalists to provide. If someone claims something that can be checked, journalists should absolutely do the legwork needed to verify or debunk it.
I'm reminded of the flap over a column by Arthur Brisbane, former public editor of The New York Times. In it, he asks whether reporters should call out lies and errors as they report them. For many journalists, the answer was 'Duh,' as Poynter blogger Chris Silverman documented.
Brisbane did have some valid points. There are pitfalls we need to watch out for. Does it make sense to call out every lie as we see them, considering how long a typical factcheck is? We can abridge them, but that might not do the claim justice. Also, there's a very real need to make sure we aren't hounding one side while letting the other side get away with worse lies.
And yet, we have to avoid suggesting a false equivalence. Sometimes the evidence is uneven. If one politician's speech is flat out wrong, I want to know that. I still want to if the publication would appear biased in the short run.
I think Adair has a sensible philosophy for avoiding false equivalence. Just don't try and keep track of which side has been rated more often. He acknowledged the need to occasionally make sure it hasn't gone completely out of whack, but evenness is not a big concern. After all, referees don't add up the penalties for both sides to make sure it comes out even. If one team fouls more, they will get more penalties.
Adair presented Politifact's techniques straightforwardly and provided plenty of practical information. Even if you aren't interested in the methods, the results are a must-read for every journalist and voter.