One of the new additions to the search aggregator Google News, Bing News, and others is a new fact check label. They are using a select group of news sites and including their articles marked as fact checks.
Fact checking can be very subjective, since often the statements are not purely measurable. To determine true or false, there is interpretation that is required to arrive at something that is measurable. This interpretation of the statement and distilling it down into something that can be verified is an area where these fact checkers fail.
For instance, if someone claims that Drew Brees is tall and that statement is fact checked the fact checkers' results could vary greatly. One fact checker could look at the population of NFL quarterbacks and declare that the statement is absolutely false, since Drew Brees is one of the shortest NFL quarterbacks. Another fact checker could look at Drew Brees' height of 6 feet and declare the he is taller than the average American man which is 5 feet and ten inches and rate this claim partially true, since it is true that he is taller than the average American man, but not by much. Still another fact checker could look at his height as compared to the average human's height which is said to be 5 feet 6 inches and declare this absolutely true.
This was a simple example compared to the more complex statements that are made. In this example the claim was clear, but in many fact checks the exact claim is not clear, so there is even more room for interpretation by the person doing the fact checking. The statement to fact check would more likely be Drew Brees is a successful quarterback, in part because he is tall. This could be distilled down to several claims and we only looked at one of the possibilities.
It would be helpful to have a range of fact checks where you have a chance to see the logic of someone who rates the statements in different ways. For instance, it is useful when researching products to look at 1 star, 3 star, and 5 star reviews. Looking at a range of reviews can often tell you the weaknesses and strengths of a product. In much the same way, looking at different fact checking processes allows you to look at multiple arguments for the fact being checked.
Fact checkers often fail to mention ideas that do not support their conclusion or weaknesses in their own argument. They are more like lawyers advocating for a certain conclusion rather than truth seekers looking at all sides of a statement. It would be useful for these news aggregators to find different results for the same fact check and allow the reader to do a comparison of the different fact checks as they already do with articles. This may require opening up the fact checking to allow a more diverse group of sites to do the fact checking, but this will yield a better result.