Readers’ Feedback: Sales Vetting

[Wandering Eyes]

A number of our readers submitted feedback in response to the feature, “Sales Vetting: Agree to Differ, or Differ to Agree?” which appeared in the Jan. 23 edition of the TDN. Their submissions can be found below.

With only about 5% of horses bred turning out to what can be reasonably considered elite runners, anyone working in this industry should understand the probabilities of outcomes. Setting aside that buyers routinely ‘fail’ a horse on the basis of a cursory visual inspection, this also explains why both buyers and sellers alike have ‘tales of woe’ when a horse that they liked is ‘failed’ by someone else in any way at a foal, yearling or 2-year-old sale. The known rarity of the event itself, an elite racehorse, makes it sting.

Almost all of the questions that revolve around X-rays are concerned with probabilities, not absolutes. If the horse has an abnormality on X-rays, what is the probability that it is going to affect its racing ability? Will it be different if I train/race it on dirt, which through concussion is a much less forgiving surface than turf or synthetic surfaces? Will the same issue be a different outcome if we have a lightly made filly or a big strong colt?

While we would like to think otherwise, currently most of the answers to these questions are largely built on subjective, experience based opinion. As humans, the opinions of veterinarians are often colored by confirmation bias (we tend to remember the horses we thought had a problem and subsequently didn’t race, and discount the ones that did) and a tendency to anchor on one piece of information when making decisions. This is compounded distance in time from when the X-Rays are interpreted at yearling stage, to when racetrack outcomes (or lack thereof) are known.

The industry has available three datasets that could be utilized to provide some of the answers to these questions, taking much of the subjectivity out of the equation:

1) The thousands of digital images that veterinarians take each year of yearlings across North America and Europe for the past decade.

2) The Jockey Club Information Systems, which has the data on these yearlings as to if they started, when they started, what surface they started on and how many starts they had on each surface.

3) The Equine Injury Database, which identifies the frequency, types and outcome of racing injuries using a standardized format.

Deep learning-based Artificial Intelligence has shown its abilities across multiple medical domains. It particularly shines at well-defined clinical tasks where most of the information necessary for the task is contained in the data, especially imaging.

In 2016, researchers associated with Google published a paper on detection of Diabetic Retinopathy from retinal photographs. (click here) where an AI network achieved 90.3% sensitivity–e.g the percentage of sick people who are correctly identified as having the condition, and 98.1% specificity–e.g., the percentage of healthy people who are correctly identified as not having the condition. In 2017, Rajpurkar and colleagues published the paper CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning (click here) to considerable commentary in the medical and artificial intelligence community, where they trained 112,120 frontal-view chest X-ray images individually labeled with up to 14 different thoracic diseases, including pneumonia. When testing the trained neural network against four practicing radiologists, CheXNet outperformed the radiologists in determining if pneumonia was present on the X-Ray.

If labeled correctly and accurately, and with enough data, every clinical finding could be identified on X-Rays and a probability assigned for each finding of it appearing on any new X-Rays. Additionally with the data from TJCIS and the Equine Injury Database models could be built that gave the probabilities of each of these findings having an impact on racing outcomes, removing a lot of the objectivity out of the process completely. Rather than relying on the experience of veterinarians with inherent human biases to interpret X-rays and assign probabilities, we would have more objective knowledge of outcomes.

Before veterinarians throw away their board certifications and look for jobs as Uber drivers, a few comments on what a well trained neural network could and could not do.

Any algorithm would only be able to give initial probabilities of an issue being present and the probability of any combined issues affecting the outcome, but it would reduce any current variation between veterinarians’ opinions as to whether an abnormality is present, and what it means for future soundness. A reduction in objectivity in this area should be welcomed by buyers and sellers alike.

It would still however require interpretation and understanding of those probabilities. If an algorithm said there was a 60% probability of a chip being in the knee and a 30% chance of it having an effect on its chances of the horse ever racing, it would still require some professional judgment on the data that the algorithm has not seen. Does that 30% chance reduce with surgery; does it reduce if the horse is a small correct light filly as opposed to a large heavy colt with offset knees? Rather than being bound in the sales repositories around the world, it would free veterinarians to do what many do best, interact with the horse in a clinical setting and work with their clients to make better objective decisions about potential purchases.

The data is there, the algorithms are built and can be easily transferred to operate in the Thoroughbred world, its just going to take the parties involved having the will to do it.

–Byron Rogers, Performance Genetics LLC

 

In the 1/23 TDN article about vetting, Carrie Brogden notes that, “And to me, the ones [farms] that raise a lot of runners don’t raise them like sales horses. They raise them tough and strong.”

It caused me to recall interviewing actor Jack Klugman for the Thoroughbred Record after his Jaklin Klugman won the 1980 California Derby. Amidst all the celebration Jack paused and offered a serious note about Jaklin Klugman. “When he hurt himself that time at Hollywood Park,” Klugman said, “it took the bloom off the rose, and we stopped trying to treat him like something that was in a bubble. He was a race horse, who races, who will win, who will lose, who will be good enough or bad enough for whatever reason comes up.”

As Ms. Brogden will appreciate, letting a horse be a horse and not an equine version of Project Runway for a sale is a key to success that is not always appreciated by breeders who raise horses in “a bubble.”

–Ron Parker, Paris, KY

 

View the full article