MARS Test Accuracy????

[csagan001]

I have been trying for some time to ascertain the statistics showing the accuracy of the MARS test. At various times the MARS Company has published accuracy rates ranging from 84% on the low side to 95% on the high side. Currently, they are stating that the MARS test has a 90%. I am not sure why they reduced the published accuracy. (Click here for reference) (Click here for a report of the 84% accuracy number)

During the K9 College Cruise, I had the pleasure of taking a course with Dr. Danika L. Bannasch, DVM, Ph.D. (Click on her name for more). She is a geneticist at University of California Davis. Her work centers on dogs and horses. I asked her about the accuracy of the MARS test. She stated, “It is completely bogus.” I was unable to get her to define, “bogus” in more statistically relevant terms, but she was insistent that the tests did not provide a level of accuracy.

So, I continued to search for information regarding accuracy. I found several stories, such as this one, which relates a story of a single dog which when tested via the MARS program resulted in different answers for each test. Well, one story is interesting, but it is hardly a statistically valid statement. So, I went further.

And yet I was unable to find one single study, conducted by someone who had no interest in the MARS Veterinary Company, who tested the accuracy of the MARS test. Simply, all the data available has been published by the MARS Company. This does not mean that the MARS Company lied. However, it does make validation of their claims of accuracy difficult. When I was attending SuperZoolast year, I saw that MARS Veterinary had a booth. I approached the booth and engaged the MARS representative in a discussion about a specific breed. He stated, “You know Dr. X? We’ve been working with Dr. X for sometime.” Since I know that X's education is limited to an AA degree in computers, or some such subject, I was surprised that he referred to her as “Doctor.” I immediately became suspicious of anything the representative would say

Never-the-less, I attempted to discuss with the representative the statistical foundation for the company’s claims about the Wisdom Panel. I asked him about Type I errors, Type II errors, and the frequency of false positives and false negatives. He could not answer even one of my questions. Frustrated, I advised him to take a statistics class and then I left the area where the MARS Vet people had set up their booth

After all of this I came to the conclusion that with the scant information that I have to go on, I had to assume that the MARS Wisdom Panel test was, as the MARS company states, 90% accurate. That means it would have 5% false positives and 5% false negatives. It is highly unlikely that any scientific process would have the same percentage of false positives as false negatives, but to continue my analysis, I felt that I had no choice but to make that assumption

OK…so all of that brings us to this point: If the MARS test says that a dog is a specific breed, how inaccurate is that? If someone believes a dog is a specific breed, has the pedigree, and sends it in, what is the probability that there will be a false positive or a false negative? In other words, what is the probability that a perfectly healthy dog will be culled from someone’s breeding program in error?

So, just to keep things interesting (for me, perhaps not for you), I included a test question on one of the exams that I give my introductory statistics students at the university. They correctly concluded that out of 1,000 dogs of a specific breed thusly tested: 50 would be false positives and 50 would be false negatives. Since most breeders, in my opinion, are most interested in the accuracy from the standpoint of removing dogs from the breeding pool, the false negatives are more important than the false positives.

In other words, the question was: “Given the information above, what is the probability that a dog identified as NOT being a specific breed actually being that breed?”

We know that out of the 1,000 dogs tested, 900 would be correctly identified. We also know of those not correctly identified, 5% would be false negatives—in other words, the test results would indicate that they are not that specific breed when, in fact, they are. So what is the probability that if a dog of a specific breed is tested, and the results show the dog is not that breed, that the test is wrong? The answer is not merely the 5% accuracy number. The correct answer is relies on the study of probabilities, and is not as intuitive as one would think at first.

The correct answer is if a dog is tested, and is identified by the test as being something other than the breed shown on the pedigree, there is a 10% chance that the test is wrong. That is, if we believe the statistics from the MARS corporation.

Of course, if you use the statistics from MARS original studies, you end up with a 16% error rate.

Whatever the real number is, this much I know: If a dog is tested and found to be not-a-biewer, then there is between a 10% and 16% chance that the test has incorrectly identified the dog’s genetics.

Comments? Did I make a mistake here?

[Wend]

I looked at their FAQ and it seems to not be saying anything lol. The figures they do use, like the 90% they attach very general terms to, such as validation testing (what type? how was it done?) and that 90% was averaged over all breeds studied (what was the sample size that was averaged? what was the median percentage?) Not to mention how reliable is it? Needless to say, no matter how valid a test is, if it is not reliable you still have nothing, and vise versa. I think you would have to look at the original study (assuming it is published) to see if it was bunk or not.

[BamaFan121s]

Per Mars themselves, the test is not designed for purebred dogs and therefore not accurate when used on one. It was designed for mutts.

[csagan001]

Originally Posted by BamaFan121s

Per Mars themselves, the test is not designed for purebred dogs and therefore not accurate when used on one. It was designed for mutts.

If you read my post above thoroughly, you must realize that the test is not valid for mutts either.

[Ladymom]

Originally Posted by BamaFan121s

Per Mars themselves, the test is not designed for purebred dogs and therefore not accurate when used on one. It was designed for mutts.

Your whole premise is wrong since the test was only designed to identify breeds in a mixed breed dog. You are trying to compare apples to oranges.

Anyone who would use the MARS test to cull dogs from their breeding program as you suggest does not understand the intended purpose of the test.

[csagan001]

Originally Posted by Ladymom

Your whole premise is wrong since the test was only designed to identify breeds in a mixed breed dog. You are trying to compare apples to oranges.

Anyone who would use the MARS test to cull dogs from their breeding program as you suggest does not understand the intended purpose of the test.

My premise is not wrong. It nbecessitates a basic understanding of DNA: If they can't ascertain the breed of a purebred, then nthey don't have accurate and thorough knowledge of the DNA markers inherrent in the breeds they claim that they test for.

In other words, a mixed breed's DNA is more confusing than a purebred's DNA. If they can't accurately analyze the simpler case, they can't accurately analyze the more complex case.

In my opinion the reason they say their test is not for purebreds is because they don't want their error rates to be made public. Theie "mutt requirement" is not a limitation of the science--it is protection for themselves for making false claims about what they can actually determine.

The reported results are completely bogus.

[Woogie Man]

Originally Posted by csagan001

My premise is not wrong. It nbecessitates a basic understanding of DNA: If they can't ascertain the breed of a purebred, then nthey don't have accurate and thorough knowledge of the DNA markers inherrent in the breeds they claim that they test for.

In other words, a mixed breed's DNA is more confusing than a purebred's DNA. If they can't accurately analyze the simpler case, they can't accurately analyze the more complex case.

In my opinion the reason they say their test is not for purebreds is because they don't want their error rates to be made public. Theie "mutt requirement" is not a limitation of the science--it is protection for themselves for making false claims about what they can actually determine.

The reported results are completely bogus.

I don't think there could be any hypothetical genetic test that would have 100% accuracy on 100% of pure-bred dogs across all breeds. To me it would vary depending on 'purity' of any given pure breed.

In my way of looking at it, the purity of any given breed, genetically speaking, is predicated on the number of generations that it has been bred pure. There are many dog breeds, some having been bred for many years and some for only a short while. For instance the Pekingese's history goes back thousands of years while the Yorkshire Terrier only goes back 150 years or so. The 'modern' Yorkie goes back even less.

Testing by using genetic markers would show the Pekingese to be 'more pure' than the Yorkshire Terrier simply due to the Pekingese, by virtue of its history, being further removed from its mixed roots (foundation stock) than the Yorkshire Terrier. So the test would more likely show the Pekingese being pure, with no markers from other breeds and the Yorkshire Terrier to be less pure with vestigal markers of its mixed roots being apparent. This would result in a pure bred Pekingese always testing more pure than a pure bred Yorkshire Terrier. You would have to presume that accuracy for any given breed would vary, depending on the age of the breed.

All the above is hypothetical and I reserve the right to be completely wrong about this . But for the little that I've seen about it, the Mars test is basically useless information for mixed breed dog owners and would likely be wrong information for many pure bred owners. I certainly wouldn't rely on it to make any decisions regarding breeding stock. It (the test) could lead to interesting things in the future but seems to be of little practical value today.

[csagan001]

Originally Posted by Woogie Man

I don't think there could be any hypothetical genetic test that would have 100% accuracy on 100% of pure-bred dogs across all breeds. To me it would vary depending on 'purity' of any given pure breed.

In my way of looking at it, the purity of any given breed, genetically speaking, is predicated on the number of generations that it has been bred pure. There are many dog breeds, some having been bred for many years and some for only a short while. For instance the Pekingese's history goes back thousands of years while the Yorkshire Terrier only goes back 150 years or so. The 'modern' Yorkie goes back even less.

Testing by using genetic markers would show the Pekingese to be 'more pure' than the Yorkshire Terrier simply due to the Pekingese, by virtue of its history, being further removed from its mixed roots (foundation stock) than the Yorkshire Terrier. So the test would more likely show the Pekingese being pure, with no markers from other breeds and the Yorkshire Terrier to be less pure with vestigal markers of its mixed roots being apparent. This would result in a pure bred Pekingese always testing more pure than a pure bred Yorkshire Terrier. You would have to presume that accuracy for any given breed would vary, depending on the age of the breed.

All the above is hypothetical and I reserve the right to be completely wrong about this . But for the little that I've seen about it, the Mars test is basically useless information for mixed breed dog owners and would likely be wrong information for many pure bred owners. I certainly wouldn't rely on it to make any decisions regarding breeding stock. It (the test) could lead to interesting things in the future but seems to be of little practical value today.

One dog that I owned I had tested twice. The dog was some sort of a dalmation/australian shepherd mix. The first test came back as 75% dalmation, and 25% unknown. Six months later the dog tested as 35% labrador, 15% dalmation, and 50% Husky. Tests are completely bogus.

[Obie]

Originally Posted by csagan001

I have been trying for some time to ascertain the statistics showing the accuracy of the MARS test. At various times the MARS Company has published accuracy rates ranging from 84% on the low side to 95% on the high side. Currently, they are stating that the MARS test has a 90%. I am not sure why they reduced the published accuracy. (Click here for reference) (Click here for a report of the 84% accuracy number)

During the K9 College Cruise, I had the pleasure of taking a course with Dr. Danika L. Bannasch, DVM, Ph.D. (Click on her name for more). She is a geneticist at University of California Davis. Her work centers on dogs and horses. I asked her about the accuracy of the MARS test. She stated, “It is completely bogus.” I was unable to get her to define, “bogus” in more statistically relevant terms, but she was insistent that the tests did not provide a level of accuracy.

So, I continued to search for information regarding accuracy. I found several stories, such as this one, which relates a story of a single dog which when tested via the MARS program resulted in different answers for each test. Well, one story is interesting, but it is hardly a statistically valid statement. So, I went further.

And yet I was unable to find one single study, conducted by someone who had no interest in the MARS Veterinary Company, who tested the accuracy of the MARS test. Simply, all the data available has been published by the MARS Company. This does not mean that the MARS Company lied. However, it does make validation of their claims of accuracy difficult. When I was attending SuperZoolast year, I saw that MARS Veterinary had a booth. I approached the booth and engaged the MARS representative in a discussion about a specific breed. He stated, “You know Dr. X? We’ve been working with Dr. X for sometime.” Since I know that X's education is limited to an AA degree in computers, or some such subject, I was surprised that he referred to her as “Doctor.” I immediately became suspicious of anything the representative would say

Never-the-less, I attempted to discuss with the representative the statistical foundation for the company’s claims about the Wisdom Panel. I asked him about Type I errors, Type II errors, and the frequency of false positives and false negatives. He could not answer even one of my questions. Frustrated, I advised him to take a statistics class and then I left the area where the MARS Vet people had set up their booth

After all of this I came to the conclusion that with the scant information that I have to go on, I had to assume that the MARS Wisdom Panel test was, as the MARS company states, 90% accurate. That means it would have 5% false positives and 5% false negatives. It is highly unlikely that any scientific process would have the same percentage of false positives as false negatives, but to continue my analysis, I felt that I had no choice but to make that assumption

OK…so all of that brings us to this point: If the MARS test says that a dog is a specific breed, how inaccurate is that? If someone believes a dog is a specific breed, has the pedigree, and sends it in, what is the probability that there will be a false positive or a false negative? In other words, what is the probability that a perfectly healthy dog will be culled from someone’s breeding program in error?

So, just to keep things interesting (for me, perhaps not for you), I included a test question on one of the exams that I give my introductory statistics students at the university. They correctly concluded that out of 1,000 dogs of a specific breed thusly tested: 50 would be false positives and 50 would be false negatives. Since most breeders, in my opinion, are most interested in the accuracy from the standpoint of removing dogs from the breeding pool, the false negatives are more important than the false positives.

In other words, the question was: “Given the information above, what is the probability that a dog identified as NOT being a specific breed actually being that breed?”

We know that out of the 1,000 dogs tested, 900 would be correctly identified. We also know of those not correctly identified, 5% would be false negatives—in other words, the test results would indicate that they are not that specific breed when, in fact, they are. So what is the probability that if a dog of a specific breed is tested, and the results show the dog is not that breed, that the test is wrong? The answer is not merely the 5% accuracy number. The correct answer is relies on the study of probabilities, and is not as intuitive as one would think at first.

The correct answer is if a dog is tested, and is identified by the test as being something other than the breed shown on the pedigree, there is a 10% chance that the test is wrong. That is, if we believe the statistics from the MARS corporation.

Of course, if you use the statistics from MARS original studies, you end up with a 16% error rate.

Whatever the real number is, this much I know: If a dog is tested and found to be not-a-biewer, then there is between a 10% and 16% chance that the test has incorrectly identified the dog’s genetics.

Comments? Did I make a mistake here?

I kept reading this post over and over again. Didn't make too much sense. Then I suddenly realized who you are. Oh brother.

The hidden clause in all this is a member of YorkieTalk going after another member. Another reason why I don't post too much here anymore. Too one sided by the far left. Hey Dan?

Oh, BTW I contacted Dr. Bannasch.

[Ladymom]

Originally Posted by Obie

I kept reading this post over and over again. Didn't make too much sense. Then I suddenly realized who you are. Oh brother.

The hidden clause in all this is a member of YorkieTalk going after another member. Another reason why I don't post too much here anymore. Too one sided by the far left. Hey Dan?

Oh, BTW I contacted Dr. Bannasch.

The OP definitely seemed to have an agenda to me, too. Not sure what it is though!

[livingdustmops]

Originally Posted by Ladymom

The OP definitely seemed to have an agenda to me, too. Not sure what it is though!

As did the original group/people that came on here and posted about the Mars test. A test to determine what your mutt is and are using it to convince people (AKC) that they have a purebred dog from 3 different breeds. People that have disagreed have been called stupid housewives that don't know anything.

I know what the agenda is for this post and it is to bad people can't say why they do not agree with the tests and or the club without being pulled through the wringer.

[Obie]

Originally Posted by livingdustmops

As did the original group/people that came on here and posted about the Mars test. A test to determine what your mutt is and are using it to convince people (AKC) that they have a purebred dog from 3 different breeds. People that have disagreed have been called stupid housewives that don't know anything.

I know what the agenda is for this post and it is to bad people can't say why they do not agree with the tests and or the club without being pulled through the wringer.

No, you are wrong. The OP DID agree with the testing when it benefitted he/she. They totally agreed with the test because their Biewers were in on it. Ask the OP what suddenly happened. See,this is why so many of you don't understand who is and is not posting on here. You are only getting one side of the story.

I believe in the MARS testing because it goes beyond the Wisdom Panel. The testing is not what most people on here think it is. The MARS people were hounded and haunted and they could not answer their questions because they are not the ones that were used to do the study. The study is not finished, and for one geneticist to call it bogus from the original OP, I find that very insubordinate to their fellow colleagues. Also, nobody really knows the story behind Dr. X. That was an insult to all you breeders on here because I took that as you have to have a PhD to be a breeder. Those who know about Dr. X knows it was said out of respect to that person for their knowledge. Too bad somebody has to take it out of context. Shame on you!


This is my take on the whole thing. Everybody just needs to settle down about the Biewer Terrier, and the Biewer a la pom pon. What ever the clubs believe in, let them be. If one club is reaching out for AKC, so what? If another club wants to have Biewer Yorkshires and hopefully get them accepted under parti yorkies, so what. Everybody just needs to back off and let the breeders of the clubs alone and take their own route. Thats my take on this whole thing.

[JeanieK]

Interesting!!!! Every statistician knows that the same statistics can be used both for and against the same issue.

[drafting]

Originally Posted by livingdustmops

As did the original group/people that came on here and posted about the Mars test. A test to determine what your mutt is and are using it to convince people (AKC) that they have a purebred dog from 3 different breeds. People that have disagreed have been called stupid housewives that don't know anything.

I know what the agenda is for this post and it is to bad people can't say why they do not agree with the tests and or the club without being pulled through the wringer.

Where is the thread that you say the "original group/people came on here and posted about the Mars testing". I would like to go back and see what they had to say about the test.
Judy

[sierrapups]

I wonder what the Mars people would have to say about these outrages accusations made by this OP? Would they take kindly to being called bogus on an open forum?
Sour grapes is what we are looking at here. I was not going to post but I find it very sad that OP was 100% behind all of the testing and would still be if not for the unfortunate circumstances of his removal from a certain club.