MARS Test Accuracy????
 

I have been trying for some time to ascertain the statistics showing the accuracy of the MARS test. At various times the MARS Company has published accuracy rates ranging from 84% on the low side to 95% on the high side. Currently, they are stating that the MARS test has a 90%. I am not sure why they reduced the published accuracy. (Click here for reference) (Click here for a report of the 84% accuracy number)

During the K9 College Cruise, I had the pleasure of taking a course with Dr. Danika L. Bannasch, DVM, Ph.D. (Click on her name for more). She is a geneticist at University of California Davis. Her work centers on dogs and horses. I asked her about the accuracy of the MARS test. She stated, “It is completely bogus.” I was unable to get her to define, “bogus” in more statistically relevant terms, but she was insistent that the tests did not provide a level of accuracy.

So, I continued to search for information regarding accuracy. I found several stories, such as this one, which relates a story of a single dog which when tested via the MARS program resulted in different answers for each test. Well, one story is interesting, but it is hardly a statistically valid statement. So, I went further.

And yet I was unable to find one single study, conducted by someone who had no interest in the MARS Veterinary Company, who tested the accuracy of the MARS test. Simply, all the data available has been published by the MARS Company. This does not mean that the MARS Company lied. However, it does make validation of their claims of accuracy difficult. When I was attending SuperZoolast year, I saw that MARS Veterinary had a booth. I approached the booth and engaged the MARS representative in a discussion about a specific breed. He stated, “You know Dr. X? We’ve been working with Dr. X for sometime.” Since I know that X's education is limited to an AA degree in computers, or some such subject, I was surprised that he referred to her as “Doctor.” I immediately became suspicious of anything the representative would say

Never-the-less, I attempted to discuss with the representative the statistical foundation for the company’s claims about the Wisdom Panel. I asked him about Type I errors, Type II errors, and the frequency of false positives and false negatives. He could not answer even one of my questions. Frustrated, I advised him to take a statistics class and then I left the area where the MARS Vet people had set up their booth

After all of this I came to the conclusion that with the scant information that I have to go on, I had to assume that the MARS Wisdom Panel test was, as the MARS company states, 90% accurate. That means it would have 5% false positives and 5% false negatives. It is highly unlikely that any scientific process would have the same percentage of false positives as false negatives, but to continue my analysis, I felt that I had no choice but to make that assumption

OK…so all of that brings us to this point: If the MARS test says that a dog is a specific breed, how inaccurate is that? If someone believes a dog is a specific breed, has the pedigree, and sends it in, what is the probability that there will be a false positive or a false negative? In other words, what is the probability that a perfectly healthy dog will be culled from someone’s breeding program in error?

So, just to keep things interesting (for me, perhaps not for you), I included a test question on one of the exams that I give my introductory statistics students at the university. They correctly concluded that out of 1,000 dogs of a specific breed thusly tested: 50 would be false positives and 50 would be false negatives. Since most breeders, in my opinion, are most interested in the accuracy from the standpoint of removing dogs from the breeding pool, the false negatives are more important than the false positives.

In other words, the question was: “Given the information above, what is the probability that a dog identified as NOT being a specific breed actually being that breed?”

We know that out of the 1,000 dogs tested, 900 would be correctly identified. We also know of those not correctly identified, 5% would be false negatives—in other words, the test results would indicate that they are not that specific breed when, in fact, they are. So what is the probability that if a dog of a specific breed is tested, and the results show the dog is not that breed, that the test is wrong? The answer is not merely the 5% accuracy number. The correct answer is relies on the study of probabilities, and is not as intuitive as one would think at first.

The correct answer is if a dog is tested, and is identified by the test as being something other than the breed shown on the pedigree, there is a 10% chance that the test is wrong. That is, if we believe the statistics from the MARS corporation.

Of course, if you use the statistics from MARS original studies, you end up with a 16% error rate.

Whatever the real number is, this much I know: If a dog is tested and found to be not-a-biewer, then there is between a 10% and 16% chance that the test has incorrectly identified the dog’s genetics.

Comments? Did I make a mistake here?

I looked at their FAQ and it seems to not be saying anything lol. The figures they do use, like the 90% they attach very general terms to, such as validation testing (what type? how was it done?) and that 90% was averaged over all breeds studied (what was the sample size that was averaged? what was the median percentage?) Not to mention how reliable is it? Needless to say, no matter how valid a test is, if it is not reliable you still have nothing, and vise versa. I think you would have to look at the original study (assuming it is published) to see if it was bunk or not.

Per Mars themselves, the test is not designed for purebred dogs and therefore not accurate when used on one. It was designed for mutts.

If you read my post above thoroughly, you must realize that the test is not valid for mutts either.

:thumbup: Your whole premise is wrong since the test was only designed to identify breeds in a mixed breed dog. You are trying to compare apples to oranges.

Anyone who would use the MARS test to cull dogs from their breeding program as you suggest does not understand the intended purpose of the test.

My premise is not wrong. It nbecessitates a basic understanding of DNA: If they can't ascertain the breed of a purebred, then nthey don't have accurate and thorough knowledge of the DNA markers inherrent in the breeds they claim that they test for.

In other words, a mixed breed's DNA is more confusing than a purebred's DNA. If they can't accurately analyze the simpler case, they can't accurately analyze the more complex case.

In my opinion the reason they say their test is not for purebreds is because they don't want their error rates to be made public. Theie "mutt requirement" is not a limitation of the science--it is protection for themselves for making false claims about what they can actually determine.

The reported results are completely bogus.

I don't think there could be any hypothetical genetic test that would have 100% accuracy on 100% of pure-bred dogs across all breeds. To me it would vary depending on 'purity' of any given pure breed.

In my way of looking at it, the purity of any given breed, genetically speaking, is predicated on the number of generations that it has been bred pure. There are many dog breeds, some having been bred for many years and some for only a short while. For instance the Pekingese's history goes back thousands of years while the Yorkshire Terrier only goes back 150 years or so. The 'modern' Yorkie goes back even less.

Testing by using genetic markers would show the Pekingese to be 'more pure' than the Yorkshire Terrier simply due to the Pekingese, by virtue of its history, being further removed from its mixed roots (foundation stock) than the Yorkshire Terrier. So the test would more likely show the Pekingese being pure, with no markers from other breeds and the Yorkshire Terrier to be less pure with vestigal markers of its mixed roots being apparent. This would result in a pure bred Pekingese always testing more pure than a pure bred Yorkshire Terrier. You would have to presume that accuracy for any given breed would vary, depending on the age of the breed.

All the above is hypothetical and I reserve the right to be completely wrong about this :p. But for the little that I've seen about it, the Mars test is basically useless information for mixed breed dog owners and would likely be wrong information for many pure bred owners. I certainly wouldn't rely on it to make any decisions regarding breeding stock. It (the test) could lead to interesting things in the future but seems to be of little practical value today.

One dog that I owned I had tested twice. The dog was some sort of a dalmation/australian shepherd mix. The first test came back as 75% dalmation, and 25% unknown. Six months later the dog tested as 35% labrador, 15% dalmation, and 50% Husky. Tests are completely bogus.

I kept reading this post over and over again. Didn't make too much sense. Then I suddenly realized who you are. Oh brother.

The hidden clause in all this is a member of YorkieTalk going after another member. Another reason why I don't post too much here anymore. Too one sided by the far left. Hey Dan?

Oh, BTW I contacted Dr. Bannasch.

The OP definitely seemed to have an agenda to me, too. Not sure what it is though!

As did the original group/people that came on here and posted about the Mars test. A test to determine what your mutt is and are using it to convince people (AKC) that they have a purebred dog from 3 different breeds. People that have disagreed have been called stupid housewives that don't know anything.:eek:

I know what the agenda is for this post and it is to bad people can't say why they do not agree with the tests and or the club without being pulled through the wringer.

No, you are wrong. The OP DID agree with the testing when it benefitted he/she. They totally agreed with the test because their Biewers were in on it. Ask the OP what suddenly happened. See,this is why so many of you don't understand who is and is not posting on here. You are only getting one side of the story.

I believe in the MARS testing because it goes beyond the Wisdom Panel. The testing is not what most people on here think it is. The MARS people were hounded and haunted and they could not answer their questions because they are not the ones that were used to do the study. The study is not finished, and for one geneticist to call it bogus from the original OP, I find that very insubordinate to their fellow colleagues. Also, nobody really knows the story behind Dr. X. That was an insult to all you breeders on here because I took that as you have to have a PhD to be a breeder. Those who know about Dr. X knows it was said out of respect to that person for their knowledge. Too bad somebody has to take it out of context. Shame on you!


This is my take on the whole thing. Everybody just needs to settle down about the Biewer Terrier, and the Biewer a la pom pon. What ever the clubs believe in, let them be. If one club is reaching out for AKC, so what? If another club wants to have Biewer Yorkshires and hopefully get them accepted under parti yorkies, so what. Everybody just needs to back off and let the breeders of the clubs alone and take their own route. Thats my take on this whole thing.

Interesting!!!! Every statistician knows that the same statistics can be used both for and against the same issue.

Where is the thread that you say the "original group/people came on here and posted about the Mars testing". I would like to go back and see what they had to say about the test.
Judy

I wonder what the Mars people would have to say about these outrages accusations made by this OP? Would they take kindly to being called bogus on an open forum?
Sour grapes is what we are looking at here. I was not going to post but I find it very sad that OP was 100% behind all of the testing and would still be if not for the unfortunate circumstances of his removal from a certain club.

I don't know which dog OP had tested but none of the dogs we had tested came back with 75% dalmatian? HMMMMMM

Why is it that nobody talks about the statistics, and the lack (orpresence of scientific studies? This thread was not supposed to be about someone having hurt feelings--it is supposed to be about hard, verifiable knowledge.

....and, yes, it is true that I was in favor of the Mars test--until I learned about the inaccuracies.

No one is called Dr/Ph.D. out of "respect" when they haven't the education....even if the person believes they should be called that or their followers (like in a cult or secret religion) want to call them that.....

.....back on subject which is the Mars test is inaccurate.....and if some other lab is doing further work on the initial findings from the Mars Laboratory wouldn't that be more inaccurate findings based on the original inaccurate findings???

It drives them crazy that they don't have evidence to back up their claims. Of course they would know that if they were as educated OR as experienced as they claim.

The real world just is not a computer that you can re-mold to fit your whims. You can't just create facts because they are convienant..By being unable to present evidence, as shown by their hurtful vindictiveness, they convict themselves.

As the old saying goes,"If you can't defend the client (idea), attack the prosecuter."

They seem to be really short on knowledge--that's not something I usually care about--unless those people try to hoodwink others.

the OP never said he tested a biewer that came back 75% dalmation. he said he tested his dalmation/australian shepard mix breed dog twice and it came back with two different answers.

I know the test results from some owners' who've had their biewers tested
and here's a list of the breeds found -

Yorkie, Borzoi, Golden Retriever, Labrador Retriever, Manchester Terrier, Chihuahua, Japanese Chin, Maltese, English Cocker, Spaniel Chinese Crested, Basset Hound, Springer Spaniel, Pekingese, Shih Tzu....

Now according to Mars themselves...they are only able to pick up markers within 3 generations....there is no way to breed OUT the characteristics of the breeds listed above within 3 generations.

I don't think it takes a rocket scientist to figure out the accuracy...

One dog that I owned I had tested twice. The dog was some sort of a dalmation/australian shepherd mix. The first test came back as 75% dalmation, and 25% unknown. Six months later the dog tested as 35% labrador, 15% dalmation, and 50% Husky. Tests are completely bogus.


His statement not mine.

re: MARS Test Accuracy????
 

Dear Yorkie Talk fans,


This is Dr. Neale Fretwell, PhD, chief geneticist for Mars Veterinary. We appreciate your inquiry and I am happy to explain as mixed breed genetics is a very complex topic.

Mars Veterinary designed Wisdom Panel MX to specifically detect the presence of two (or more) breeds in a mixed breed dog. By definition, a mixed breed is a dog crossed with two or more breeds in its recent ancestry.

You ask a great question about the accuracy of the test. Back in 2007, when Mars Veterinary launched Wisdom Panel MX, the accuracy was 84%. We continuously work to improve our accuracy, so since launch, we have improved our breed count and accuracy level to 90%. While it is not yet at 100%, we are very proud to boast industry-leading accuracy. You can read more about our test enhancements since launch at http://wisdompanel.com/care_resources/view_news_item.aspx?id=28


How do we validate our accuracy?
The accuracy of the test has been statistically tested and validated on with a large panel of known first generation cross-bred dogs of proven heritage, featuring a wide range of different breed combinations.

Because Mars prides itself on providing the most reliable and scientifically sound products available, the extensive test development, performance testing and validation protocol, was fully reviewed by experts from an external peer review panel. This panel consists of world-renowned veterinarians, statisticians and canine geneticists.


About the Study:
Wisdom Panel MX was run on blood samples collected under veterinary supervision from more than 180 F1 hybrids (i.e. Goldendoodles, Puggles, etc.) bred from parents registered to a recognized Kennel Club. More than 25 breeds were represented in the sample set.
Two metrics were calculated, positive predictive value (PPV) and sensitivity of breed detection (S). PPV (true positive breed calls (TP) / (TP + false positive calls) was 90%. S (TP/ (TP + false negative calls), was 97%. This work has been presented externally on two separate occasions. First, at the American College of Veterinary Internal Medicine (ACVIM) meeting in Seattle, Washington (July 2007) and more recently at the World Small Animal Veterinary conference in Sao Paulo, Brazil (July 2009). The abstract for this presentation was published in the proceedings of the meeting.

Every Dog breed (of which over 400 are recognized worldwide) contains many individuals, each of which are comprised of a very complex mixture of many different genetic variants, and as such, what constitutes or defines a member of a breed is established by the rules determined by the various kennel clubs and breed registries (described for each in the established breed standards), not by Mars Veterinary. These rules generally deal with common physical traits between members of a breed and a register of stated pedigrees from other breed members than any scientific data.

The observed diversity of breeds and genetic variation within breeds, together with the fact that most recently created breeds are crosses of other existing breeds, means that occasional inaccuracies are inevitable in any test designed to establish recent ancestry from a specific breed.


We are aware that on occasion, dog owners have attempted to use the test to prove the "purity" of a dog from a given breed. This is not a purpose for which the test was designed and we do not advocate its use for this purpose. We expect testing of purebred dogs with Wisdom Panel MX to cause inaccuracies in the results of the analysis because our statistical algorithm was designed for the purpose of finding multiple breeds in a dog’s ancestry.


Our Work with Purebreds:
Additionally, Mars Veterinary geneticists’ have extensive experience with purebred canines in addition to mixed-breed dogs. They have participated in a number of research projects using the proprietary genetic markers used by Wisdom Panel MX. Some of these studies include; the genetic variation within breeds, and a study to help identify potential ancestral breed influences of newly developed breeds and breed variants. These projects have primarily used more appropriate methods to study the variation between purebred dogs, rather than the algorithm that is used for Wisdom Panel MX.

Since launching, Mars Veterinary has proudly worked with many enthusiastic veterinary practices, mixed-breed dog owners and industry experts who are excited about the preventative health and wellness benefits available through learning a dog’s breed history with the Wisdom Panel MX.


For any additional questions, myself or our Veterinary Geneticist Dr. Angela Hughes, DVM, would be happy to explain in more detail. Please contact us via our customer care department at customercare@marsveterinary.com.

Warm regards,
Dr. Neale Fretwell, PhD
www.wisdompanel.com

Thank you Dr. Fretwell. :thumbup::thumbup::thumbup::thumbup::thumbup::D

"We are aware that on occasion, dog owners have attempted to use the test to prove the "purity" of a dog from a given breed. This is not a purpose for which the test was designed and we do not advocate its use for this purpose. We expect testing of purebred dogs with Wisdom Panel MX to cause inaccuracies in the results of the analysis because our statistical algorithm was designed for the purpose of finding multiple breeds in a dog’s ancestry."

There in her, Dr.Fretwell, own words, not to be used to prove lineage of new breed.


(and btw Marketing 101 and Statistics 101 are not the same classes.)

Yet even the good researcher, Dr. Fretwell, does not tell us what the Type I error rate and the Type II error rate is. He is continuing to make the same claims as are made on their web site.

Without data, there can be NO validation NOR any acceptance of the tests. It doesn't matter who someone is, what they beleive, nor who they work for. All that matters is the data.

Why is everyone afraid to provide the hard data. If they are honest, then what are they hiding?

Just out of curiousity why don't you contact him and ask him? He said he would be willing to answer questions:)

During the K9 College Cruise, I had the pleasure of taking a course with Dr. Danika L. Bannasch, DVM, Ph.D. (Click on her name for more). She is a geneticist at University of California Davis. Her work centers on dogs and horses. I asked her about the accuracy of the MARS test. She stated, “It is completely bogus.” I was unable to get her to define, “bogus” in more statistically relevant terms, but she was insistent that the tests did not provide a level of accuracy.

It seems to me that you are placing more validity in someone that has never conducted any of this testing or where are her statistics? What facts is she basing her statements on? Has she read the research papers and done her own analyzes?
Dan, you should know that you are skating on thin ice. I would think twice before I would call a very reputable institute's research bogus. You have now taken it upon yourself to publicly defame Mars and Dr. Neale Fretwell. I am sure that his expertise in the field of genetics far surpasses any knowledge you might presume to hold.

Has any of the Mars testing been submitted to an outside non-biased source for confirmation?

I generally don't read these threads, but this one intrigues me. The most interesting point is that the scientists who developed this test also have a financial intrest - meaning this is a product/idea that they sell. Where is confirmation of the Mars Test by an outside non biased source that has no financial interest in the outcome?

:yeahthat: