Home What's Brewing? Bow WOW! Microsoft App “Fetch” Identifies Dog Breeds

Bow WOW! Microsoft App “Fetch” Identifies Dog Breeds


Fetch__New_Microsoft_Garage_app_uses_artificial_intelligence_to_name_that_breed___News_CenterHow do you demonstrate that object recognition is something anyone could understand and interact with? Ask Microsoft’s Mitch Goldberg.

 “We wanted to bring artificial intelligence to the canine world,”said Mitch, a Microsoft Research development director who leads the Cambridge, U.K-based team.

Under Goldberg’s team, Microsoft developed Fetch. Using your iPhone camera or photo library, Fetch can identify and classify dogs by breeds and tell you what kind of human personality fits best with specific breeds. Just for fun, the app will even take an informed guess on what kind of dog you or your friends might be.

The breed-identification technology was first shown at a Microsoft Research event two years ago, and has evolved in the time since to a consumer-friendly app, with some extra features. In addition to identifying the breed of dog within a certain degree of certainty, for example, the app can take an “informed guess” as to what type of breed a person most resembles.

Released through the Microsoft Garage just in time for the American Kennel Club’s Meet & Compete and the Westminster Kennel Club Dog Show, this mobile app demonstrates the potential for Microsoft  researchers’ continued advances in artificial intelligence, which have already appeared in other playful ways through Microsoft Project Oxford-powered experiences such as HowOld.netTwinsOrNot.net,  MyMoustache.net and Mimicker Alarm. In Fetch!, Project Oxford works together with some powerful new machine learning technology to deliver interesting results for all kinds of photos.

Fetch__New_Microsoft_Garage_app_uses_artificial_intelligence_to_name_that_breed___News_CenterFetch! is designed for repeat use, and after giving it a couple tries, it’s easy to see how addictive it can be. You start with your dog, or your friends’ dogs. If the dog’s breed is unknown, the app will show a percentage of the closest breed. Tapping the percentage rosette leads to the top five breeds that could be in the dog. Clicking on the arrow in the corner leads you to more information on the breed.

“If you want to take photos of dogs, it will tell you what dog breed it is, if it’s one of our supported breeds,” Goldberg says. “If I choose to take a photograph of a flower, it’ll say, ‘No dogs found! Hmmm… This looks more like…flower?’ But if you take a picture of a person, it’ll kick into its hidden fun mode. And in a playful way, it’ll communicate to you not only what type of dog it thinks you are, but also why. It’s fun to see if the app knows it’s not a dog. A lot of the time, it’ll tell you what that image is. When there’s not a dog, you still want to use it.”

No two pictures yield the same result. You could resemble a Doberman Pinscher in one photo (sunglasses, no makeup) or a Pekingese (no glasses, makeup) in another. If you photograph an inanimate object, it might tell you, “No dogs found!” and make an informed guess at what it is.

If you like what you see, you can share the image on your social networks and through email.

Two years ago, in July 2014, a demonstration of the technology behind Fetch! appeared on stage at the 15th annual Microsoft Research Faculty Summit. Dogs stole the show, as they always do.

Even then, the technology was able to tell the difference between people and dogs, with Cortana saying, “I believe this is not a dog.”

In the two years since that demo, a lot has happened.

The team has developed a website, What-Dog.net that has photos you can play with to find out about the breeds, and you can also submit your own photos and share them.

Download Fetch! for free for iOS from the Apple App Store or via What-Dog.net and learn more about it at the Microsoft Garage. [24×7]