First, Welcome to YT! This site is full of great Yorkie knowledge and the members are generous with their knowledge - I can safely say that having been the benefactor of so much help!
Clearly, Indy has not been trained or socialized!
I am sure others will come forth with their help, also
There is a booklet, called HOW TO HOUSEBREAK YOUR DOG IN 7 DAY, by Shirlee Kalstone, which is a brief and concise look at the potty training issue (which Yorkies can be very stubborn about)! However, don't plan on the 7 days, as Indy has some things to un-learn as well, but i would hope it would give you a place or two to start. It is pretty strongly suggested that dogs that are not house broken need to live in a very restricted area until they "get it". Have free range will make it worse.
If you vet does not believe the circling is neurological, then you will just need to go with that for awhile until you get a better handle on the who potty training thing. Good news that he walks well on a leash.
I'm sure he does NOT know how to play if he has been socialized much. The main key will be patience. If you fuss at him he will withdraw - positive reinforcement is best (which you probably already know) - treats and praise when does what you like. It takes longer to in-learn than to learn so take your time.
Bless you for giving this pup a new (and more normal) pet life
I personally believe that positive basic obedience training is good for all dogs. Helps them with anxiety and to understand what is expected of them and that good dog acts pay off!
Keep us posted please!