Sounds like the vet was a bit hindered due to your fear of xrays. I would hope he attempted to assure you that it is not that harmful. I would personally rather be safe than sorry. There is no way as you know to definitively say there is no fracture without an xray. I do hope it is not fractured.
Dog Broken Bones - Broken Bones in Dogs | petMD Common Injury in Dogs – Bone Fractures