They are strong squirmy little things to be so tiny aren't they! Maybe try a warm wet cotton ball. Is mommy licking them and attempting to keep them clean??? Mine are almost six weeks old and Bitsy STILL can't stand it if something is on them and she cleans and licks til it's gone if I let her
If you can get a clump of it soft enough and mash it out with your fingersnails, that's what I had to do with the dried milk on mine. Now I use baby wipes to clean them after each feeding. I swear it's just like glue especially when you mix it with the step two food. Let us know when you find out what it is!!