she may also be scared of the baby. baby noises are loud and unexpected.
i read somewhere you should take something the baby sleeps with (ie blanket) and introduce it to your little one. this way the scent becomes part of the household. hopefully you have a little time to do this.
next thing you know, the baby will cry and your new "babysitter" will be all over you like it's time for dinner!
"mom! mom! the baby! it's making that noise! mom!! mom!!" etc and i don't mean your 4 year old