Hi Charlie,
If you can avoid cleaning the eggs I would say don't wash as the dirt can transfer through the egg and cause illness, defects in the chicks or even chicks not hatching.
If the eggs are really unclean there is sometimes the chance that this can contaminate the incubator so if you do decide to wash then as Jeni says use warm water. The water needs to be warmer than the egg so that the dirt doesn't get through the porous shell and don't rub too hard as this will rub off the protective layer on the shell.
Here is a discussion we had about it earlier this week http://farmingfriends.com/forums/topic.php?id=470
You can get a disinfectant I think I mentioned it in the above posts.
This is always a tricky one to answer. I would say if they are not too dirty then put them in unclean.
Kind regards
sara @ farmingfriends