Yizkor: Who Stays In and Who Goes Out

What are the reasons for saying Yizkor on Yom Tov? Why don't we say it on Rosh Hashanah? Why do people walk out for Yizkor... and is that proper?