This is another one of my favorite songs from the early 80's, from Dirty Mind (1980). This is from a Paris concert (1981). Of course, there's also a great version on Cyndi Lauper's She's So Unusual.
Prince - When You Were Mine (live mp3)
Find this track and a treasury of live Prince material here.