Yellow is only preferred by five percent of people as their favorite color according to a survey by Dulux Paints. But I do not think that is the reason why it is not used in paintings that often. Think about it, how much yellow can you see when you look out your window? Maybe we just do not see that much yellow in our everyday life so we tend to forget the color as artists too.
I have the same theory about purple. However, looking at different surveys, research, and diagrams, purple is actually more popular than yellow.
Interestingly, however, both yellow and purple are more popular as a favorite color than orange for most people! So, I guess that goes to show that a persons favorite colors has nothing to do with how common the color is in art.