I realised also that I don't have a problem with "Happy Holiday", singular. Because in that case it doesn't matter which of the religious or secular festivals the person is celebrating, nor are you trying to make any assumptions. Instead, you're wishing that the other person will have a peaceful rest from work and/or an enjoyable time with their family for a few days. Holiday in the sense of "public holiday" rather than the sense of "festival".
Yes, I am weird. This is not news.