I would say that it could be both. What makes it one or the other is the reason behind the exclusion. "I don't like black/white/Asian/Latin/etc. people, they're dark/wimpy/flat-chested/crazy/etc." I would consider racist. On the other hand, "I prefer 'enter ethnicity here,' because I find their features appealing," is not. The difference is reasoning. If you dislike a group on principle, because you think they're below you, dirty, or something, then that's one thing. If you're simply not attracted to them, well, there's not much you can do.
I tend to prefer Asian, or Latin, features. Though, I'll admit there is the occasional black, white, Arab, etc. man that might turn my head. The only people I have a REALLY hard time being attracted to are Indians, I don't care for the color of their skin, normally. Oddly, though, I thought Freddie Mercury, an Indian Parsi, was extremely attractive. But, he didn't have the typical Indian celebrity chubbiness/softness in the face, or the fatter nose, and whatnot.