It should definetly be used sparingly if you're a white dude. Dave Chappelle was talking about his "Black White Supremacist" skit, and on the streets, a white guy would come up to him and say something like, "Oh, that sketch about them niggers was funny!" He wouldn't be pissed, he'd just be shocked.
Also, once, i was hanging out in Cambridge with my friend Luke and a couple of girls (All four of us where white), and Luke was giving one girl a piggyback ride. They fell, and a couple of big black guys came up, and said something like "Aw, *****, are you alright?" So that was kinda strange. But, either way, I try to use it very, very sparingly.