This "race card" keeps giving me a losing hand

Terms I hate: the race card (you know because racism is just a game after all. Like a more discriminatory version of Blackjack! Why I bet the black cards have to use different drinking fountains from the red cards!)

If you're a person of color, a commenter on a message board that casually mentions the dreaded "R" word, or a put upon member of Nevada's assembly you've probably heard this question (or some variation) asked by some frustrated but cluelessly naive  (or contrarian troll):

Why do they keep playing the race card?

Um...because the dealer keeps handing it out. Where do people think it comes from? It's not some ace that's been hidden up our sleeve (after all, as mentioned in the title of this post, it's not like it's some successful trump card.) Why do "we" keep talking about it? Because it keeps coming up. Often a narrative boundary is superimposed on the issue stating that since the Civil Rights movement that has "won", racism should now be declared "over." You know, like how after a comic book arc where Batman defeats the Joker in which case the Joker is defeated for all eternity and he never becomes again, the end (Batman: "well I've now gotten rid of all crime in Gotham. Guess I gotta spend my time snorting blow off of supermodels now!") That sound you hear is millions of comic book geeks laughing. And yet people keep interpreting "victory" this way in real life, ignoring such concepts as "push back" and the fact that such achievements aren't set in stone.

Ultimately I would answer the above question with a question: why is talking about race considered a bigger sin that actual racism? Or to beat the metaphor I've been using into submission: why do we complain about that "race card" instead of trying to remove it from the deck?

UPDATE: Sigh!  In a sad and infuriating turn of events,  it appears that, nope we still haven't removed it from the deck. Yep there goes the video camera playing the "race card," making a policeman look like a possible murderer just because he shot at an unarmed black man. In the back. Multiple times. Over a traffic violation. Ugh!