I don't like the word colored because that reminds me of what happened in Apartheid South Africa, where they used that as a term of division.
I think 'people of colour' is pretty much everyone who is 'colourful' and thats a more positive way of using the word. Diversity is also another.
In NZ we have a different way of speaking about different ethnicities, still has some problems, but most of them are not derogatory, some are affectionate or seen as a positive way, also it is in the way you use or say the word.
We tend to not grade people by skin colour, that's a holdover from slave/apartheid times. In NZ we did not have that, but in the Pacific there used to be 'blackbirding' and also, White Australia was super-racist. Indigenous or first nations people have raised the issue and it's only right that they have an apology on behalf of the govt that treated them so badly, as if they were not human as well.
At the store I worked at in an area with the great divide between whites and blacks, people didn't quite know how to take me -- because I was neither.
I had a lot of fun with that.
I'd tease my white bosses with things like "What about the minorities here?!" and they'd laugh because most people saw me a white. Technically, they knew I was a minority, but they usually didn't "see" me as one.
One day I was talking to a black co-worker and when our white co-worker (who'd gone to the all-black schools) walked up, I said, "I'm sorry, but this conversation is just for US PEOPLE OF COLOR," and they both started rolling with laughter -- and telling me I didn't count as a "person of color," because they saw me as being stark white.
When I went to college, I worked at a cafe where our delivery guys were a white guy and a black guy who were best friends.
One day I somehow mistook the white guy for someone else, and jokingly brushed it off with, "All you people look alike!" The black guy started laughing so hard, he was gasping for air.
Now I realize some might see this as me of perpetuating stereotypes while later accusing others of racism.
The thing about me is that I was always in-between, never all or nothing, so people never saw me as being completely one thing or the other.
But people also realized that because I was still "different", I was joking from a place of experiencing racism in a way no know understand unless they've gone through it themselves, no matter what race they were -- and that's why they would laugh.