When someone says that what you said/did etc was racist they arent always calling YOU a racist they are just pointing out that you took part in an activity which has racist implications.Weather it was your intention or not is irrelevant.So Instead of getting defensive about it and bringing up all the times you “werent racist” or talking about your friends of color you can just learn,apologize and not do it again in the future.