I think maybe that we tend to side with women (or at least I do) because historically they have been on the weak side of the power balance. So then we see something where the victim is not weak, and we question our prejudice and want to be “fair” even when we cannot possibly have enough insight into either party to know either way.