True that X is the Greek symbol for Christ.
But most people who use Xmas instead of Christmas do it either to leave Christ out, or just out of laziness.
The holiday is Christmas, that's what the calender says. Each of us have the right to celebrate it as a religious event, as a secular holiday, or to not recognize it at all. Why do we need to be all politically correct, when we are free to make these choices?