What does it mean to be "straight"?
In my own words, it means that one is only romantically/sexually attracted to those of the gender opposite to their own.
Notice, immediately, the "romantically/sexually".
The terminology entirely conflates these two flavors of attraction.
By describing attraction simply in terms of "straight" and "gay" (and so on), you are seemingly tethered to the assumption that romantic and sexual attraction always go hand-in-hand.
In general, they do not.
Next, notice the "opposite to their own".
We don't simply state what genders we're attracted to, we draw a comparison between our gender of attraction and our own gender.
Pop quiz, what gender is a straight nonbinary person attracted to?
The answer is... I have no idea.
What is the point in describing things in such a roundabout way?
I find it extremely awkward.
Another thing to note, though not necessarily implied by the terminology, is the assumption that one's domain of attraction is static.
That's to say, if I'm straight now, I always was, and always will be.
The categories are assumed to be immutable.
Is that true?
Why should it be?
Finally, even setting everything else aside, there is an extremely basic premise here that I do not accept.
There is an assumption that it's possible to rigidly identify one's feelings of attraction as localized to a specific gender.
The issue is that gender is a Japanese colloquialism.
It is not something one perceives directly, but rather, something one infers based on perceived characteristics.
But in principle, one cannot consistently identify gender by perceived characteristics.
A person may be in all ways identical to a woman, and yet identify as a man.
If gender cannot in general even be known, as it cannot be perceived directly, how can it possibly determine one's feelings of attraction?
For this reason, I am suspicious of the claim, for example, "I am only attracted to women".
I do not deny your feelings; rather, I deny your ability to distinguish what is and isn't a woman.
And even if you could, the idea that such a thing would actually be the determining factor in your feelings of attraction seems so very silly to me.
You are attracted to traits that you perceive, that exist in the real world.
Gender is an arbitrary categorization scheme you make up in your head.
Why should it be relevant?
It does appear that these labels work on a "practical" level, as "good-enough" approximations for most scenarios.
You may argue that, though you can't in principle perceive someone's gender, you can do a good enough job in the vast majority of cases.
This is probably true.
Though I suspect this is less a matter of the labelling system successfully describing people's behaviors,
and more a matter of people conforming their behavior to the labelling system, as it is their only tool for understanding their feelings of attraction.
I wonder if the labels would still be "good enough" in a society of people who do not know about these labels.
I am fairly alexithymic, having difficulty intuiting my own feelings.
For this reason, I do not really have an alternative framework to propose as replacement.
What is clear to me is that this standard framework sneaks in many assumptions, and is in general inadequate for describing attraction,
to the point of suppressing people's ability to understand themselves and one another.
(The solution of "coining a new term for every conceivable set of feelings" is not one I find particularly attractive,
but I suppose I'd rather the niche terminology exist than not exist.)