As someone who has spent the better half of her life on top of a 1000 pound animal that could kill me at any second, people who tell me "horseback riding is not a sport" just might receive a punch in the face from me.
First, let me tell you that I could probably beat you at arm wrestling among other strength tests simply because I have to carry large bucket of water around the barn, lift hay bales, and 50 pound feed bags. All of the work in the barn makes me sore and at that point, I haven't even gotten on my horse yet. Those who play football and soccer may get beat up a bit, but when they fall, it's over their feet. I fall over 5 feet and it's on either the gravel or a hard dirt arena. Last time I checked, a soccer ball doesn't have a mind of it's own, and a football isn't going to kick you in the face if you accidently walk behind it without warning.
There are multiple reasons why horseback riding is a sport and I'll most likely get into more detail later. Before you go up to an equestrian and tell them that horseback riding isn't a sport, just think for a second about what you're saying. Go jump on a football player, kick them in the ribs and tell them where to go. Now go do that on something ten times bigger and don't forget that horses don't speak English! Nonverbal cues only, and do it flawlessly.