The kind of ether you're thinking about; a ubiquitous medium that fills the universe through which light propagates contradicts relativity directly and has never once been observed.
More specifically, nothing predicted by the ether hypothesis have ever been observed (and not for lack of trying), whereas relativity was a huge increase in our ability to predict the world, one which has stood up to ever finer degrees of scrutiny as time goes on.
Why would anyone say "of course" ether exists? Is there an experiment which gives results most consistent with the existence of ether? If so, they were published in a very poor journal, because as of today modern science has not been turned upside down by those findings.