Welcome to Gaia! ::

In the Sci&Tech, there is a creature of mild interest who goes by the name of Kaz-Balan. She asks why scientists believe in the general theory of relativity instead of the simpler theory of Newton and the Aetherists. While not a particularly intelligent question, it does have some pedagogical merit, so I attempted to answer her.
Here's a slightly less context-dependent assortment of my thoughts on why physical theories end up being complicated:

There are, from what I can tell, four main points to consider:
1: Accuracy: The important of being right
2: Integrity: One thing versus many things
3: Intuition: Simple but not easy to understand
4: Language: Intrinsic versus linguistic complexity

1: The first point is the most relevant to science. The goal of science is not to be pretty, but to be accurate. Art can be pretty all it wants, but science has the rather dirty job of trying to predict the universe and to do so requires accuracy more than it requires beauty.
There are many ideas that are simple and wrong. "Blue is the only color" is simple but demonstrably wrong. We can easily show that there are colors other than blue. So the statement "blue is the only color" is useless scientifically, even though it is a very simple statement and is indeed simpler than acknowledging that there are other colors.
Would it be better to say "Blue is the only color", or to go through the rather long list of colors that actually exist? Is it better to have an idea that is simple and wrong, or an idea that is complicated and right?
We used to think that planetary orbits were all perfect circles, but now we know that they're more like ellipses. Circles are simpler than almost-ellipses, but that's not what orbits are shaped like. So we don't care about circular orbits anymore.
Blue as the only color is simple. But it's wrong. Circular orbits are simple. But they're wrong. And so we discard these ideas for ones that are harder to explain but aren't wrong.

2: Suppose you see a square shadow on the sidewalk. Later on, in the same place you see a rectangular shadow. And later still you come back to the spot and find a hexagonal shadow. So you have something that is sometimes a square, sometimes a rectangle, and sometimes a hexagon.
Alternatively, you could look up and see that the object casting the shadows is a cube. A cube is a three-dimensional object, inevitably more complicated than a two-dimensional object like a square or a rectangle. On the other hand, a square, a rectangle and a hexagon (and the other shapes that the shadow of a cube can take on) together are more complicated than a single cube. So which is simpler, many simple objects, or one slightly complicated object?
Another example is a Newtonian description of the Universe versus an Einsteinian one. Newton's description has space, time, forces, fields, mass, and many kinds of energy. Einstein's has just spacetime and energy, and there is only one kind of energy in Einstein's universe. Which description is simpler?

3: Something being simple does not mean that it is easy to understand. The number 1 is simple, but I would not say that the concept of 1 is easy to fully understand. Our notion of 1 is very dependent on rather silly things, empirical observations, hand-picked lists of acceptable examples. We can say 1 cat, but it is harder to say 1 milk, or 1 air. Can you explain why? Why are all 1s the same, except for 1 milk or 1 air?
Truth should be simple, but philosophers have been struggling to define truth since the beginning of philosophy. Things that are simple are often difficult to understand because we do not perceive a world of simple things. We perceive a world of complicated things, and we perceive a world far away from the things that are fundamental. The world we perceive is built upon many, many layers of interactions of fundamental things, so in order to get to the things that are fundamental, we have to go through all of these layers. Thus even if the fundamental things are simple, we have to say a lot to describe them in terms of what we perceive.

4: A lot of mathematics and physics looks pretty ridiculous linguistically. Jargon, jargon, jargon everywhere. It might as well all be in another language, mainly because it is. Natural language isn't terribly good at describing things precisely; it's not built for that. Natural language is built for communicating the terribly imprecise world we experience.
So mathematicians and physicists make up words to describe things precisely, and describe things in terms of these made-up words. And if you don't know these words, it's just like trying to read a different language. Of course it's going to be difficult for you. That doesn't mean that it's difficult, just that you don't speak the language.
It's like going up to a non-Greek speaking child, giving them a text in Greek and blaming the Greeks when the child doesn't understand what the text says. "Why must the Greeks make everything so complicated?" you moan, flipping through your Greek-to-English dictionary, never minding the Greek child half your age flipping through their English-to-Greek dictionary and cursing the British for spreading their multiply-bastardized language.

To recap, there are four things to keep in mind:
1: Accurate but complicated theories are more useful than simple but incorrect theories
2: A single complicated but comprehensive thing is simpler than many individually simple things
3: Simple and especially fundamental things appear more difficult to understand because we don't experience fundamental things
4: Mathematics and physics are written in a different language out of necessity, but that doesn't make it more complicated, just different.

The difficulty with that comparison comes from mainly point four, in that the metrical description of GTR certainly appears more complicated than GMm/R² and the usual forms of Newton's laws. Sure, if you translate Newton-Cartan gravity into a metrical form, you get something obviously more complicated than GTR, but Newtonian gravity doesn't seem to necessitate such a translation, and hence appears simpler due to language.

You must admit that the language barrier (and, perhaps, the math and abstraction barrier) is a pretty strong one, however. Physically and mathematically simple differ quite a bit from linguistically/pedagogically simple. You can teach Newtonian gravity in terms of Newton's formula to high-school kids. GTR? Not so much.

I fully agree with you on that point; theories are getting simpler in terms of the number of fundamental concepts needed per phenomenon, and the language in which these theories are cast are getting "simpler" alongside them, at least internally.
Externally is different. The need to construct new languages (or perhaps I should say paradigms? Branches of mathematics? Wikipedia pages?) makes the teaching aspect difficult, even if the theories in the abstract are simpler. Each language is progressively simpler, but each word carries more and more meaning, with more background knowledge necessary to comprehend it in terms of the colloquial. It doesn't help that the languages typically give their definitions in terms of previous theories, so that newcomers must chase the chain of definitions and concepts all the way back before reaching English.
We end up with a physics-to-English translation dictionary with only a few entries, but each entry is cast in terms of different set of auxiliary translation dictionaries, each containing themselves but a few entries, which are defined in terms of the contents of yet more translation dictionaries. This is the natural structure of knowledge but from the bottom of the pyramid one does not see the single block at the top, but rather the myriad blocks at the base. I have a metaphor problem.

That doesn't seem very respectful. Did she do something to piss you off?

Well, OK then!!!

You seem to possess a certain amount of passion over this. Keep in mind you didn't actually waste your time, as I'm sure threads you've posted to in an attempt to educate this person surely educated others.