Updated: Dec 10, 2020
Roger Penrose is not only a mathematical physicist: he’s also a pure mathematician. So it’s not a surprise that Penrose expresses the deep relation between mathematics and the world (or nature) in the following way:
“[T]he more deeply we probe the fundamentals of physical behaviour, the more that it is very precisely controlled by mathematics.”
“[T]he mathematics that we find is not just of a direct calculational nature; it is of a profoundly sophisticated character, where there is subtlety and beauty of a kind that is not to be seen in the mathematics that is relevant to physics at a less fundamental level.”
Penrose is (rather obviously) profoundly aware of the importance of mathematics to (all) physics. Yet, more relevantly to this piece, he’s also aware that maths alone can sometimes (or often) lead the way in physics… and sometimes in a negative manner! So despite the eulogies to mathematics above, Penrose offers us these words of warning:
“In accordance with this, progress towards a deeper physical understanding, if it is not able to be guided in detail by experiment, must rely more and more heavily on an ability to appreciate the physical relevance and depth of the mathematics, and to ‘sniff out’ the appropriate ideas by use of a profoundly sensitive aesthetic mathematical appreciation.”
Platonism and Pythagoreanism in Contemporary Physics
Roger Penrose is a Platonist, not a Pythagorean. (Or at least he’s a Platonist in certain respects — see here, here and my ‘Platonist Roger Penrose Sees Mathematical Truths’ ) One reason why this can be argued is that Penrose admits that he
“might baulk at actually attempting to identify physical reality within the reality of Plato’s world”.
To the Pythagorean, the world literally is mathematical. Or, perhaps more accurately, the world literally is mathematics (i.e., the world is literally constituted by numbers, equations, etc.). That may sound odd. However, if we simply say that “the world is mathematical”, then that may (or does) only mean that the world can be accurately — even if very accurately — described by mathematics. The Pythagorean, however, states such phrases as “things are numbers”. He therefore establishes a literal identity between maths and the world (or parts thereof).
To the Platonist, on the other hand, the mathematical world is abstract and not at all the same as “physical reality”. (Plato often actively encouraged philosophers and mathematicians to turn their eyes — or souls — away from the physical world.) Yet it’s still undoubtedly the case that abstract mathematics — even Platonic mathematics — is a fantastic means to describe the world. Despite that, the Platonic world is still abstract and not identical to the physical world. In other words, there is no identity between the physical world and the Platonic world. However, there is an identity between the physical world and the Pythagorean world.
“this mode of observing nature, which led in part to a true dominion over natural forces and thus contributes decisively to the development of humanity, in an unforeseen manner vindicated the Pythagorean faith”.
All that may depend on what Heisenberg meant by the word “Pythagorean”. After all, it’s often the case that the word “Pythagorean” is simply used as a literal synonym for the word “Platonic”. Thus having said all the above, such distinctions between Platonism and Pythagoreanism (at least in these specific respects) may be a little vague or even artificial.
This may apply to Roger Penrose’s position too.
Take Penrose’s own (as it were) quasi-Pythagorean reading of the “complex-number system”. He writes:
“Yet we shall find that complex numbers, as much as reals, and perhaps even more so, find a unity with nature that is truly remarkable. It is as though Nature herself is as impressed by the scope and consistency of the complex-number system as we are ourselves, and has entrusted to these numbers the precise operations of her world at its minutest scales.”
Since the passage above is fairly poetic, it’s difficult to grasp exactly how Pythagorean it actually is. More clearly, surely the words
“[i]t is as though Nature herself is as impressed by the scope and consistency of the complex-number system”
are purely poetic — even if there’s a non-poetic “base” that’s expressed by the poeticisms. Penrose does, after all, prefix the statement above with the words “[i]t is as though”. So surely it can be said that Nature doesn’t need (or require) the complex-number system. It is us human beings (or physicists) who need that system in order to describe Nature.
In any case, Penrose’s strongest (possibly Pythagorean) claim is:
The complex numbers “find a unity with nature”.
Now is that “unity” also an identity? Not necessarily. After all, numbers may be united with Nature only in the sense that they can describe it perfectly. Saying that numbers are identical with nature, on the other hand, is something else entirely. As it is, the phrase “unity with nature” is hard to untangle. (Hence my use of the word “poetic” earlier.)
Having put a quasi-Pythagorean position on (at the least) complex numbers, Penrose then puts a (literally) down-to-earth position on the real numbers. Penrose writes:
“Presumably this suspicion arose because people could not ‘see’ the complex numbers as being presented to them in any obvious way by the physical world. In the case of the real numbers, it had seemed that distances, times, and other physical quantities were providing the reality that such numbers required; yet the complex numbers had appeared to be merely invented entities, called forth from the imaginations of mathematicians.”
Despite using the phrase “down-to-earth position” before the quote above, this passage is at least partly Pythagorean in that it states that
“distances, times, and other physical quantities  provid[ed] the reality” which real numbers “required”.
This can be read as meaning that the real numbers are (as it were… or not) embodied in distances, times and other physical quantities. Yet — historically at least — it seems that complex numbers didn’t pass that Pythagorean test.
Examples: Paul Dirac, Etc.
It’s undoubtedly the case that various well-known (as well as largely unknown) physicists have often been led by mathematics when it comes to their theories. That is, they certainly haven’t always been led by experiments or by observation.
Take the case of Paul Dirac.
Penrose calls Dirac’s finding of the equation for the electron an “aesthetic leap”. However, Penrose also says that it arose
“from the sound body of mathematical understanding that had arisen from the experimental findings of quantum mechanics”.
That basically means that although Dirac’s mathematics was (as it were again) pure, “the experimental findings of quantum mechanics” must still have been swirling around in Dirac’s head as he carried out his pure mathematics.
The Dirac case also shows us the to and thro between (pure) maths and experimental findings. That is, even if we have aesthetic and/or mathematical leaps, the mathematical physicists concerned were clearly still aware of the experimental findings which proceeded their abstract leaps. What’s more, Dirac’s own leaps were “made with great caution and subsequently confirmed in observation”. Indeed in both Dirac’s cases, confirmation came very quickly.
A purely philosophical slant can be put on the Dirac case. (Although I’m a little wary of shoehorning philosophical terms — or ways of thinking - onto what physicists have done.) As the philosopher James Ladyman (technically) puts it:
“Sophisticated inductivism is not refuted by those episodes in the history of science where a theory was proposed before the data were on hand to test it let alone suggest it... Theories may be produced by any means necessary but then their degree of confirmation is a relationship between them and the evidence and is independent of how they were produced.”
We can now say that in Dirac’s case there was no “data  on hand to test it let alone suggest it”. Actually, the last clause (“let alone suggest it”) may be a little strong in that previous experiments in (quantum) physics must surely have suggested various things to Dirac. The thing is, Dirac still had no (hard) data to back up his prediction or equation. Despite that, Dirac’s theories were “produced by any means necessary” (or by any mathematical means necessary) and only then were they confirmed.
To get back to Penrose.
“physical considerations — ultimately observational ones — have provided the overriding criteria for acceptance”.
Opposed to that, Penrose goes on to say that
“[i]n many of the modern ideas for fundamentality advancing our understanding of the laws of the universe, adequate physical criteria — i.e. experimental data, or even the possibility of experimental investigation — are not available”.
Penrose then concludes by saying that
“we may question whether the accessible mathematical desiderata are sufficient to enable us to estimate the chances of success of these ideas”.
All above shows us that Penrose is still acknowledging that (in a basic sense at least) the mathematics comes first. That is, Penrose believes that any “acceptance” of the “ideas” for “our understanding of the laws of nature” often comes after the (pure) mathematics. (That’s if the maths is ever truly pure in that previous experiments, observations, physical theories, etc. will — or may — be swilling around in the mathematical physicist’s head.) To repeat: the mathematical speculation (or theorising) comes first, and only then do physicists expect the “physical considerations” to provide the “overriding criteria for acceptance”.
When it comes to many (or some) “modern ideas” (Penrose mainly has string/M theory in mind — see here), on the other hand, “physical criteria” are “not available”. Yet that was also true — as we’ve seen — of the examples which Penrose himself cites (i.e., quantum mechanics, general relativity and Dirac’s equation for the electron). In these example, physical criteria were not available at the times these ideas were first formulated. This means that the observations, confirmations, experiments, etc. came after — even if very soon after.
So what if the experiments haven’t been done? Which precise experiments must guide the physicist? And what if there are no currently relevant or possible experiments which can guide the theoretical physicist? Of course it can now be argued that if there are no relevant, actual or possible experiments (or observations), then in what sense is any given physicist — even if mathematical physicist — doing physics at all?
String Theory and Penrose’s Twistor Theory
Despite Penrose’s emphasis on the fundamentally important role of maths in physics (which is hardly an original emphasis), Penrose is still highly suspicious of the nature of string theory.
Although Penrose doesn’t always name names, he still stresses “the mathematics that is relevant to physics”. He warns that
“if it is not able to be guided in detail by experiment, [it] must rely more and more heavily on an ability to appreciate the physical relevance and depth of the mathematics, and to ‘sniff out’ the appropriate ideas by use of a profoundly sensitive aesthetic mathematical appreciation”.
This squares with what British science writer and astrophysicist John Gribbin has to say.
Gribbin too talks in terms of what he calls a “physical model” of “mathematical concepts”. He writes (in his Schrodinger’s Kittens and the Search for Reality) that “a strong operational axiom” tells us that
“literally every version of mathematical concepts has a physical model somewhere, and the clever physicist should be advised to deliberately and routinely seek out, as part of his activity, physical models of already discovered mathematical structures”.
Yet even in Gribbin’s case, it’s still clear that a “mathematical concept” comes first and only then is a “physical model” found to square with it.
Penrose’s words on his own twistor theory are also very relevant here in that after criticising string theorists for seemingly divorcing their mathematics from experiment, prediction, observation, etc., he then freely confesses that he’s — at least partly - guilty of exactly the same sin.
Firstly, Penrose tells us about the pure mathematics of twistor theory. He writes:
“Yet twistor theory, like string theory, has had a significant influence on pure mathematics, and this has been regarded as one of its greatest strengths.”
Penrose then cites a couple of very-specific examples:
“Twistor theory has had an important impact on the theory of integrable systems  on representation theory, and on differential geometry.”
And then we have the mathematical aesthetics of twistor theory:
“Twistor theory has been greatly guided by considerations of mathematical elegance and interest, and its gains much of its strength from its rigorous and fruitful mathematical structure.”
Finally, the confession:
“That is all very well, the candid reader might be inclined to remark with some justification, but did I not complain  that a weakness of string theory was that it was largely mathematically driven, with too little guidance coming from the nature of the physical world? In some respects this is a valid criticism of twistor theory also. There is certainly no hard reason, coming from modern observational data, to force us into a belief that twistor theory provides the route that modern physics should follow… The main criticism that can be levelled at twistor theory, as of now, is that it is not really a physical theory. It certainly makes no unambiguous physical predictions.”
So how does Penrose extract himself from this problem? Well, to be honest, he doesn’t go into great detail — at least not after these specific passages.
The obvious question to ask now is this:
What is twistor theory doing right that string theory is doing wrong?
Is the answer to that question entirely determined by how close each theory is to “the nature of the physical world”? But don’t we (as it were) get to the physical world only through theory? As Stephen Hawking once put it:
“If what we regards as real depends on our theory, how can we make reality the basis of our philosophy? But we cannot distinguish what is real about the universe without a theory… Beyond that it makes no sense to ask if it corresponds to reality, because we do not know what reality is independent of theory.”
In any case, perhaps it’s the case that (as Penrose may believe) the mathematics of twistor theory is superior to the mathematics of string theory.
String theory particularly has been criticised for not making “unambiguous physical predictions”. Yet here’s Penrose saying exactly the same thing about his own twistor theory.
Finally, there probably never is (to use Penrose’s own words) “a hard reason” to “force” us to believe any physical theory — at least not in the early days of such theories. This obliquely brings on board the largely philosophical idea of the underdetermination of theory by data in that the “modern observational data” which Penrose mentions will never be enough to force the issue of which theory to accept. In other words, whatever observational data there is can be interpreted (or theorised about) in many ways. Alternatively, the same observational data can produce — or be explained by — numerous (often rival) physical theories.