Douglas Hofstadter is a physicist turned cognitive scientist who wrote the Pulitzer Prize-winning book Gödel, Escher, Bach: An Eternal Golden Braid. As a student in the 1970s, he discovered a fractal pattern describing the energy spectrum of electrons in atomic lattices in magnetic fields; initially scorned by his advisor, this “Hofstadter butterfly” has very recently been experimentally verified.

In late 1974, as a graduate student at the University of Oregon in Eugene, I accompanied my PhD advisor, the distinguished solid-state physicist Gregory Wannier, on a six-month visit to Regensburg, Germany.

In the early 1960s, I had discovered a family of graphs having a novel and beautiful kind of behavior while researching number-theoretical questions as an undergraduate math major. The main graph, which I dubbed “INT,” was continuous at every irrational number but had a jump discontinuity at every rational.

This was very weird. Seen from far away, INT looked like the 45 degree line y = x. But if you looked at it more closely, you would see it actually consisted of an infinite number of backslashes. If you looked in greater detail at each backslash, you would see it in turn break up into an infinite number of forward slashes. And so on.

In short, INT consisted of an infinite number of copies of itself (each of which consisted of yet smaller copies, ad infinitum). Today such wild shapes are known as “fractals” and are familiar even to people who have never studied mathematics, but back then this was a totally unfamiliar and wonderfully counterintuitive phenomenon. Thanks to my exploration of the INT family in the early 1960s, such “nested” behavior became part of my most intimate mental makeup.

Fast-forward to the mid-1970s, when I was a physics grad student working on the unsolved problem of the energy spectrum of electrons in a crystal in a uniform magnetic field. Such a situation is characterized by the number of “flux quanta,” hc/2πe, passing through each cell of the crystal lattice.

I had been lured to study this problem by Wannier’s strange statement that the problem could be attacked for rational values of this dimensionless quantity but was in principle unsolvable for irrational values. This claim tantalized me, first because I was intimately familiar with number-theoretical functions that acted differently at rationals and irrationals, and second because it struck me as impossible that any physical phenomenon could exhibit such weird behavior. (What would you say to someone who asked you if you prefer keeping your house at rational or irrational temperatures?)

I studied the problem very diligently, soon discovering, to my chagrin, that I was not skilled enough to do the virtuosic kinds of equation manipulations that Wannier and his Regensburg colleagues could do. Out of desperation, I decided to use a small desktop computer to explore the spectrum’s numerical behavior. This was not a typical approach for a theoretical physicist back then—the ones I knew never used computers and weren’t interested in them.

When I started plotting (by hand, using colored pens) the energy bands whose coordinates the small, slow computer calculated in overnight sessions, I saw, to my astonishment, old familiar patterns of nesting. Even when the graph was plotted very crudely, I could see that it had to be made out of an infinite number of smaller-sized copies of itself. Thanks to my discovery of INT many years earlier, I had a “prepared mind.”

The analogy between INT and Gplot (as I called my graph, later dubbed the “Hofstadter butterfly” by other people) was very exciting to me, but when I explained my findings to Wannier, he was bewildered and utterly skeptical. He had never seen anything like INT. The few dozen bands making up the hand-plotted graph I showed him didn’t seem to him to exhibit any regularity, so he scornfully called the nesting pattern that I claimed to see “mere numerology.” He told me I would be unable to get a PhD for such work; instead, I would have to write a “library thesis” summarizing other people’s research.

Though deeply hurt by Wannier’s harsh words, I was sure my ideas were correct, so I persevered in private. After a few months, I had produced some very detailed graphs of Gplot, including graphs of tiny regions of it, which looked just like the whole thing. Some months after we’d returned to Eugene, I at last showed Wannier these detailed images, and he carefully scrutinized them. All at once, the scales fell from his eyes and he realized that my “numerology” had been right, after all. Eventually he fell deeply in love with the unprecedented self-similar structure I’d discovered (no such thing had cropped up in physics before), becoming a champion of my ideas. He totally forgot that he had voiced any skepticism about what I’d told him in Regensburg. That was most ironic, of course, but I never pointed out the irony to him.

As I look back on this crucial episode of my life, I see that it typifies situations where one finds oneself pitted against an expert, with the two people holding diametrically opposite opinions. Is there any general recipe for what to do in such a situation? Well, one general approach would be always to kowtow to the expert, to always cave in and convert to the expert’s beliefs. I obviously didn’t do that.

Another general recipe would be to always insist on one’s own ideas, no matter what other people say, no matter how famous or accomplished they might be. This is not a general strategy that I would advise either, even though it’s exactly what I did in my story—but in that case, I had very good reasons to do so.

The subtle judgment call that has to be made in such situations of conflict depends on many ineffable factors, such as whether one has ever been in analogous situations before (and I had been, thanks to my deep explorations of INT many years earlier), and the degree to which the expert is really an expert in this domain (Wannier, although undeniably a world expert in solid-state physics, had never studied number theory, which turned out to be crucial to this problem). That Wannier had resorted to terms of scorn to describe my claims revealed that he was mystified by my findings and was in some sense putting up a smokescreen of resistance to them.

My “words of wisdom” thus cannot be put into simple terms, because it would be foolish always to believe in oneself over experts, and yet sometimes one has to do just that. And so, when? When some prior experience from one’s past is so profoundly analogous to the current situation that one knows in one’s bones that one is onto something deep. That’s when one must cling to one’s beliefs and not let oneself be bullied into giving up. //

Read More

To learn more about Hofstadter’s butterfly check out www.nature.com/news/physicists-net-fractal-butterfly-1.13717.