In the spring of 2001, Alph Bingham was running R&D strategy at Eli Lilly, one of the world’s foremost pharmaceutical companies.
On his desk sat twenty-one scientific problems that had stumped some of the most qualified chemists on the planet. These were PhDs in hyper-specialized domains. People who had spent decades studying molecules most of us would struggle to pronounce.
They were all stuck.
Bingham himself was no lightweight. He had a PhD in organic chemistry. “If there’s not a carbon in it, I’m technically not qualified,” he would joke about his capabilities.
But even though he understood and appreciated specialization, he had noticed something peculiar in grad school: the cleverest solutions rarely came from within the curriculum. They came from odd bits of outside knowledge. Once, he had solved a molecular synthesis problem using an idea related to cream of tartar, a baking ingredient he happened to know from childhood. It wasn’t in any textbook.
Drawing inspiration from his own past experience, he wondered, what if the solution doesn’t lie inside the building? What if they could somehow crowdsource the solutions? The world was big, and it had enough smart people.
He persuaded the company to post some of these unsolved problems online, visible to anyone willing to attempt them. It was met with resistance initially. Why would anyone outside, without deep training in that exact subfield, be able to solve what their best scientists could not? And why on earth would they advertise their incompetence in public?
But the problems had been sitting for too long. They were desperate for answers. The site went live, but it wasn’t anything more than a shot in the dark at the time.
A few days later, solutions began to arrive.
And one by one, they blew everyone’s minds.
One molecular problem was cracked by a lawyer who had been working on chemical patents. Another came from a scientist in an entirely different discipline who saw parallels no insider had noticed. Roughly a third of the challenges posted were eventually solved. Many by people with no formal background in the narrow domain of the problem.
For those in-house experts, it appeared that their expertise had confined them to what Bingham later called “local search.” They looked for answers only where they had looked before. They followed the processes they had been taught to be right. But outsiders had no such baggage. They were free to roam in search of solutions.
Bingham spun this idea into a company called InnoCentive. Organizations could post “challenges” and offer cash rewards to anyone in the world who solved them. The solver community grew to include chemists, engineers, physicists, dentists, programmers, retirees, hobbyists — people who, on paper, had no business solving certain problems.
NASA tried it too.
For nearly thirty years, NASA had struggled with predicting solar particle radiation — bursts from the sun that could endanger astronauts. It was a notoriously difficult astrophysics problem. Through InnoCentive, the solution came from a retired radio engineer with no formal background in heliophysics. He adapted signal-processing techniques from another field and outperformed existing models.
What had left NASA scientists puzzled for three decades was solved by someone who did not belong to the field.
InnoCentive is a fascinating case study in the specialist v/s generalist debate. In a Harvard survey of InnoCentive solvers, it was found that the further the problem was from the solver’s expertise, the more likely they were to solve it.
So the hardest problems that stumped the best minds of a particular field needed not the yet more accomplished expert from within the domain, but smart people from outside who could reframe the problem and approach it differently. It was their ability to connect the dots better and not deep specialist knowledge that led to solutions.
This challenges the conventional belief that hard problems can only be solved by people with deep functional expertise in that field. At the same time, we can't ignore the benefits of specialization, especially in domains that require deep research. After all, most progress in different fields has been driven by people who dedicated their lives to a narrow domain.
~
This brings me to the question: As AI becomes increasingly prevalent, who has a better chance at prosperity — specialists or generalists?
To answer that, let’s first understand what AI has already done.
Until recently, if you knew the syntax of a useful programming language well enough, you’d be in reasonable demand. You might not have earned the big bucks like the other prolific techies, but syntactical knowledge alone would put food on the table.
The same was true for accountants. If you knew how to create the balance sheet and P&L statement from basic business information, you’d probably have a seat somewhere.
Even ‘content writers’ whose job was to string words together to create an average blog was an actual role.
Now, all such roles are already extinct. AI gives a far better output than any beginner with bare minimum skills at not just these jobs, but dozens of others. Even experienced folks are hanging by a thread. Many of them have already bitten the dust. If AI can provide a similar or even a slightly inferior output at 10x the speed and 1/10th the cost, rational companies will eventually retain just their top 10% and have them produce more output alongside AI.
We don’t know what happens later, but today, you have a better shot at being in your job if you clearly know what an excellent outcome looks like and exactly the ingredients that can produce it.
For example, a coder today needs to understand what the final product intends to do for the consumer, and work backwards to create it most efficiently. Knowledge of programming will be necessary, but it won’t be of much use without an understanding of consumer behaviour, pricing, competitive landscape, and other market dynamics. The one who can see the larger picture clearly and connect all the necessary dots is likely to create the best final product. This is becoming true across industries and functions.
Effectively, this points to the good old T-shaped knowledge base. Specialize deeply in one, and have enough knowledge in multiple domains. But I’d add one more layer to it.
More than just the ‘T’ shape, it’s the area under that T that matters more today. This means, if you’re a specialist, you must generalize more. And if you’re a generalist, you must obtain deeper knowledge of more domains. That’s what builds the muscle to connect dots powerfully, and makes you a better problem solver. It’s also evident from the earlier examples I stated.
The lawyer who solved a chemistry problem was not a generalist in the casual sense. He was specialized — in chemical patents. The radio engineer who helped NASA was not broadly knowledgeable about everything. He had decades of depth in signal processing.
Their advantage was not lack of specialization, but the portability of specialization. They had depth that could travel across domains.
The in-house experts at Eli Lily had depth that was locked inside one domain.
AI is creating the same distinction inside organizations today. The people who thrive will not be those who merely execute tasks within a silo. Nor those who skim across many subjects without mastery. It will be those whose knowledge can travel, whose depth in one area helps them reframe problems in another.
If you are a domain-locked specialist, AI competes with you. If you are a shallow generalist, AI outperforms you.
But if you are a deep thinker who can reframe problems across domains, AI amplifies you. That is the real “area under the T.”
A question for you
Who do you think has a better chance of success in this age — specialists or generalists? And why?
Do comment what you think. I’d love to see some arguments challenging what I’ve stated here :)
Subscriber Spotlight
Last week, on our newsletter titled Love In The Age Of Infinite Choices, Apoorva had this to say:
“The herd of choices available, fear of missing out on a potentially better partner is coming in way of establishing a real connection. Love is tough, takes time and warrants a good fight, one often misconstrued as a liability. Looking forward to next Monday for another insightful read :)”
What we’re reading
Range by David Epstein. I first read it exactly six years ago, and it challenged so many of my beliefs about the specialization v/s generalization debate. I picked it up recently again to read it in the current context. I must say, it feels as relevant as it did in 2020. One of the most insightful books I’ve read.
Until next time.
Best,
