AI reinvents the way we invent

The problem is that human researchers can only investigate a small part of what is possible. It is estimated that there are as many as 1060 potentially drug-like molecules – more than the number of atoms in the solar system. But going through seemingly unlimited possibilities is what machine learning is good at. Trained in large databases of existing molecules and their properties, the programs can explore all possible related molecules.

The discovery of medicines is a very expensive and often frustrating process. Medical chemists have to guess which compounds good medicines can make, using their knowledge of how the structure of a molecule influences its properties. They synthesize and test numerous variants, and most are errors. “Coming up with new molecules is still an art because you have so many options,” says Barzilay. “It takes a long time to find good candidate drugs.”

Sign up for The algorithm – artificial intelligence, demystified

By speeding up this critical step, deep learning could offer chemists far more opportunities to pursue, allowing drugs to be discovered much faster. An advantage: machine learning is often idiosyncratic imagination. “Maybe it is going in a different direction than a human being would not go,” says Angel Guzman-Perez, a researcher at Amgen who works with Barzilay. “It thinks differently.”

Others use machine learning to invent new materials for clean-tech applications. Among the items on the wish list are improved batteries for storing electricity on the electricity grid and organic solar cells, which can be much cheaper than the current silicon-based large ones.

Such breakthroughs have become more difficult and expensive because chemistry, materials science and drug research have become staggeringly complex and saturated with data. Although the pharmaceutical and biotech industries are investing money in research, the number of new drugs based on new molecules has remained flat in recent decades. And we are still stuck with lithium-ion batteries from the early 1990s and designs for silicon solar cells that are also decades old.

The complexity that has slowed progress in these areas is where deep learning excels. Searching in multidimensional space to make valuable predictions is “AI’s sweet spot,” says Ajay Agrawal, an economist at the Rotman School of Management in Toronto and author of the best-selling prediction machines: the simple economy of artificial intelligence.

In a recent article, economists at MIT, Harvard and Boston University argued that the greatest economic impact of AI could stem from its potential as a new “method of invention” that ultimately “reforms the nature of the innovation process and the organization of R&D.” “

Iain Cockburn, a BU economist and co-author of the newspaper, says: “New inventive methods with broad applications are rare, and if our guess is right, AI could dramatically change R&D costs in many different areas. A lot of innovation includes predictions based on data. In such tasks, Cockburn adds, “machine learning can be much faster and cheaper in size.”

In other words, AI’s most important legacy is perhaps not driving without cars or searching for images or even the ability of Alexa to take orders, but the ability to come up with new ideas to nurture innovation itself.

Ideas become expensive

At the end of last year, Paul Romer won the Nobel Prize in Economics for work done in the late 1980s and early 1990s, showing that investing in new ideas and innovation leads to strong economic growth. Previous economists had noticed the link between innovation and growth, but Romer gave an excellent explanation of how it works. In the decades that followed, Romer’s conclusions have been the intellectual inspiration for many in Silicon Valley and help explain how it has achieved such wealth.

But what if our pipeline dries up with new ideas? Economists Nicholas Bloom and Chad Jones at Stanford, Michael Webb, a graduate student at the university, and John Van Reenen at MIT looked at the problem in a recent paper entitled “Are ideas harder to find?” (Their answer was “Yes.”) Looking at drug discovery, semiconductor research, medical innovation, and efforts to improve crop yields, economists found a common story: investment in research is rising sharply, but payments are constant.

From the perspective of an economist, that is a productivity problem: we pay more for a comparable amount of output. And the numbers look bad. Research productivity – the number of researchers needed to produce a particular result – decreases annually by around 6.8% for the task of expanding Moore’s law, which requires us to find ways to get more and smaller components on package a semiconductor chip to keep making computers faster and more powerful. (They found it more than 18 times as many researchers to double the chip density today as they did in the early 1970s.) To improve seeds, measured by crop yields, research productivity drops by around 5% per year. For the US economy as a whole, it is falling by 5.3%.

It takes more researchers and money to find productive new ideas, according to economists at Stanford and MIT. That is a likely factor in the overall slow growth in the US and Europe in recent decades. The graph below shows the pattern for the total economy, with an emphasis on the total productivity of the US (per decade average and for 2000-2014) – a measure of the contribution of innovation – versus the number of researchers. Similar patterns apply to specific areas of research.

source: bloom, jones, van reenen and webb

Every negative effect of this decrease has so far been compensated by the fact that we are putting more money and people into research. So we still double the number of transistors on a chip every two years, but only because we are dedicating many more people to the problem. We will have to double our investment in research and development over the next 13 years to keep entering water.

It may of course be that fields such as crop science and semiconductor research are getting old and the chances for innovation are increasing. However, the researchers also discovered that overall growth coupled with innovation in the economy was slow. Investments in new areas and inventions that they have generated have not changed the overall story.

The decline in research productivity appears to be a trend of decades. But it is particularly worrying for economists since we have seen a general slowdown in economic growth since the mid-2000s. In a time of brilliant new technologies such as smartphones, driverless cars and Facebook, growth is slow and the share attributed to innovation – called total factor productivity – is particularly weak.

The continuing effects of the 2008 financial collapse can hamper growth, says Van Reenen, as well as the continuing political uncertainties. But bleak research productivity certainly makes a contribution. And he says that if the decline continues, this can seriously damage future prosperity and growth.

It is logical that we have already chosen a lot of what some economists call ‘low-hanging fruit’ in terms of inventions. Could it be that the only remaining fruit is a few withered apples on the furthest branches of the tree? Robert Gordon, an economist at Northwestern University, is a strong proponent of that vision. He says it is unlikely that we will match the flourishing of the discovery of the late 19th and early 20th centuries, when inventions such as electric light and power and the combustion engine led to a century of unprecedented prosperity.

If Gordon is right and there are fewer major inventions, we are doomed to a bleak economic future. But few economists think this is the case. It is rather logical that there are great new ideas; it only becomes more expensive to find them as science becomes more and more complex. The chance that the next penicillin will simply fall into our laps is small. We will need more and more researchers to gain insight into advancing science in areas such as chemistry and biology.

It is what Ben Jones, a Northwestern economist, calls “the burden of knowledge.” Researchers are becoming more specialized, which makes it necessary to form larger – and more expensive – teams to solve problems. Research by Jones shows that the age at which scientists reach their top productivity is rising: it takes longer for them to have the expertise they need. “It’s an innate by-product of the exponential growth of knowledge,” he says.

“Many people tell me that our findings are depressing, but I don’t see it that way,” says Van Reenen. Innovation may be more difficult and more expensive, but in his view that simply points to the need for policy measures, including tax incentives, that will encourage investment in more research.

“As long as you invest resources in R&D, you can maintain healthy productivity growth,” says Van Reenen. “But we have to be prepared to spend money on that. It doesn’t come for free. “

Giving up science

Can AI creatively solve the kind of problems that such innovation requires? Some experts are now convinced that this is possible, given the kind of progress that is being shown by the AlphaGo game machine.

AlphaGo controlled the old game of Go and defeated the reigning champion by studying the almost unlimited possible movements in a game that has been played for thousands of years by people who are highly dependent on intuition. In addition, it sometimes came with winning strategies that no human player had thought of trying. Likewise, the in-depth learning programs that are trained on large amounts of experimental data and chemical literature may think that new substances that scientists had never imagined could come up with.

Could an AlphaGo-like breakthrough help the growing armies of researchers to use ever-increasing scientific data? Can AI make fundamental research faster and more productive, thereby restoring areas that have become too expensive for companies to revive?

In recent decades, our R&D efforts have undergone an enormous revolution. Since the days that Bell Labs from AT&T and PARC from Xerox produced world-changing inventions, such as the transistor, solar cells and laser printers, most major companies in the US and other rich economies have abandoned fundamental research. In the meantime, US federal R&D investments have remained flat, particularly in areas other than life sciences. So while we continue to increase the number of researchers in general and turn incremental progress into commercial opportunities, areas that require long-term research and a basic science base have hit.

The invention of new materials in particular has become a commercial inland waterway. That has stopped the necessary innovations in clean technology – things like better batteries, more efficient solar cells and catalysts to make fuels directly from sunlight and carbon dioxide (think of artificial photosynthesis). Although the prices of solar panels and batteries are falling steadily, this is largely due to improvements in production and economies of scale, rather than fundamental advances in the technologies themselves.

Could an AlphaGo-like breakthrough help the growing armies of researchers to use ever-increasing scientific data?

It takes an average of 15 to 20 years to invent a new material, says Tonio Buonassisi, a mechanical engineer at MIT who works with a team of scientists in Singapore to speed up the process. That is far too long for most companies. It is impractical even for many academic groups. Who wants to spend years on a material that may or may not work? This is why venture-backed startups, who have generated much of the innovation in software and even biotech, have long since relinquished clean technology: venture capitalists generally need returns within seven years or earlier.

“A 10x acceleration (in the speed of material discovery) is not only possible, it is also necessary,” says Buonassisi, who runs a photovoltaic research laboratory at MIT. His goal, and that of a loosely connected network of fellow scientists, is to use AI and machine learning to reduce that time frame from 15 to 20 years to around two to five years by attacking the various bottlenecks in the laboratory and automate as much of the process as possible. A faster process gives scientists much more possible solutions to test, enables them to find deadlines in hours instead of months and helps optimize materials. “It transforms how we think as researchers,” he says.

It could also turn the discovery of materials into a viable business activity. Buonassisi refers to a graph with the time it took to develop different technologies. One of the columns with the label “lithium-ion batteries” shows 20 years.

Another, much shorter column is labeled as “new solar cell”; at the top is “2030 climate target”. The point is clear: we cannot wait another 20 years for the next breakthrough in clean-tech materials.

1
Atomwise
2
Kebotix
3
Deep Genomics
What they do
Use neural networks to search through large databases to find small drug-like molecules that bind to targeted proteins.

Develop a combination of robotics and AI to accelerate the discovery and development of new materials and chemicals.

Use artificial intelligence to search for oligonucleotide molecules to treat genetic diseases.

Why it matters
Identifying such molecules with desirable properties, such as potency, is a critical first step in drug discovery.

It takes more than ten years to develop a material. By shortening that time, we can tackle issues such as climate change.

Oligonucleotide treatments promise many different diseases, including neurodegenerative and metabolic disorders.

The AI-driven lab

“Come to a free country”: Alán Aspuru-Guzik is now inviting an American visitor to his laboratory in Toronto. In 2018, Aspuru-Guzik left his permanent position as a professor of chemistry at Harvard and moved to Canada with his family. His decision was driven by a strong aversion to President Donald Trump and his policies, particularly in the area of ​​immigration. However, it did not hurt that Toronto quickly became a mecca for artificial intelligence research.

Aspuru-Guzik is not only a professor of chemistry at the University of Toronto, but also has a position at the Vector Institute for Artificial Intelligence. It is the AI ​​center co-founded by Geoffrey Hinton, whose groundbreaking work in the field of deep learning and neural networks is largely credited with today’s leap in AI.

In a remarkable newspaper from 2012, Hinton and his co-authors demonstrated that a deep neural network, trained on a large number of photos, could identify a mushroom, a leopard and a Dalmatian dog. It was a remarkable breakthrough at the time and it quickly led to an AI revolution using in-depth algorithms to understand large data sets.

Researchers quickly found ways to use such neural networks to help driverless cars navigate and recognize faces in a crowd. Others changed the in-depth teaching resources so that they could train themselves; these tools include GANs (generative opponent networks) that can take pictures of scenes and people who have never existed.

In a follow-up of 2015, Hinton provided indications that deep learning could be used in chemistry and material research. His article praised the ability of a neural network to discover “complex structures in high-dimensional data” – in other words, the same networks that can navigate millions of images to find, for example, a dog with spots, can sort millions of molecules to identify one with certain desired properties.

Aspuru-Guzik is energetic and full of ideas and is not the type of scientist who spends two decades patiently figuring out whether a material will work. And he has quickly adjusted in-depth learning and neural networks to try to reinvent materials. The idea is to add artificial intelligence and automation to all steps of material research: the initial design and synthesis of a material, testing and analysis, and finally the multiple refinements that optimize performance.

On an ice-cold day at the beginning of January, Aspuru-Guzik has his hat pulled tightly around his ears, but otherwise seems unaware of the bitter Canadian weather. He has other things on his mind. First, he is still waiting for the delivery of a $ 1.2 million robot, now on a ship from Switzerland, which will be the center of the automated AI-driven laboratory he envisaged.

In the laboratory, deep-learning tools such as GANs and their cousin, a technique called autoencoder, will present promising new materials and figure out how to make them. The robot will then make the connections; Aspuru-Guzik wants to create an affordable automated system that can spit out new molecules upon request. After the materials are made, they can be analyzed with instruments such as a mass spectrometer. Additional machine learning tools will understand that data and “diagnose” the properties of the material. These insights will then be used to further optimize the materials and adjust their structures. And then Aspuru-Guzik says: “AI will select the next experiment to close the loop.”

The idea is to add artificial intelligence and automation to all steps of material research and drug discovery.

Once the robot is installed, Aspuru-Guzik expects to produce around 48 new materials every two days, based on insights into machine learning to keep improving their structures. That is a promising new material every hour, an unprecedented pace that could completely change the productivity of the lab.

It’s not just about coming up with “a magic material,” he says. To really change material research, you have to attack the entire process: “What are the bottlenecks? You want AI in every piece of the lab. “Once you have a proposed structure, you still have to figure out how to make it. It can take weeks to months to resolve what chemists call “retrosynthesis” – working backwards from a molecular structure to determine the steps needed to synthesize such a compound. Another bottleneck is understanding the amount of data produced by analytical equipment. Machine learning can speed up any of these steps.

What motivates Aspuru-Guzik is the threat of climate change, the need for improvements in clean technology and the essential role of materials in producing such progress. His own research looks at new organic electrolytes for power batteries, which can be used to store and pump back excess electricity from power networks when needed, and at organic solar cells that would be much cheaper than silicon-based cells. But if his design works for a self-contained, automated chemical lab, he argues, it could make chemistry more accessible to almost everyone. He calls it the “democratization of material discovery.”

“This is where the action is,” he says. “AIs that drive cars, AIs that improve medical diagnostics, AIs for personal shopping – the economic growth of AIs applied to scientific research can combine the economic impact of all those other AIs.”

The Vector Institute, the Toronto magnet for AI research, is less than 1.5 km away. From the large open office space you look out over the Ontario Parliament building. The proximity of experts in AI, chemistry and business to the government seat in downtown Toronto is no coincidence. There is a strong belief among many in the city that AI will transform business and the economy, and some are increasingly convinced that it will radically change the way we do science.

But if it does, a first step is to convince scientists that it’s worth it.

Amgens Guzman-Perez says that many of his colleagues in medicinal chemistry are skeptical. Over the past decades, the field has seen a series of so-called revolutionary technologies, from computer design to combinatorial chemistry and high-throughput screening, which have automated the rapid production and testing of multiple molecules. Each has proved somewhat useful but limited. None, he says, “you get a new medicine magically.”

It’s too early to know for sure if deep learning can ultimately be the game changer, he acknowledges, “and it’s hard to know the time frame.” But he encourages the speed with which AI has transformed image recognition and other search tasks.

Sign up for The algorithm – artificial intelligence, demystified

“Hopefully it can happen in chemistry,” he says.

We are still waiting for the AlphaGo moment in chemistry and materials – for in-depth algorithms that outsmart the most skilled person to come up with a new drug or material. But just as AlphaGo won with a combination of creepy strategy and an inhuman imagination, today’s latest AI programs were soon able to prove themselves in the lab.

And that has some scientists who dream big. The idea, says Aspuru-Guzik, is to use AI and automation to reinvent the lab with tools such as the $ 30,000 molecular printer he hopes to build. It is then up to the imagination of scientists – and that of AI – to explore the possibilities.