💥State of AI Report 2020: Highlight commentary enclosed!
Welcome back to regular readers and hello to everyone who joined since last month!
This edition of Your Guide to AI brings you the State of AI Report 2020 (read here). For the third year running, my co-author Ian Hogarth and I analyse the most interesting developments in AI over the last 12 months. By publishing this work openly on the internet, we aim to engage with all of you and trigger an informed conversation about the state of AI and its implication for the future.
The report is circa 170 slides, so I will highlight key points for you below.
If you'd like to discuss the report further, Ian and I will be running an Ask Me Anything session over Zoom next week. You can register here. As usual, you can always reply to this email to find me on the other side 🕵️
Help spread the word: If you enjoy this issue, consider forwarding it to a couple of friends. There's something for everyone 😉
AI Research is less open than you think
Open sourcing AI research is important for accountability, reproducibility and driving progress in AI but, in reality, the vast majority of AI research papers are behind corporate lock and key. The industry has not evolved much since 2016 when 90% of research did not publish code along with their papers. We hope to see positive change in this regard.
Elsewhere in the report, we show that NeurIPS and ICLR both propose new ethical principles and expectations of researchers, but no mandatory code and data sharing. In this aspect, machine learning is behind leaders in life sciences, such as the Wellcome Trust or Nature. Example: “A condition of publication in a Nature Research journal is that authors are required to make materials, data, code, and associated protocols promptly available to readers without undue qualifications.”
--> What do you think? Hit reply!
Welcome to the Billion Parameter club
Only a few labs that are led or funded by the likes of Google, Microsoft, and Facebook can afford the high cost of computing power needed to pursue the most exciting AI research. It most likely cost Microsoft-funded OpenAI $10 million to train GPT-3. Practitioners fear innovation could stagnate...
Smaller, task-specific models can compete against BERT
There is huge amount of value-add innovation driven by startups that operate around the periphery of the compute-intensive R&D work conducted by big tech and others who believe in the scaling hypothesis (more progress requires more compute). How, you ask? Startups trade research for engineering by focusing on adapting novel architecture ideas with task or domain-specific insights to yield robust solutions that work in the real world. Here, we showcase work from London-based startup, PolyAI, which was recently party to the first interaction between two AI call center agents in the wild: Google Duplex vs. PolyAI. You can listen to the conversation here.
Going beyond causation: Spotlight on causal inference
Most ML applications utilise statistical techniques to explore correlations between variables. This requires that experimental conditions remain the same and that the trained ML system is applied on the same kind of data as the training data. It ignores a major component of how humans learn - by reasoning about cause and effect. To fix this problem, researchers are working on causal reasoning. We showcase work from Babylon Health and UCL that applies causal reasoning to medical diagnosis.
Biology is experiencing its "AI moment"
Astute readers will have noticed my emphasis on biology as the new frontier for AI in the last years. I have previously shared a number of exciting problem areas where AI is making an impact and a rationale for why AI-first biology will be big. In the slides below, we dig into more evidence:
Deep learning finds antibiotics
AI-first drug discovery startups raising mega rounds
In spite of pharma doubts, AI-powered medical companies started rapidly industrialising and scaling. Major companies have raised mega-rounds, generated asset-focused spinoffs and closed big asset licensing deals.
US health system reimburses a deep learning diagnostic
The first-ever AI-created drug in clinical trials and the first medical imaging product to be acknowledged by the US government (Medicare/Medicaid). This is a big deal because it creates the needed financial incentive for physicians to prescribe new treatments and procedures.
The Great Brain Drain
Not much of a surprise to most of you who are regulars in the field, but the numbers are still quite shocking. Most of the companies who are actively recruiting academic talent try to give back to academia by endowing professorships or sponsoring PhD programmes. However, more can be done to replenish the ranks in universities.
The American AI industry is sustained by foreign talent
Given how dependent America’s AI industry is on immigrants there has been a strong backlash to Trump’s proclamation to suspend H1-B visas. Eight federal lawsuits and hundreds of universities objected. This is important because 92% of top international US AI PhD graduates work in the US post-graduation and 80% intend to stay if they can.
US military AI
AI-powered military is exploding and US-based startups raise large rounds (e.g. Anduril, Rebellion, Skydio) to support the US military industrial complex. As military applications grow, so do calls for new AI safety measures. With another wave of countries declaring national AI strategies, more governments doubled down on the military adoption of AI technology.
From AlphaGo and AlphaStar to AlphaDogfight
The dual-use nature of AI technology and the line of sight from published research to applications on the battlefield is clear. In the context of the AlphaDogfight competition that pitted RL agents against human-controlled airplanes in a virtual dogfight, Defense Secretary Dr. Mark T. Esper stated "The AI agent's resounding victory demonstrated the ability of advanced algorithms to outperform humans in virtual dogfights...These simulations will culminate in a real-world competition involving full-scale tactical aircraft in 2024." He referenced the “tectonic impact of machine learning on the future of warfare, referenced China a competitor and stated: "History informs us that those who are first to harness once-in-a-generation technologies often have a decisive advantage on the battlefield for years to come".
TSMC as the linchpin in the geopolitical tussle over chips
Semiconductors amplify the geopolitical significance of Taiwan and particularly TSMC, which is the world-leading manufacturing company that many of the tech giants depend on. Indeed, the US technology industry and TSMC are significantly co-dependent with 60% of TSMC sales coming from the US. Meanwhile, China is the world’s largest importer of semiconductors, totalling $200B/year. Because of this, the Chinese government set up an additional $29B state-backed fund reduce its dependency on American semiconductor technology. China's SMIC is also heavily poaching from TSMC to close the (wide) gap in semiconductor capabilities.
8 predictions for the next 12 months
Read this year's report to see how our 2019 prediction faired!
Media coverage of the State of AI Report 2020
Tech investors predict Nvidia's $40 billion Arm acquisition will be blocked — www.cnbc.com Nvidia's $40 billion acquisition of chip designer Arm will most likely be blocked, according to two technology investors in Britain.
High computational costs keep AI research unequal - Axios — www.axios.com Just a handful of big companies are able to afford top-flight AI research.
The state of AI in 2020: Democratization, industrialization, and the way to artificial general intelligence | ZDNet — www.zdnet.com From fit for purpose development to pie in the sky research, this is what AI looks like in 2020.
State of AI report: what's next for European tech? | Sifted — sifted.eu Opportunities in AI for European startups are plentiful, and those in the biology and defence sectors are already eyeing it up.
—
Signing off,
Nathan Benaich, 4 October 2020
Air Street Capital | Twitter | LinkedIn | RAAIS | London.AI
Air Street Capital is a venture capital firm that invests in AI-first technology and life science companies. We’re a team of experienced investors, engineering leaders, entrepreneurs and AI researchers from the World’s most innovative technology companies and research institutions.