Your guide to AI in Q1 2018, by nathan.ai
Reporting from 1st January 2018 through March 18th 2018
Hello from London ❄️⛄ - yet another snowy day! I’m Nathan Benaich. I’m excited to share my guide to AI in Q1 2018 with you. Here, I’ll synthesise a narrative analysing and linking important news, data, research and startup activity from the AI world. Grab your hot beverage of choice ☕ and enjoy the read! A few quick points before we start:
1. Lots of you took me up on my offer to brainstorm use cases and research avenues for intelligent systems. It’s been fun, so don’t hesitate to hit reply and drop me a line to chat :)
2. Our upcoming London.AI #12 on April 19th has taken shape: We’re excited to bring you an exclusive premier of Blue Vision Labs (collaborative AR, raised $14.5M Series A), Unbabel (machine translation, raised $23M Series B) and Scortex (manufacturing inspection). Register your interest to attend here.
3. The 4th Research and Applied AI Summit is just over 3 months away. We're featuring a full day of talks and conversations with research group leaders and entrepreneurs flying in from Stanford, DeepMind, Google, Cloudera, and many leading technology startups. We have 300 spots, so register your interest quick.
Referred by a friend? Sign up here. Help share by giving this it a tweet :)
🆕 Technology news, trends and opinions
🚗 Department of Driverless Cars
a. Large incumbents
The Uber vs. Waymo lawsuit finally drew to a close 12 months after Waymo originally filed their lawsuit claiming damages over $1B. The two companies agreed to a settlement in which Waymo will acquire a 0.34% equity position in Uber that is worth $245M at the $72B valuation. Uber must also sign an agreement to ensure that Waymo confidential information is not incorporated into Uber technology, which is otherwise given the greenlight to move forward in its development of self-driving technology. The original settlement offer was $500M, which Uber’s board rejected. To be honest, given the huge financing rounds being raised for self-driving car companies these days and the size of the prize for those companies who win local markets, it feels like paying off Waymo with equity to distance itself from future claims is actually a very good deal for Uber. It’s another sign that Dara is doing a top job at Uber’s CEO. He gave a strong impression at Goldman’s Internet Research conference in SF a few weeks ago, where he said that Uber’s ultimate mission to so give users A to B transport through whatever combination of mobility mediums possible.
Fun sidenote: Anthony Levandowski, the man at the center of the trail, had entered a plan for a self-driving offroad motorbike into the 2004 DARPA Challenge (see video in the article). IEEE pieced together the story, which shows a project quite ahead of its time. The bike used an AMD 64-bit CPU with 512 MB of RAM that was able to process only about one frame every 4 seconds! The main sensors were optical, with a range of 40 meters. While it did not manage to finish the qualification course, GhostRider earned the attention of Sebastian Thrun in 2006. His car, Stanley, won the DARPA challenge and he’d invited Levandowski to work on a mapping project called VueTool. It’s team members were then hired by Google, which ultimately led Levandowski and Thrun working on Google’s self driving car project. The rest is history :)
Street level mapping, as we know, is one of the major war fronts in self-driving. Most companies have built their self-driving stack in a way that requires high-dimensional (HD) maps of the roads on which their cars will drive. Because these maps are the core infrastructure for autonomy, serious contenders want to build and own this part of the stack to reduce third party dependency on a core competency. This piece runs through the current state of affairs in the HD mapping arena, involving Google, HERE, TomTom, Uber, DeepMap, Mapbox, Intel/Mobileye and a ton more. What’s interesting is that human drivers do not have HD maps of the worlds they navigate, but are still able to generalise, learn quickly and adapt to diverse environmental situations. The HD mapping game implies that we’re not making necessary technology advances to progress towards human-level scene understanding and reasoning.
Baidu released ApolloScape, an open dataset that includes RGB videos with millions of high-resolution image sequences for scene parsing. The company also dropped its lawsuit against JingChi, another self-driving car company formed by ex-Baidu employees, as the two companies team up on technology development via Baidu Apollo. Eyebrows raised...
Lyft and Tier 1 OEM Magna have signed a deal for the co-development of autonomous vehicle software and joint ownership over IP. This deal gives Magna an exclusive agreement to outfit Lyft’s fleet with autonomous driving equipment, while still allowing the Lyft’s open marketplace to grow. In addition, Magna will purchase $200M worth of equity in Lyft, which pushes their recent financing round to $1.7B. This gives Lyft access to automotive-grade equipment and relationships with the sector.
The California DMV are expecting to receive state approval on new rules that would let autonomous car companies take the human safety driver out of the driver's seat, and also apply for permits to take passengers.
General Motors have published their 2018 self-driving safety report, which is really more of a descriptive account of their build processes, technology components, vehicle design and safety procedures that it is a quantitative account of their safety results. The company also announced a $100M investment to upgrade two factories such that they are fit to manufacture production versions of the autonomous Chevy Bolt EV.
Apple has increased the size of its autonomous car fleet in California from 3 to 27. It’s still unclear what the company’s efforts are, however.
Uber is pushing forward with their trucking business, which is is already making $100M’s a year according to their CEO. The company published their economic modelling exploring various modalities for launching self-driving trucks. This is what the analysis showed: “In our baseline projections without self-driving trucks, the number of trucking jobs nationwide increased 766,000 by 2028. When we add self-driving trucks into the scenario above, truck driving jobs increase even more, with many long haul jobs shifting to local haul to support growing freight volume moving in and out of transfer hubs. Why? The deployment of self-driving trucks improves efficiency on long haul routes, lowering the overall cost of trucking and reducing the total cost of the goods being shipped. When goods are cheaper, consumers buy more of them. And when consumers buy more, more new goods need to be shipped than before, which drives truck freight volume up. In this scenario, when 1 million self-driving trucks are operating on highways, we would expect to see close to 1 million jobs shift from long haul to local haul, plus about 400,000 new truck driving jobs will be needed to keep up with the higher demand.”
Wired ran a long piece on Didi Chuxing, the immense Chinese ridesharing company with 400M registered customer who take 25M rides a day in more than 400 Chinese cities. This volume is almost 2x Uber and all global ridesharing apps combined. The company that famously beat Uber out of China is working with carmakers to redesign vehicles for sharing, as well as projects on electric and autonomous vehicle too.
b. Startups
Aurora have announced their first commercial partner, Volkswagen Group, with the goal of creating a new mobility-as-a-service solution that integrates Aurora full-stack technology into VW cars. A few weeks later, Aurora announced a whopping $90M Series A round led by Greylock and Index.
Voyage have launched a Level 4 autonomous transport service to 125,000 residents of a private residential area called The Villages. The area includes 750 miles of road, 100 dining options, 8 major grocery stores, 80 tennis courts, a regional hospital, a school and more. Residents will be able to summon door-to-door transport through Voyage. This go-to-market strategy is the absolute right one in my view - it lets you test a fully autonomous transport system in the real world without throwing cars in the deep end of city traffic. In doing so, Voyage will be able to measure all the benefits of self-driving in a more controlled environment. Mapping of the streets is conducted using Carmera technology.
Nuro came out of stealth with two big announcements: a) they’ve designed and built a delivery focused self-driving vehicle that kind of looks like a shopping basket on wheels, and b) they’ve raised $92M already to get there. The vehicle will navigate on roads instead of pavements (like Starship’s delivery robot). It’s quite an impressive product.
💪 The giants
Bloomberg ran a long profile on SoftBank’s Masayoshi Son, citing his vision of “a trillion connected devices, generating data that will be analyzed by artificial intelligence to supposedly make the world a better place.” Their investor presentations are really worth a flick through, especially to understand the high level connectedness of the assets they’re acquiring stakes in.
In this widely watched presentation about the future for WeChat, founder Allen Zhang said that they will still not interfere or curate service providers who reach users through the app. In an age where Chinese users conduct most of their lives through WeChat, I would have thought that automated curation would be the next step (much like it has been for products led by US technology companies).
Amazon opened their first Go-powered grocery and convenience store in Seattle in January, letting anyone with an Amazon account, the Amazon Go app to collect and purchase goods without needing to go through a checkout line. Customers scan a QR code at the entrance and are followed by cameras and weight sensors on the shelves to track what they “buy”. The company now plans to open up to 6 more stores in 2018, powered by technology that’s been under development for four years. Several startups have since followed suit, include Bodega.
🍪 Hardware
Broadcom’s hostile $142B takeover bid for Qualcomm, which could have been the largest tech deal ever, has been blocked by the Trump administration. It believes there is credible evidence that Broadcom, which is based in Singapore, might take action via Qualcomm that ‘threatens to impair the national security of the United States”. The Europeans worry that Broadcom will have access to data about German citizens given that Qualcomm’s subsidiary makes chips for their passports.
The story of Apple’s investments into in-house semiconductor design and production is a rich one. Jobs long believed that the company should own the technologies within their products instead of procuring them from third parties. The company has chip building and testing facilities in California and Israel, originally from its acquisition of P.A. Semi in 2008. Apple has also been hiring talent from Qualcomm, with whom it been engaged in legal battles over licensing fees. Apple also made news for their Neural Engine and custom GPUs for iPhone 8 and X.
🏥 Healthcare
Palliative care clinics are centers that care for patients for whom there are no further treatment options to cure their declining health state. The NYT ran a story about developing an AI model for predicting the time window during which patients would die. Quite a morose thought for sure, but the motivation was to optimally allocate very strained resources to deliver an overall improved quality of life for all patients. The model ascribed low probabilities of death for 95% of patients who ended up living longer than 12 months.
Two papers came out applying deep learning models to diagnostic images of the eye (optical coherence tomography or retinal fundus). The use cases were to 1) predict cardiovascular risk factors (e.g. age, gender, smoking status, systolic blood pressure) and 2) classify age-related macular degeneration and diabetic macular edema. There was already lots of evidence that non-deep learning methods (such as logistic regression with standard risk factors) can be used to infer cardiovascular risk from retinal images. Patients in the UK Biobank (those used in this study) are still largely healthy to begin with. It’ll be super interesting to see how these predictions pan out as patients age. The second study, however, faces a few issues. The authors use a model that analyses a 2D image, whereas an OCT scan is 3D (i.e. it’s built from multiple slices to create a volume). They in fact use slices taken from the same eye (i.e. the same volume scan) in the training and test set, which means the model is actually allowed to ‘cheat’ by overfitting the data. Both studies also use attention mechanisms to draw heatmaps on their images to claim that the neural networks focus on the right parts of the image to make their inferences. It is, however, great to see cross-disciplinary machine learning and life sciences studies published in journals read by biologists (rather than the standard arxiv), as its through this channel that more life scientists will become familiar with machine learning tools. The key question in all of this is whether the diagnostic task is clinically-validated such that we can be confident neural network models can indeed learning clinically-relevant representations of data instead of spurious overfitting of training data. Thanks to Pearse Keane for comments here!
Innovation in the West vs the East: In the West, we’re witnessing a rather large number of healthcare startups developing doctor-on-demand services, medical imaging technology, and health monitoring solutions via wearable devices. While I’m bullish on these opportunities, the regulatory hurdles and clinical trial processes to establish these companies as part of the mainstay are significant. On the one hand, it feels like the risk/reward discussion is skewed towards the system slowing down progress. On the other hand, careful evaluation and study of any new technology, especially if it concerns human health, is paramount. Reading about medical innovation in the East, however, gives me the sense that the West is seriously falling behind. In this piece about Tencent’s medical ecosystem in China, we’re given a window into the company’s WeChat Intelligent Healthcare product suite. Through WeChat, consumers can book appointments (in person or online), make payments, obtain medical insurance, access their medical data, and benefit from AI-driven diagnostics that improve turnaround time. All of this occurs within the Tencent network. The company is also heavily investing in a 3rd party ecosystem of medical technology startups, as well as launching its own clinic’s, Tencent Doctorwork. I’m curious to hear any of your opinions on the ground! Interestingly, Alibaba has recently bid $1.4B for iKang, a chain of 110 private healthcare clinics in China, which indicates another push into an Alibaba technology-driven healthcare network.
Reform, an independent, non-party think tank focused on public services and economic prosperity, produced a report on AI in the UK’s National Health Service. They suggest that “AI can help reduce the“AI could help address the health and wellbeing gap by predicting which individuals or groups of individuals are at risk of illness and allow the NHS to target treatment more effectively towards them….AI could help address the efficiency and funding gap by automating tasks, triaging patients to the most appropriate services and allowing them to self-care.” The report recommends the NHS to fully digitise its data in a machine-readable format, implement a clear AI software procurement process, require IT suppliers to build interoperability of their systems right from the start, and provide a list of secure, training datasets. Also mentioned is the need for a framework of AI explainability, which includes tests of bias and making data preprocessing procedures and training data available to the regulatory agency.
Applying ML to human genomes to discern the mechanisms of disease: This piece profiles Barbara Engelhardt’s work in developing new statistical tools that search for the expected biological patterns to map our the genome’s real but elusive “ground truth”. What’s interesting here is the how she uses unsupervised techniques as feature extractors, then rationalises which features make biological sense, then uses ‘classical’ latent factor models to glean interpretable mechanistic insights into biology. This strategy lends itself well to the fact that genomics datasets have huge amounts of noise and very little signal, in addition to enormous feature spaces from very few samples.
🔮 Where AI is heading
Rodney Brooks of MIT and Rethink Robotics fame shares his predictions about the next 32 years of AI progress, what to expect and when. He focuses on self-driving cars, AI research and robotics achievements, as well as space travel. Short summary of a great piece: “It always takes longer than you think. It just does.”
Google use video data from their Clips product to understand how a human-centered design process elevates AI. I like the emphasis they place on anchoring user expectations for AI-driven systems. In particular, they note: a) that ML won’t figure out what problems to solve and thus we need to align ML systems with human ned, b) the goals of an ML system should be clear to a user such that their expectations are calibrated, c) the field has to be multi-disciplinary.
Markus Wulfmeier of the Oxford Robotics Institute penned a tremendous review of the subsystems required to build autonomous platforms, as well as the state of learning based methods for these, including uncertainty and introspection. In particular, I like how he makes the point that it is “often more efficient simply in terms of human effort to port our prior knowledge into algorithmic prior structure, which will be the most practical way for short to medium term research with potential for real world impact.” Using both learned approaches and prior knowledge based on geometry, dynamics and kinematics is suggested as a healthy hybrid approach.
OpenAI published their second Requests for Research, in which they outline seven unsolved problems.
Why is machine learning (and thus progress towards general AI) hard to achieve? One reason is that machine learning is a fundamentally hard debugging problem. Compared to regular software engineering that requires awareness of the trade offs of competing frameworks, tools and techniques and judicious design decisions, machine learning adds two additional dimensions along which bugs are common: the model itself and the data.
Interpretability is an increasingly large deal in the ML world - in this piece, Google researchers We explore the powerful interfaces that arise when you combine them — and the rich structure of this combinatorial space.
📑 Careers (new section!)
TwentyBN, the video understanding company are hiring for engineering, research and product roles in Berlin and Toronto. Reach out to Ingo Bax.
Graphcore, the creator of the intelligence processing unit, are hiring for software and hardware engineering, testing, verification, sales and marketing, and more in Oslo, Bristol, and Palo Alto. Reach out to them here.
PROWLER.io, the decision company, are hiring across software engineering, machine learning, UX, product and research in Cambridge, UK. Reach out to Joy Bemrose.
Optimal Labs, the developer of environment control systems for greenhouses, is hiring across engineering, product and research. Reach out to Dave Hunter.
QuantumBlack are hiring for data engineering, machine learning and data science in London and Boston. Reach out to Jonathan Durnford-Smith.
Tractable, the AI for visual inspection tasks, are hiring a VP of sales, operations lead and deep learning researcher in London and San Francisco. Reach out to Giovanni Lotti.
PolyAI, the machine learning platform for conversational agents, is hiring across front and backend engineering, machine learning, and natural language processing in London. Reach out to Nikola Mrksic.
Typeform, the leader in data collection through interactive forms, is hiring for machine learning engineers in Barcelona (job spec). Reach out to Bernat Vazquez.
GoSpotCheck is hiring for computer vision engineers to work on their mobile enterprise software solutions for field teams (job spec). Reach out to Lindsay Fox.
🔬 Research
Here’s a selection of impactful work that caught my eye:
Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks, MIT and Columbia. CNNs are well known to be state of the art image classifiers and many claim this is because their architectures are biologically inspired. In this work, the authors tested the limits these deep learning networks on image recognition tasks by systematically comparing the behavioral responses of these models with the behavioral responses of humans and monkeys. They found that deep learning networks are often confused by the same objects than humans or monkeys (e.g. confusing a camel for a dog). However, all tested CNN models significantly diverged from primate behavior on an image-level tasks even when images are not optimized to be adversarial. As such, these results suggest that CNN models do not precisely capture the neural mechanisms underlying primate object vision and such experiments presented here could be used to improve these models.
Adversarial Patch, Google Brain. In this work, the authors explore whether they can develop a universal strategy to fool computer vision models using an adversarial example that does not need to be imperceptible to the human eye. They develop an image-independent patch that attracts the attention of a neural network over and above any other object in a visual scene. The patch can be placed anywhere in the scene and it works under a wide variety of transformations, thus demonstrating that it is robust and universal. The example result they present is a DeepDream-resembling toaster sticker that makes any vision model think there’s a toaster in the scene :) The patch basically encompases the visual signals that maximally excite the neurons in a network that result in the class label “toaster”. Oh how brittle deep learning is….To hear more from author Justin Gilmer, make sure to register for our RAAIS event on June 29th!
Decentralised learning in systems with many, many strategic agents, PROWLER.io and INAOE Mexico. In this paper, the authors study how selfish agents behave in non-cooperative settings where they must compete to obtain a sequence of rewards within an unknown environment. In these settings, agents do not have full information of the environment from the start and directly computing their optimal behaviour is often prohibitively complex. Using reinforcement learning to compute multi-agent equilibria suffers from exponentially increasing complexity as the number of agents scale. To this end, the authors introduce a model-free, fully decentralised learning procedure that allows equilibrium policies of multi-agent systems to be computed even when the size of the population is extremely large. The technique only requires that agents observe local state information and their realised rewards. They use experiments in control theory and economics to prove their theories.
Deep learning: A critical appraisal, NYU. Kicking off the year with a bang, Gary Marcus (of Geometric Intelligence, acq. Uber) presents a 27 page piece on the limitations of deep learning as the tool of choice in AI today. Of note, he rightly points to the data hunger of deep learning models and that trained models don’t transfer well from one problem domain to another (although AlphaZero is a strong step in that direction). He also notes that deep learning learns correlations between sets of features that are themselves ‘flat’ or non-hierarchical and thus they have difficulty representing data that has intrinsic hierarchical structure (like language). His point about generalisation beyond training examples isn’t an issue for deep learning specifically, either. Marcus proposes the following angles: unsupervised learning, bolder challenges for our AI models, more insight from cognitive and developmental psychology, and symbol-manipulation (i.e. good old fashion AI). The article provoked lots of critique, to which Marcus responds here.
Deep learning scaling is predictable, empirically, Baidu Research (blog post here). A question I am very keen to get to the bottom of is the following: How much labeled data do I need to train a specific machine learning model given access to a certain amount of compute such that it performs well enough to be useful in the real world. To this end, Baidu researchers present a large scale empirical characterisation of generalisation error and model size growth as training sets grow. They test models on the following tasks: machine translation, language modelling, image processing and speech recognition. The results show a “power-law generalization error” that scales across model types and tasks. Model improvements only shift the error but don’t affect the power-law exponent. The model training process is represented in the following diagram, which highlights that the most challenging part to answering my question is knowing what the value of the exponent is during the power-law region. This exponent describes the level of difficulty in learning high quality representations from the training data.
Other highlights:
The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets, Berkeley, Singapore and Google Brain. This paper presents exposure, a simple-to-compute metric that can be applied to any deep learning model for measuring the memorization of secrets. Using this metric, they show how to extract those secrets efficiently using black-box API access.
Berkeley’s AI lab published a review of digital and physical adversarial attacks against object recognition models. It shows different attack vectors that abrogate the classifier’s ability to work properly. This kind of work exposes that vision models don’t learn a real semantic understanding of the world. However, the attacks are quite artificial and rather different from real world strategies used by hackers. Studying how to defend against these attacks doesn’t apply well to the real world. Here is another survey on adversarial attacks on deep learning models for computer vision.
DeepMind released their Control Suite, a set of continuous control tasks with standardised structure and interpretable rewards. It is built on the MuJoCo physics engine and includes benchmarks for several reinforcement learning agents. It bears a lot of similarity with OpenAI’s Gym.
Ever wondered whether it’s possible to project the images we have in our mind? Researchers in Kyoto published work showing we can. In their paper Deep image reconstruction from human brain activity, they measure human functional magnetic resonance imaging patterns and translate this data into a deep neural network for the same input image and reconstruct the image that the human subject is thinking about. For example, the subject is shown an image of a swan, then the fMRI data passes through a neural network that generates a representation of a swan. The Sun’s sensationalised write up here.
Despite its bombastic title, DeepMind’s Machine Theory of Mind proposes the use of meta-learning to build models of other agents from observations of their behaviour alone.
Yann LeCun (FB), Eric Horvitz (MSR), and Peter Norvig (Google) ran a Reddit AMA. Questions ranged from the dominance of deep learning today vs. other methods, the effects of AI on the labor economy, and whether putting together many state-of-the-art models will lead to general intelligence.
📑 Resources
A list of 100 UK AI-driven startups that launched in 2017.
Microsoft has published a tutorial on how to train and test an end-to-end deep learning model for autonomous driving using data collected from the AirSim simulation environment.
Facebook AI Research open sourced the Detectron project, a state-of-the-art platform for object detection powered by Caffe2. The group also released Tensor Comprehensions, a C++ library and mathematical language that helps bridge the gap between researchers, who communicate in terms of mathematical operations, and engineers focusing on the practical needs of running large-scale models on various hardware backends.
UCLA hosted a 2 day workshop featuring talks exploring new deep learning techniques. The videos are here and well worth a watch.
Uber AI labs have launched a residency program. Check it out here.
MIT are adding all the videos to their AIG course to YouTube - tons of great speakers.
ElementAI published a report on the availability of ML talent worldwide. They find 22k PhD-educated researchers worldwide and only 3,074 candidates looking for work.
A consortium from Munich, the EBI and Stanford have created Kipoi, a repository for predictive models in genomics. It It provides a unified framework to archive, share, access, use and build on models developed by the community. Kipoi models come with code to preprocess and load input data in major file formats, which facilitates easy application to new datasets and creation of new derived models.
Snips, the Paris-based AI company, have open sourced their natural language understanding library that drives their embedded voice platform. Technical overview here and blog post here.
The FHI published a review on the context, components, capabilities and consequences of China’s strategy to lead the world in AI.
💰 Venture capital financings and exits
271 deals (64% US, 20% EU, 10% Asia) totalling $5.5B (73% US, 7% EU, 18% Asia). Note a significant uptick (+40%, vs. $3.9B) in deal value from Q4 2017.
Timescale, the open source database for time series, raised a $12.4M Series A led by Benchmark. The database has been downloaded over 100k times in 9 months and is used for industrial data analysis, complex monitoring systems, operational data warehousing, financial risk management, and geospatial asset tracking.
Pony.ai, the Chinese self-driving car company launched by a team of ex-Baidu autonomous car project engineers, raised a whopping $112M Series A led by Morningside, Legend Capital and Sequoia China. They operate in Guangzhou, a port city northwest of Hong Kong with 14 million inhabitants. The city has become favored by AV companies because it is permissive to testing on public roads.
UiPath, the rocketship growing European robotic process automation software supplier, raised a $153M Series B led by Accel (US), following swiftly from their $30M Series A led by Accel (EU).
Blue Vision Labs, the London-based developer of augmented reality technology, emerged from 2 years of stealth development with a $14.5M Series A led by GV and the release of their developer tools for collaborative AR experiences.
25 acquisitions, including the following big deal:
Ring, the connected doorbell company, was acquired by Amazon for a reported >$1B. Ring’s value proposition is to improve neighborhood safety and convenience in letting people you know into your home. For Amazon, this could mean reducing the % of failed deliveries when customers aren’t home. Interestingly, while Ring employed 650 or so FTE in the US, they also had a Ukraine-based entity, Ring Ukraine (formerly Ring Labs), that employed 236 to further AI-based software and hardware R&D for the company.
---
Signing off,
Nathan Benaich, 18 March 2018
Air Street Capital | Twitter | LinkedIn | RAAIS | London.AI