Tearing Down the Deep Learning Performance Wall
Lecture: CITRIS Research Exchange: Big Data: People and Robots Initiative | February 15 | 12-1 p.m. | Sutardja Dai Hall, 310, Banatao Auditorium
Greg Diamos, Senior Researcher, Baidu
CITRIS and the Banatao Institute
Just this year, deep learning has fueled significant progress in computer vision, speech recognition, and natural language processing. We have seen a computer beat the world champion in Go with help from deep learning, and a single deep learning algorithm learn to recognize two vastly different languages, English and Mandarin. At Baidu, we think that this is just the beginning, and high performance computing is poised to help.
It turns out that deep learning is compute limited, even on the fastest machines that we can build. This talk will provide empirical evidence from our Deep Speech work that application level performance (e.g. recognition accuracy) scales with data and compute, transforming some hard AI problems into problems of computational scale.
This talk will describe the performance characteristics of Baidu’s deep learning workloads in detail, focusing on the recurrent neural networks used in Deep Speech as a case study. It will cover challenges to further improving performance, and outline a plan of attack for tearing down the remaining obstacles standing in the way of strong scaling deep learning to the largest machines in the world.
——–
Greg Diamos is a senior researcher at Baidu’s Silicon Valley AI Lab (SVAIL). Previously he was on the research team at NVIDIA. Greg holds a PhD from the Georgia Institute of Technology, where he contributed to the development of the GPU-Ocelot dynamic compiler, which targeted CPUs and GPUs from the same program representation. His PhD thesis pioneered execution models for heterogeneous processors.
———
Free and open to the public. Register online by Monday for a free lunch at UC Berkeley. The CITRIS Research Exchange Seminar Series is a weekly dialogue highlighting leading voices on societal-scale research issues. Each one-hour seminar starts at 12pm Pacific time and is hosted live at Sutardja Dai Hall on the UC Berkeley campus.
Live broadcast at https://www.youtube.com/user/citrisuc/live. All talks may be viewed on our YouTube channel .
Live webcasting of each CITRIS Research Exchange seminar is available at these CITRIS campuses:
CITRIS @ Davis: 1065 Kemper Hall, College of Engineering, UC Davis
Free
Free lunch available (limited #s). You must register by the Monday before the event for lunch.
(510) 664-4508
Webcast. Event will be webcast live during the event and then available on the CITRIS Youtube site.