MIT and University of Washington Workshop on AI Implementations and Applications: ML Architecture, Systems and Programming Environments - Day 1

June 11, 2021 at 9am-1pm PDT.

Organizer Organizer Organizer

Agenda - Videos of Presentations

  • 9:00 - 9:05 AM PDT - Welcome - Conference Organizers
  • 9:05 - 9:35 AM - Zhizhen Zhong (MIT): Arrow and Bow: Bayesian Optimization System for Reconfigurable Wide-Area Networks
  • 9:35 - 10:05 AM - Mehrdad Khani (MIT): SiP-ML: High-Bandwidth Optical Network Interconnects for Machine Learning Training
  • 10:05 - 10:35 AM - Parimarjan Negi (MIT): Flow-Loss: Learning Cardinality Estimates that Matter to a Query Optimizer
  • 10:35 - 11:00 AM - Break on Gather Online
  • 11:00 - 11:30 AM - Yuchen Jin (OctoML - work done while at UW): AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
  • 11:30 - 12:00 PM - Ryan Marcus (MIT): Machine Learning for Query Optimization
  • 12:00 - 12:30 PM - Liang Luo (Facebook - work done while at UW): Throughput And Cost Optimizer for Public Cloud-based Distributed Training
  • 12:30 - 12:35 PM - Wrap-Up
  • 12:35 - 1:00 PM - Open Networking and Discussion Time on Gather Online

Speaker Abstracts and Bios (in alphabetical order by last name)

Yuchen Jin (OctoML - work done while at UW)

Abstract: The learning rate (LR) schedule is one of the most important hyper-parameters needing careful tuning in training DNNs. However, it is also one of the least automated parts of machine learning systems and usually costs significant manual effort and computing.

We propose an efficient method, AutoLRS, which automatically optimizes the LR for each training stage by modeling training dynamics.

Bio: Yuchen Jin is an MLSys engineer at OctoML. He graduated from University of Washington recently, where he worked with Prof. Arvind Krishnamurthy on machine learning systems and networked systems.

Mehrdad Khani (MIT)

Abstract: We propose optical network interconnects as a key enabler for building high-bandwidth ML training clusters with strong scaling properties.

Bio: Mehrdad is a Ph.D. student at MIT CSAIL, advised by Prof. Mohammad Alizadeh.

Liang Luo (Facebook - work done while at UW)

Abstract: We present a comprehensive throughput and cost-efficiency study across a wide array of instance choices to help prune the optimal VM search space.

Bio: Liang Luo is a research scientist at Facebook AI System Codesign team.

Ryan Marcus (MIT)

Abstract: This talk will explore possibilities for replacing or improving those heuristics with machine learning in the context of query optimization.

Bio: Ryan Marcus is a postdoc at MIT CSAIL working with Tim Kraska.

Parimarjan Negi (MIT)

Abstract: We introduce a new loss function, Flow-Loss, for learning cardinality estimation models, which optimizes explicitly for better query plans.

Bio: Parimarjan is a third year PhD student at MIT.

Zhizhen Zhong (MIT)

Abstract: We leverage Bayesian Optimization (BO) as a sample-efficient framework to optimize this “black-box” system for wavelength deployment at Facebook optical backbone.

Bio: Zhizhen Zhong is a postdoc researcher at MIT CSAIL with Prof. Manya Ghobadi.

QUOTES FROM PREVIOUS CLOUD WORKSHOPS

  • Professor Ken Birman, Cornell University: “I actually thought it was a fantastic workshop, an unquestionable success...”
  • Professor David Patterson, UC Berkeley: “I saw strong participation at the Cloud Workshop, with some high energy and enthusiasm...”
  • Professor Christos Kozyrakis, Stanford University: “I think of these IAP workshops as ‘Hot Chips meets ISCA’...
  • Professor Hakim Weatherspoon, Cornell University: “I have participated in three IAP Workshops since the first one at Cornell in 2013...
  • Dr. Carole-Jean Wu, Facebook Research: “The IAP Cloud Computing workshop provides a great channel for valuable interactions...
  • Nathan Pemberton, UC Berkeley: “IAP workshops provide a valuable chance to explore emerging research topics...”
  • Vishal Shrivastav, Cornell University: “Attending the IAP workshop was a great experience and very rewarding...”
  • Ana Klimovic, Google Brain: “I have attended three IAP workshops and I am consistently impressed...”
  • Dr. Pankaj Mehra, Samsung: “Terrific job organizing the Workshop that gave all parties something worthwhile...
  • Dr. Richard New, Western Digital: “IAP workshops provide a great opportunity to meet with professors and students..."