Big-Data applications impose significant challenges in computing infrastructures. In this talk, I will introduce some of our recent projects on the systems support for Big-Data computation at Microsoft Research, in collaboration with product teams. Our journey begins with understanding real production systems at Microsoft: focusing on efficient batch processing (Dryad) and effective sharing of cluster resources (Apollo). Based on the experience gained, we identified and enabled new capabilities such as matrix computation (MadLINQ) and real-time streaming (TimeStream, SCOPE Streaming). I will focus on the abstractions for building related Big-Data systems, as well as the design choices we made according to the real, production requirements and workloads at Microsoft. I will also briefly talk about other related work in the Systems Research Group at Microsoft Research Asia.

Dr. Zhengping Qian received his Ph.D. from South China University of Technology and joined Microsoft Research Asia as a researcher in 2009. His research interests are in distributed and data-parallel computation, as well as emerging and/or challenging applications. Dr. Qian is a recipient of the Best Paper Award in EuroSys 2012. Currently, he enjoys identifying general abstractions for building Big-Data systems.