Past Events

Industrial Problems Seminar

Sarah Milstein (NextEra Analytics)

Abstract

In this talk, I will describe some of the renewable energy projects that I've worked on in my current role as a data scientist at NextEra Analytics. These projects include determining the optimal size and design for a new solar power plant and recommending power trades between utilities. Before this job, while I was completing my math PhD at UMN, I spent much of my time doing internships, projects, and independent studies in data science. I will also describe some of this pre-job experience, giving suggestions for ways that you might prepare for your own career. 

Industrial Problems Seminar

Katy Micek (Paramount)

Abstract

Over the past decade, there has been an enormous amount of buzz and enthusiasm about data science. Consider, for example, the alluring title of a Harvard Business Review’s article published in October 2012: “Data Scientist: The Sexiest Job of the 21st Century.” This article was published shortly after I started working in the data science space, and I’ve had a front-row seat to the field’s evolution since that time. The goal of my talk is to provide a perspective on what a career in data science is like. I’ll start by sharing my experience working in the field across various industries and how technical roles have changed during that time. I will then explain how organizations seek to utilize data science and provide examples of the challenges (technical and organizational) that arise while implementing solutions. Finally, I will offer strategies for finding a job that is a good fit in the broadly defined, rapidly changing field of data science.

Data Science Seminar

Konstantinos Spiliopoulos (Boston University)

Abstract

We study the effect of normalization on the layers of deep neural networks. A given layer $i$ with $N_{i}$ hidden units is allowed to be normalized by $1/N_{i}^{\gamma_{i}}$ with $\gamma_{i}\in[1/2,1]$ and we study the effect of the choice of the $\gamma_{i}$ on the statistical behavior of the neural network’s output (such as variance) as well as on the test accuracy on the MNIST and CIFAR10 data sets. We find that in terms of variance of the neural network’s output and test accuracy the best choice is to choose the $\gamma_{i}$’s to be equal to one, which is the mean-field scaling. We also find that this is particularly true for the outer layer, in that the neural network’s behavior is more sensitive in the scaling of the outer layer as opposed to the scaling of the inner layers. The mechanism for the mathematical analysis is an asymptotic expansion for the neural network’s output and corresponding mean field analysis. An important practical consequence of the analysis is that it provides a systematic and mathematically informed way to choose the learning rate hyperparameters. Such a choice guarantees that the neural network behaves in a statistically robust way as the $N_i$'s grow to infinity.

Industrial Problems Seminar

Vasileios Ioannidis (Amazon Search AI)

Please note the 10:10am start time.

Abstract

Graph neural networks (GNNs) learn from complex graph data and have been remarkably successful in various applications and across industries. This presentation first introduces GNNs via the message passing framework and dives into popular GNN variants. Next, it explores the fusion of textual data with heterogeneous graph structures to improve semantic and behavioral representations. It introduces the Language Model GNN (LM-GNN), a framework that efficiently combines large language models and Graph Neural Networks (GNNs) through fine-tuning. LM-GNN supports various tasks like node classification and link prediction and demonstrates its effectiveness. Another aspect addressed is the challenge of effective node representation learning in textual graphs. The Graph-Aware Distillation (Grad) framework is proposed, which encodes graph structures into a Language Model (LM) to enable fast and scalable inference. Grad optimizes GNN and a graphless student model, resulting in superior performance in node classification tasks. Finally, the presentation discusses pre-training text and graph models on large, heterogeneous graphs with textual data using the Graph-Aware Language Model Pre-Training (GALM) framework. It highlights the framework's effectiveness through experiments on real datasets.