Fourth International Workshop on Symbolic-Neural Learning (SNL-2020)

Keynote Talks:

1. Nima Mesgarani (The University of Columbia in the City of New York)

   

   Cortical processing of linguistic information in the human brain

   Abstract:
   Speech perception in real-world situations requires a listener's auditory system to extract and represent linguistic features often in the presence of interfering sound sources and changing background conditions. This complex process includes nonlinear, dynamic, and adaptive transformations of the speech signal as it propagates through the auditory pathway. The behavioral consequence of these neural processes is the remarkable human ability to perceive speech in adverse acoustic conditions. Despite the immense importance of this research question, the nature of these transformations in the human brain remains unclear. The progress is lacking because cortical processing of speech is highly spatially precise, temporally feeling, and computationally nonlinear, far beyond the resolution of noninvasive methods and the capacity of common computational models. To address these shortcomings, we use invasive human electrophysiology and deep learning models to determine the where, when, and how of speech processing in the human auditory cortex. This combination of experimental and computational approaches advances our knowledge of the functional organization of human auditory cortex and pave the way toward more complete models of speech processing in the human brain.

   Bio:
   Nima Mesgarani is an associate professor at Zuckerman Mind-Brain-Behavior Institute of Columbia University in the City of New York. He received his Ph.D. from the University of Maryland and was a postdoctoral scholar in Center for Language and Speech Processing at Johns Hopkins university and the Neurosurgery Department of the University of California San Francisco. He has been named a Pew Scholar for Innovative Biomedical Research and UNICEF-Netexplo top-10 innovator in 2018. He has received the National Science Foundation Early Career and Auditory Neuroscience Young Investigator Awards. His interdisciplinary research combines theoretical and experimental techniques to model the neural mechanisms involved in human speech communication which critically impacts research in modeling speech processing and speech brain-computer interface technologies.

2. Yejin Choi (The University of Washington)

   

   Abstract:
   

   Bio:
   Yejin Choi is an associate professor at the Paul G. Allen School of Computer Science & Engineering at the University of Washington and also a senior research manager at AI2 overseeing the project Mosaic. Her research interests include language grounding with vision, physical and social commonsense knowledge, language generation with long-term coherence, conversational AI, and AI for social good. She was a recepient of Borg Early Career Award (BECA) in 2018, among the IEEE's AI Top 10 to Watch in 2015, a co-recipient of the Marr Prize at ICCV 2013, and a faculty advisor for the Sounding Board team that won the inaugural Alexa Prize Challenge in 2017. Her work on detecting deceptive reviews, predicting the literary success, and interpreting bias and connotation has been featured by numerous media outlets including NBC News for New York, NPR Radio, New York Times, and Bloomberg Business Week. She received her Ph.D. in Computer Science from Cornell University.

3. In So Kweon (Korea Advanced Institute of Science and Technology)

   

   Scene Understanding with Image Captioning and Visual Relation Detection

   Abstract:
   Image captioning is often regarded as a data-hungry task. Training a model for captioning requires a large-scale human-labeled dataset, more than 100K images with grammatically plausible sentences one-by-one (at least, 5 captions per image). It is not easy to construct such datasets in the real world. Constructing a large-scale image-caption dataset requires vast human effort. In this talk, we present two approaches to deal with the data hungry problem of image captioning. Firstly, we present a novel semi-supervised framework by learning with "unpaired image-language data", since colleting unpaired images and captions separately becomes an easy task. The semi-supervised framework exploits unpaired data by only adding scarce paired samples without any external data, such as labels and pivot languages. Specifically, we present a GAN-based model which learns to distinguish between real/fake image-caption pairs and exploit the GAN model again to produce pseudo-labels. Our method can save near half (40%) of the human labeling effort of constructing a dataset. The baseline with 10% paired data starts to produce plausible captions while ours requires only 1%. Sometimes, our model generates captions even more accurate than the baseline trained on the 100% paired data. Secondly, the traditional relationship datasets often show very limited diversity. Our relational captioning dataset is automatically built to have more natural sentences with richer expressions by utilizing Visual Genome (VG) relationship dataset. A novel triple-stream LSTMs demonstrates the feasibility of the dataset with the SOTA performance in sentence-based region-pair retrieval task, which is a very difficult task for both Scene Graph based and VRD approaches.

   Bio:
   In So Kweon received the B.S. and the M.S. degrees in Mechanical Design and Production Engineering from Seoul National University, Korea, in 1981 and 1983, respectively, and the Ph.D. degree in Robotics from the Robotics Institute at Carnegie Mellon University in 1990. He worked for Toshiba R&D Center, Japan, and joined KAIST in 1992. He is now a KEPCO Chair professor of School of Electrical Engineering and the director for the National Core Research Center - P3 DigiCar Center at KAIST. His research interests include computer vision and robotics. He has co-authored several books, including "Metric Invariants for Camera Calibration," and more than 500 technical papers. He served the Korea Computer Vision Society as the founding president for two years since 2016, and had been on the Editorial Board member for "International Journal of Computer Vision" for ten years since 2005. He is a member of many computer vision and robotics conference program committees and had been a general and program co-chairs for several conferences and workshops, including ACCV'2012 and ICCV'2019. He received several awards from the international conferences, including "The Best Paper Award of the IEEE Transaction on CSVT 2014", "The Best Student Paper Runnerup Award in the IEEE-CVPR 2009". He is a member of the KROS, KCVS and IEEE.

4. Abhinav Gupta (Carnegie Mellon University)

   

   Abstract:
   

   Bio:
   Abhinav Gupta is an Associate Professor at the Robotics Institute, Carnegie Mellon University. and Research Manager at Facebook AI Research (FAIR). Abhinav's research focuses on scaling up learning by building self-supervised, lifelong and interactive learning systems. Specifically, he is interested in how self-supervised systems can effectively use data to learn visual representation, common sense and representation for actions in robots. Abhinav is a recipient of several awards including ONR Young Investigator Award, PAMI Young Research Award, Sloan Research Fellowship, Okawa Foundation Grant, Bosch Young Faculty Fellowship, YPO Fellowship, IJCAI Early Career Spotlight, ICRA Best Student Paper award, and the ECCV Best Paper Runner-up Award. His research has also been featured in Newsweek, BBC, Wall Street Journal, Wired and Slashdot.