Papers.

SemanticDB: a common data model for Scala developer tools (invited talk)

Nowadays, there is a clear trend towards standards for communication between developer tools. Language Server Protocol (LSP), a protocol that connects programming language implementations and text editors, has gained strong industrial support and at the time of writing has implementations for many programming languages and editors. Build Server Protocol (BSP) follows in LSP’s tracks with an ambition to define a protocol for communication between language servers and build tools. While lots of work in the open-source community has been invested in unifying user experience (by codifying commonly used operations like go to definition or find all references), relatively less work went into unifying implementor experience. For example, at the moment, there exist five different LSP implementations for Scala. They all implement the same protocol that works with code, but they all use different data structures to represent that code. In this talk, we will present SemanticDB - a data model for semantic information such as symbols and types about programs in Scala and other languages. We will discuss our experience of building an ecosystem of developers tools around SemanticDB and using these tools on Twitter source code - arguably the largest Scala codebase on the planet. SemanticDB is joint work between Eugene Burmako, Twitter and Ólafur Páll Geirsson, EPFL who together developed and maintain the open-source specification and implementations at https://github.com/scalameta/scalameta. The project has received contributions from the Advanced Scala Tools team at Twitter - Eugene Burmako, Shane Delmore, Max Ovsiankin and Uma Srinivasan - as well as many open-source enthusiasts.

How machine learning impacts the undergraduate computing curriculum

The growing importance of machine learning creates challenging questions for computing education.

MacroBase

Size Matters: Improving the Performance of Small Files in Hadoop

Netco: Cache and I/O Management for Analytics over Disaggregated Stores

We consider a common setting where storage is disaggregated from the compute in data-parallel systems. Colocating caching tiers with the compute machines can reduce load on the interconnect but doing so leads to new resource management challenges. We design a system Netco, which prefetches data into caches.

Streams and Tables: Two Sides of the Same Coin

Interflow: interprocedural flow-sensitive type inference and method duplication

Scala heavily relies on a number of object-oriented abstractions to support its feature-rich collections library. There are known techniques that optimize those abstractions away in just-in-time (JIT) compilers, but applying them in the ahead-of-time (AOT) setting is problematic. Profile-guided optimization (PGO) lets AOT compilers apply some of the same optimizations that JIT compilers employ, but it comes at a high complexity cost.

Artwork personalization at netflix

For many years, the main goal of the Netflix personalized recommendation system has been to get the right titles in front of our members at the right time. But the job of recommendation does not end there. The homepage should be able to convey to the member enough evidence of why a title may be good for her, especially for shows that the member has never heard of. One way to address this challenge is to personalize the way we portray the titles on our service. An important aspect of how to portray titles is through the artwork or imagery we display to visually represent each title. The artwork may highlight an actor that you recognize, capture an exciting moment like a car chase, or contain a dramatic scene that conveys the essence of a movie or show. It is important to select good artwork because it may be the first time a member becomes aware of a title (and sometimes the only time), so it must speak to them in a meaningful way. In this talk, we will present an approach for personalizing the artwork we use on the Netflix homepage. The system selects an image for each member and video to give better visual evidence for why the title might be appealing to that particular member. There are many challenges involved in getting artwork personalization to succeed. One challenge is that we can only select a single piece of artwork to represent each title. In contrast, typical recommendation engines present multiple items (in some order) to a member allowing us to subsequently learn about preferences between items through the specific item a member selects from the presented assortment. In contrast, we only collect feedback from the one image that was presented to each member for each title. This leads to a training paradigm based on incomplete logged bandit feedback [1]. Moreover, since the artwork selection process happens on top of a recommendation system, collecting data directly from the production experience (observational data) makes it hard to detangle whether a pl

A tablet game to target dyslexia screening in pre-readers

Behind every great deep learning framework is an even greater programming languages concept

In many areas, such as image recognition, natural language processing, search, recommendation, autonomous cars, systems software and infrastructure, and even Software Engineering tools themselves, Software 2.0 (= programming using learned models) is quickly swallowing Software 1.0 (= programming using handcrafted algorithms). Where the Software 1.0 Engineer formally specifies their problem, carefully designs algorithms, composes systems out of subsystems or decomposes complex systems into smaller components, the Software 2.0 Engineer amasses training data and simply feeds it into an ML algorithm that will synthesize an approximation of the function whose partial extensional definition is that training data. Instead of code as the artifact of interest, in Software 2.0 it is all about the data where compilation of source code is replaced by training models with data. This new style of programming has far-reaching consequences for traditional software engineering practices. Everything we have learned about life cycle models, project planning and estimation, requirements analysis, program design, construction, debugging, testing, maintenance and implementation, … runs the danger of becoming obsolete.

Research for practice

Mind Your State for Your State of Mind

Traces how the interaction between applications and storage has changed as systems scaled out, and what the resulting semantics, performance and failure models mean for developers.

How to fix email: making communication encrypted and decentralized with autocrypt

Pangloss: Fast Entity Linking in Noisy Text Environments

Entity linking is the task of mapping potentially ambiguous terms in text to their constituent entities in a knowledge base like Wikipedia. This is useful for organizing content, extracting structured data from textual documents, and in machine learning relevance applications like semantic search, knowledge graph construction, and question answering. Traditionally, this work has focused on text that has been well-formed, like news articles, but in common real world datasets such as messaging, resumes, or short-form social media, non-grammatical, loosely-structured text adds a new dimension to this problem. This paper presents Pangloss, a production system for entity disambiguation on noisy text. Pangloss combines a probabilistic linear-time key phrase identification algorithm with a semantic similarity engine based on context-dependent document embeddings to achieve better than state-of-the-art results (>5% in F1) compared to other research or commercially available systems. In addition, Pangloss leverages a local embedded database with a tiered architecture to house its statistics and metadata, which allows rapid disambiguation in streaming contexts and on-device disambiguation in low-memory environments such as mobile phones.

COTA: Improving the Speed and Accuracy of Customer Support through Ranking and Deep Networks

Customer support is a central objective at Uber as it helps to ensure an oustanding experience and safety for both riders and drivers. We describe COTA, a system for improving the speed and reliability of customer support ticket resolution at Uber, combining information retrieval and deep learning to suggest ticket contact types and reply templates to customer support agents.

Consistently Eventual

Relay: a new IR for machine learning frameworks

Machine learning powers diverse services in industry including search, translation, recommendation systems, and security. The scale and importance of these models require that they be efficient, expressive, and portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to better accommodate them we propose a new high-level intermediate representation (IR) called Relay. Relay is being designed as a purely-functional, statically-typed language with the goal of balancing efficient compilation, expressiveness, and portability. We discuss the goals of Relay and highlight its important design constraints. Our prototype is part of the open source NNVM compiler framework, which powers Amazon's deep learning framework MxNet.

The Evolution of Content Analysis for Personalized Recommendations at Twitter

We present a broad overview of personalized content recommendations at Twitter, discussing how our approach has evolved over the years, represented by several generations of systems.

A Test Collection for Coreferent Mention Retrieval

This paper introduces the coreferent mention retrieval task, in which the goal is to retrieve sentences that mention a specific entity based on a query by example in which one sentence mentioning that entity is provided. The development of a coreferent mention retrieval test collection is then described. Results are presented for five coreferent mention retrieval systems, both to illustrate the use of the collection and to specify the results that were pooled on which human coreference judgments were performed. The new test collection is built from content that is available from the Linguistic Data Consortium; the partitioning and human annotations used to create the test collection atop that content are being made freely available.

Quantitative Information Extraction From Social Data

Social data is a rich data source for identifying trends and topics of interest based on user activity. Social data also provides opportunities to collect numerical data about events like elections, sport games, disasters or economic news. We propose the problem of identifying relevant quantitative information from social data as annotations for a topic. We investigate how to extract quantitative information and perform a number of experiments and analysis with Twitter data.

Bias on the web

Research for practice

Towards Language Independent Detection of Dyslexia with a Web-based Game

Ryu: fast float-to-string conversion

We present Ryu, a new routine to convert binary floating point numbers to their decimal representations using only fixed-size integer operations, and prove its correctness. Ryu is simpler and approximately three times faster than the previously fastest implementation.

Never-Ending Learning

Research for practice

Hyperledger Fabric: A Distributed Operating System for Permissioned Blockchains

Generalization and equilibrium in generative adversarial nets (GANs) (invited talk).

TempWeb 2018 Chairs' Welcome and Organization

The Shifting Landscape of Web Search and Mining

Q#: Enabling Scalable Quantum Computing and Development with a High-level DSL

Quantum computing exploits quantum phenomena such as superposition and entanglement to realize a form of parallelism that is not available to traditional computing. It offers the potential of significant computational speed-ups in quantum chemistry, materials science, cryptography, and machine learning. The dominant approach to programming quantum computers is to provide an existing high-level language with libraries that allow for the expression of quantum programs. This approach can permit computations that are meaningless in a quantum context; prohibits succint expression of interaction between classical and quantum logic; and does not provide important constructs that are required for quantum programming. We present Q#, a quantum-focused domain-specific language explicitly designed to correctly, clearly and completely express quantum algorithms. Q# provides a type system; a tightly constrained environment to safely interleave classical and quantum computations; specialized syntax; symbolic code manipulation to automatically generate correct transformations of quantum operations; and powerful functional constructs which aid composition.

MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis

Sketching Linear Classifiers over Data Streams

Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark

Debugging data flows in reactive programs

Reactive Programming is a style of programming that provides developers with a set of abstractions that facilitate event handling and stream processing. Traditional debug tools lack support for Reactive Programming, leading developers to fallback to the most rudimentary debug tool available: logging to the console.

Research for practice

Research for practice

Research for practice

Cloud Computed Machine Learning Based Real-Time Litter Detection using Micro-UAV Surveillance

Litter can remain undetected and uncollected for extended periods of time, leading to detrimental consequences on the environment. Solutions to mitigating these effects focus on severe legal action directed towards offenders or litter collection events, all of which are not automated. Therefore, to reduce the amount of manual labor required for current solutions, this project aims to implement an automated micro-unmanned aerial vehicle (UAV) capable of real time litter detection from UAV surveillance footage using cloud-based machine learning.

Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models

Interpretation and diagnosis of machine learning models have gained renewed interest in recent years with breakthroughs in new approaches. We present Manifold, a framework that utilizes visual analysis techniques to support interpretation, debugging, and comparison of machine learning models in a more transparent and interactive manner. Conventional techniques usually focus on visualizing the internal logic of a specific model type (i.e., deep neural networks), lacking the ability to extend to a more complex scenario where different model types are integrated. To this end, Manifold is designed as a generic framework that does not rely on or access the internal logic of the model and solely observes the input (i.e., instances or features) and the output (i.e., the predicted result and probability distribution). We describe the workflow of Manifold as an iterative process consisting of three major phases that are commonly involved in the model development and diagnosis process: inspection (hypothesis), explanation (reasoning), and refinement (verification). The visual components supporting these tasks include a scatterplot-based visual summary that overviews the models' outcome and a customizable tabular view that reveals feature discrimination. We demonstrate current applications of the framework on the classification and regression tasks and discuss other potential machine learning use scenarios where Manifold can be applied.

Scalable Dynamic Graph Summarization

Multiple-Swarm Ensembles: Improving the Predictive Power and Robustness of Predictive Models and Its Use in Computational Biology

An Architecture for Analysis

Domain Randomization and Generative Models for Robotic Grasping

Deep learning-based robotic grasping has made significant progress thanks to algorithmic improvements and increased data availability. However, state-of-the-art models are often trained on as few as hundreds or thousands of unique object instances, and as a result generalization can be a challenge. In this work, we explore a novel data generation pipeline for training a deep neural network to perform grasp planning that applies the idea of domain randomization to object synthesis. We generate millions of unique, unrealistic procedurally generated objects, and train a deep neural network to perform grasp planning on these objects. Since the distribution of successful grasps for a given object can be highly multimodal, we propose an autoregressive grasp planning model that maps sensor inputs of a scene to a probability distribution over possible grasps. This model allows us to sample grasps efficiently at test time (or avoid sampling entirely). We evaluate our model architecture and data generation pipeline in simulation and the real world. We find we can achieve a >90% success rate on previously unseen realistic objects at test time in simulation despite having only been trained on random objects. We also demonstrate an 80% success rate on real-world grasp attempts despite having only been trained on random simulated objects.

On the Ability of Mobile Sensor Networks to Diffuse Information

We examine the ability of networks formed by mobile sensor nodes to diffuse information in the case when communication is only possible during opportunistic encounters. Our setting assumes that mobile nodes are continuously sensing the world and acquiring new information. We form an abstract model of this situation and show by theoretical analysis, simulation, and real mobility data that the diffusion of information in this setting cannot be as efficient as when we allow arbitrary contact patterns between the nodes with the same overall contact statistics. This establishes a fundamental asymptotic limitation on the information diffusion capacity of such opportunistic mobile sensor networks - the encounter patterns arising out of physical motions in a geometric space are not ideal for information diffusion.

Detecting Poisoning Attacks on Machine Learning in IoT Environments

Machine Learning (ML) plays an increasing role in Internet of Things (IoT), both in the Cloud and at the Edge, using trained models for applications from factory automation to environmental sensing. However, using ML in IoT environments presents unique security challenges. In particular, adversaries can manipulate the training data by tampering with sensors' measurements. This type of attack, known as a poisoning attack has been shown to significantly decrease overall performance, cause targeted misclassification or bad behavior, and insert "backdoors" and "neural trojans". Taking advantage of recently developed tamper-free provenance frameworks, we present a methodology that uses contextual information about the origin and transformation of data points in the training set to identify poisonous data. Our approach works with or without a trusted test data set. Using the proposed approach poisoning attacks can be effectively detected and mitigated in IoT environments with reliable provenance information.

Enhancement and Analysis of Conversational Speech: JSALT 2017

Measuring Uncertainty in Deep Regression Models: The Case of Age Estimation from Speech

A Pruned Rnnlm Lattice-Rescoring Algorithm for Automatic Speech Recognition

CEA: Clinical Event Annotator mHealth Application for Real-time Patient Monitoring

This research develops a novel dynamic mobile health (mHealth) application (app), called the Clinical Event Annotator (CEA). The CEA comprises of a native Android tablet app and an administrative web app. The native app is used at the patient bedside to manually annotate clinical events in real-time. Event types include patient monitor alarms, routine care, clinical interventions, and patient movements. The app can be dynamically updated with user-defined customized events. The web app generates reports of the annotation sessions. The CEA app is developed to support a clinical study that explores the use of pressure-sensitive mats (PSM) in the neonatal intensive care unit (NICU) to detect the respiratory rate (RR), heart rate (HR), and movement of critically ill neonatal patients. High-fidelity CEA app annotations are synced with a backend database that enables integration and synchronization with independently acquired patient monitoring data, such as RR, HR, and contact pressure data from the PSM. The gold standard CEA annotations serve the purpose of retrospectively training machine learning algorithms for clinical event detection. Preliminary test results from use of the app in the clinical study are presented. Development of the CEA app is a unique and novel contribution that addresses the well-known problem of manually annotating physiologic data streams to support clinical data mining applications.