Papers.

The ngEHT Analysis Challenges

Comparison of category and letter fluency tasks through automated analysis

How are AI assistants changing higher education?

Evolution of linguistic markers of agency, centrality and content during metacognitive therapy for psychosis

Characterizing the Discourse of Popular Diets to Describe Information Dispersal and Identify Leading Voices, Interaction, and Themes of Mental Health: Social Network Analysis

Do Case Reports and Case Series Generate Clinical Discoveries About Preeclampsia? A Systematic Review

Background Preeclampsia is a leading cause of maternal and perinatal mortality and morbidity. The management of preeclampsia has not changed much in more than two decades, and its aetiology is still not fully understood. Case reports and case series have traditionally been used to communicate new knowledge about existing conditions. Whether this is true for preeclampsia is not known. Objective To determine whether recent case reports or case series have generated new knowledge and clinical discoveries about preeclampsia. Methods A detailed search strategy was developed in consultation with a medical librarian. Two bibliographic databases were searched through Ovid: Embase and MEDLINE. We selected case reports or case series published between 2015 and 2020, comprising pregnant persons diagnosed with hypertensive disorders of pregnancy, including preeclampsia. Two reviewers independently screened all publications. One reviewer extracted data from included studies, while another conducted a quality check of extracted data. We developed a codebook to guide our data extraction and outcomes assessment. The quality of each report was determined based on Joanna Briggs Institute (JBI) critical appraisal checklist for case reports and case series. Results We included 104 case reports and three case series, together comprising 118 pregnancies. A severe presentation or complication of preeclampsia was reported in 81% of pregnancies, and 84% had a positive maternal outcome, free of death or persistent complications. Only 8% of the case reports were deemed to be of high quality, and 53.8% of moderate quality; none of the case series were of high quality. A total of 26 of the 107 publications (24.3%) included a novel clinical discovery as a central theme. Conclusion Over two-thirds of recent case reports and case series about preeclampsia do not appear to present new knowledge or discoveries about preeclampsia, and most are of low quality.

Radar de Parité: An NLP system to measure gender representation in French news stories

We present the Radar de Parite, an automated Natural Language Processing (NLP) system that measures the proportion of women and men quoted daily in six Canadian French-language media outlets. We outline the system's architecture and detail the challenges we overcame to address French-specific issues, in particular regarding coreference resolution, a new contribution to the NLP literature on French. We also showcase statistics covering over one year's worth of data (282,512 news articles).

An AI Method for Assessing Coding Consistency in a Large Dataset

Predicting disease severity in Multiple Sclerosis using multimodal data and machine learning

ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment

We present a comprehensive evaluation of large language models for multilingual readability assessment. Existing evaluation resources lack domain and language diversity, limiting the ability for cross-domain and cross-lingual analyses. This paper introduces ReadMe++, a multilingual multi-domain dataset with human annotations of 9757 sentences in Arabic, English, French, Hindi, and Russian, collected from 112 different data sources. This benchmark will encourage research on developing robust multilingual readability assessment methods. Using ReadMe++, we benchmark multilingual and monolingual language models in the supervised, unsupervised, and few-shot prompting settings. The domain and language diversity in ReadMe++ enable us to test more effective few-shot prompting, and identify shortcomings in state-of-the-art unsupervised methods. Our experiments also reveal exciting results of superior domain generalization and enhanced cross-lingual transfer capabilities by models trained on ReadMe++. We will make our data publicly available and release a python package tool for multilingual sentence readability prediction using our trained models at: https://github.com/tareknaous/readme

JUAGE at SemEval-2023 Task 10: Parameter Efficient Classification.

News Signals: An NLP Library for Text and Time Series

Towards Agile Text Classifiers for Everyone.

The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding

Symbol tuning improves in-context learning in language models.

H2O Open Ecosystem for State-of-the-art Large Language Models

Large Language Models (LLMs) represent a revolution in AI. However, they also pose many significant risks, such as the presence of biased, private, copyrighted or harmful text. For this reason we need open, transparent and safe solutions. We introduce a complete open-source ecosystem for developing and testing LLMs. The goal of this project is to boost open alternatives to closed-source approaches. We release h2oGPT, a family of fine-tuned LLMs of diverse sizes. We also introduce H2O LLM Studio, a framework and no-code GUI designed for efficient fine-tuning, evaluation, and deployment of LLMs using the most recent state-of-the-art techniques. Our code and models are fully open-source. We believe this work helps to boost AI development and make it more accessible, efficient and trustworthy. The demo is available at: https://gpt.h2o.ai/

Muted: Multilingual Targeted Offensive Speech Identification and Visualization

Offensive language such as hate, abuse, and profanity (HAP) occurs in various content on the web. While previous work has mostly dealt with sentence level annotations, there have been a few recent attempts to identify offensive spans as well. We build upon this work and introduce Muted, a system to identify multilingual HAP content by displaying offensive arguments and their targets using heat maps to indicate their intensity. Muted can leverage any transformer-based HAP-classification model and its attention mechanism out-of-the-box to identify toxic spans, without further fine-tuning. In addition, we use the spaCy library to identify the specific targets and arguments for the words predicted by the attention heatmaps. We present the model's performance on identifying offensive spans and their targets in existing datasets and present new annotations on German text. Finally, we demonstrate our proposed visualization tool on multilingual inputs.

Automated De-Identification of Arabic Medical Records

As Electronic Health Records (EHR) become ubiquitous in healthcare systems worldwide, including in Arabic-speaking countries, the dual imperative of safeguarding patient privacy and leveraging data for research and quality improvement grows. This paper presents a firstof-its-kind automated de-identification pipeline for medical text specifically tailored for the Arabic language. This includes accurate medical Named Entity Recognition (NER) for identifying personal information; data obfuscation models to replace sensitive entities with fake entities; and an implementation that natively scales to large datasets on commodity clusters. This research makes two contributions. First, we adapt two existing NER architectures-BERT For Token Classification (BFTC) and BiLSTM-CNN-Char -to accommodate the unique syntactic and morphological characteristics of the Arabic language. Comparative analysis suggests that BFTC models outperform Bi-LSTM models, achieving higher F1 scores for both identifying and redacting personally identifiable information (PII) from Arabic medical texts. Second, we augment the deep learning models with a contextual parser engine to handle commonly missed entities. Experiments show that the combined pipeline demonstrates superior performance with micro F1 scores ranging from 0.94 to 0.98 on the test dataset, which is a translated version of the i2b2 2014 de-identification challenge, across 17 sensitive entities. This level of accuracy is in line with that achieved with manual de-identification by domain experts, suggesting that a fully automated and scalable process is now viable.

Characterizing and detecting delirium with clinical and computational measures of speech and language disturbance

Towards Auto-Generated Data Systems

After decades of progress, database management systems (DBMSs) are now the backbones of many data applications that we interact with on a daily basis. Yet, with the emergence of new data types and hardware, building and optimizing new data systems remain as difficult as the heyday of relational databases. In this paper, we summarize our work towards automating the building and optimization of data systems. Drawing from our own experience, we further argue that any automation technique must address three aspects: user specification, code generation, and result validation. We conclude by discussing a case study using videos data processing, along with opportunities for future research towards designing data systems that are automatically generated.

OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance from Database Query Event Logs

Epoxy: ACID Transactions across Diverse Data Stores

Cloud as a malware infiltration channel

Changes in Digital Speech Measures in Asymptomatic Carriers of Pathogenic Variants Associated With Frontotemporal Degeneration

Sex differences in the temporal dynamics of autistic children’s natural conversations

Understanding Gambling Esports Markets

Data from The Impact of EGFR T790M Mutations and BIM mRNA Expression on Outcome in Patients with EGFR-Mutant NSCLC Treated with Erlotinib or Chemotherapy in the Randomized Phase III EURTAC Trial

Data from The Impact of EGFR T790M Mutations and BIM mRNA Expression on Outcome in Patients with EGFR-Mutant NSCLC Treated with Erlotinib or Chemotherapy in the Randomized Phase III EURTAC Trial

Supplementary Tables 1 - 9, Figures 1 - 5 from The Impact of EGFR T790M Mutations and BIM mRNA Expression on Outcome in Patients with EGFR-Mutant NSCLC Treated with Erlotinib or Chemotherapy in the Randomized Phase III EURTAC Trial

Supplementary Tables 1 - 9, Figures 1 - 5 from The Impact of EGFR T790M Mutations and BIM mRNA Expression on Outcome in Patients with EGFR-Mutant NSCLC Treated with Erlotinib or Chemotherapy in the Randomized Phase III EURTAC Trial

Report on the Workshop on Learning and Evaluating Recommendations with Impressions (LERI) at RecSys 2023

The Workshop on Learning and Evaluating Recommendations with Impressions (LERI) was held in conjunction with the 17th ACM Conference on Recommender Systems (RecSys 2023). The program included a keynote, a panel discussion and 7 paper presentations. The proceedings of the workshop are available online.1 The LERI workshop focused on all aspects related to the use of impressions for recommendation with the aim to bring the community together and share experience and perspectives. Recommender systems typically rely on past user interactions as the primary source of information for making predictions. Impressions are an important source of information that indicate the items displayed on screen when the user interacted (or not) with them, and have the potential to impact the field of recommender systems in several ways. Impressions present new research questions and opportunities, but also bring new challenges. Date: 19 September 2023. Website: https://recsyspolimi.github.io/leri2023/.

Report on the 13th Workshop on Temporal Web Analytics (TempWeb 2023) at WWW 2023

Capturing Types

Type systems usually characterize the shape of values but not their free variables. However, many desirable safety properties could be guaranteed if one knew the free variables captured by values. We describe CC < :◻ , a calculus where such captured variables are succinctly represented in types, and show it can be used to safely implement effects and effect polymorphism via scoped capabilities. We discuss how the decision to track captured variables guides key aspects of the calculus, and show that CC < :◻ admits simple and intuitive types for common data structures and their typical usage patterns. We demonstrate how these ideas can be used to guide the implementation of capture checking in a practical programming language.

Uncovering Bias in Personal Informatics

LURK: Lambda, the Ultimate Recursive Knowledge (Experience Report)

A Local-First Approach for Green Smart Contracts

Navigating the Feedback Loop in Recommender Systems: Insights and Strategies from Industry Practice

Understanding and measuring the impact of feedback loops in industrial recommender systems is challenging, leading to the underestimation of their deterioration. In this study, we define open and closed feedback loops and investigate the unique reasons behind the emergence of feedback loops in the industry, drawing from real-world examples that have received limited attention in prior research. We highlight the measurement challenges associated with capturing the full impact of feedback loops using traditional online A/B tests. To address this, we propose the use of offline evaluation frameworks as surrogates for long-term feedback loop bias, supported by a practical simulation system using real data. Our findings provide valuable insights for optimizing the performance of recommender systems operating under feedback loop conditions.

Reward innovation for long-term member satisfaction

Recommender systems commonly train on user engagements because of their abundance, immediacy of feedback, and the insights they provide into users preferences. However, this approach may unintentionally prioritize optimizing short-term engagements over a product’s or business’s long-term objectives. At Netflix, our recommender systems are designed with the goal of maximizing long-term member satisfaction. To achieve this objective, we adopt a practical approach that augments engagement data with reward signals aligned with long term member satisfaction. This process of identifying, evaluating, and integrating reward signals into an existing learning algorithm is what we term reward innovation. In this work, we present the challenges of applying this approach to a large-scale recommender system and share our approach to addressing them.

Workshop on Learning and Evaluating Recommendations with Impressions (LERI)

Recommender systems typically rely on past user interactions as the primary source of information for making predictions. However, although highly informative, past user interactions are strongly biased. Impressions, on the other hand, are a new source of information that indicate the items displayed on screen when the user interacted (or not) with them, and have the potential to impact the field of recommender systems in several ways. Early research on impressions was constrained by the limited availability of public datasets, but this is rapidly changing and, as a consequence, interest in impressions has increased. Impressions present new research questions and opportunities, but also bring new challenges. Several works propose to use impressions as part of recommender models in various ways and discuss their information content. Others explore their potential in off-policy-estimation and reinforcement learning. Overall, the interest of the community is growing, but efforts in this direction remain disconnected. Therefore, we believe that a workshop would be useful in bringing the community together.

LearnedSort as a learning-augmented SampleSort: Analysis and Parallelization.

Efficient Memory Management for Large Language Model Serving with PagedAttention

High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically. When managed inefficiently, this memory can be significantly wasted by fragmentation and redundant duplication, limiting the batch size. To address this problem, we propose PagedAttention, an attention algorithm inspired by the classical virtual memory and paging techniques in operating systems. On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage. Our evaluations show that vLLM improves the throughput of popular LLMs by 2--4× with the same level of latency compared to the state-of-the-art systems, such as FasterTransformer and Orca. The improvement is more pronounced with longer sequences, larger models, and more complex decoding algorithms. vLLM's source code is publicly available at https://github.com/vllm-project/vllm.

Prefetching Using Principles of Hippocampal-Neocortical Interaction

Memory prefetching improves performance across many systems layers. However, achieving high prefetch accuracy with low overhead is challenging, as memory hierarchies and application memory access patterns become more complicated. Furthermore, a prefetcher's ability to adapt to new access patterns as they emerge is becoming more crucial than ever. Recent work has demonstrated the use of deep learning techniques to improve prefetching accuracy, albeit with impractical compute and storage overheads. This paper suggests taking inspiration from the learning mechanisms and memory architecture of the human brain---specifically, the hippocampus and neocortex---to build resource-efficient, accurate, and adaptable prefetchers.

The Gradient of Generative AI Release: Methods and Considerations

The Impact of the Web on Information Retrieval

Evaluation of Submission Limits and Regression Penalties to Improve Student Behavior with Automatic Assessment Systems.

LO

Invited Paper: Initial Steps Toward a Compiler for Distributed Programs

In the Hydro project we are designing a compiler toolkit that can optimize for the concerns of distributed systems, including scale-up and scale-down, availability, and consistency of outcomes across replicas. This invited paper overviews the project, and provides an early walk-through of the kind of optimization that is possible. We illustrate how type transformations as well as local program transformations can combine, step by step, to convert a single-node program into a variety of distributed design points that offer the same semantics with different performance and deployment characteristics.

Type Theory as a Unifying Paradigm for Modern Databases

Fair Multilingual Vandalism Detection System for Wikipedia

SCALO: An Accelerator-Rich Distributed System for Scalable Brain-Computer Interfacing

SCALO is the first distributed brain-computer interface (BCI) consisting of multiple wireless-networked implants placed on different brain regions. SCALO unlocks new treatment options for debilitating neurological disorders and new research into brain-wide network behavior. Achieving the fast and low-power communication necessary for real-time processing has historically restricted BCIs to single brain sites. SCALO also adheres to tight power constraints, but enables fast distributed processing. Central to SCALO's efficiency is its realization as a full stack distributed system of brain implants with accelerator-rich compute. SCALO balances modular system layering with aggressive cross-layer hardware-software co-design to integrate compute, networking, and storage. The result is a lesson in designing energy-efficient networked distributed systems with hardware accelerators from the ground up.