Papers.

LST-Bench: Benchmarking Log-Structured Tables in the Cloud

Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment

Optimizing Distributed Protocols with Query Rewrites

Distributed protocols such as 2PC and Paxos lie at the core of many systems in the cloud, but standard implementations do not scale. New scalable distributed protocols are developed through careful analysis and rewrites, but this process is ad hoc and error-prone. This paper presents an approach for scaling any distributed protocol by applying rule-driven rewrites, borrowing from query optimization. Distributed protocol rewrites entail a new burden: reasoning about spatiotemporal correctness. We leverage order-insensitivity and data dependency analysis to systematically identify correct coordination-free scaling opportunities. We apply this analysis to create preconditions and mechanisms for coordination-free decoupling and partitioning, two fundamental vertical and horizontal scaling techniques. Manual rule-driven applications of decoupling and partitioning improve the throughput of 2PC by 5× and Paxos by 3×, and match state-of-the-art throughput in recent work. These results point the way toward automated optimizers for distributed protocols based on correct-by-construction rewrite rules.

Responsible AI Day

Trinity: A Fast Compressed Multi-attribute Data Store

With the proliferation of attribute-rich machine-generated data, emerging real-time monitoring, diagnosis, and visualization tools ingest and analyze such data across multiple attributes simultaneously. Due to the sheer volume of the data, applications need storage-efficient and performant data representations to analyze them efficiently. We present TRINITY, a system that simultaneously facilitates query and storage efficiency across large volumes of multi-attribute records. Trinity accomplishes this through a new dynamic, succinct, multi-dimensional data structure, MdTrie. MdTrie employs a combination of novel Morton code generalization, a multi-attribute query algorithm, and a self-indexed trie structure to achieve the above goals. Our evaluation of TRINITY for real-world use-cases shows that compared to state-of-the-art systems, it supports (1) 7.2-59.6× faster multi-attribute searches, (2) storage footprint comparable to OLAP columnar stores and 4.8-15.1× lower than NoSQL stores and OLTP databases, and (3) point query throughput comparable to NoSQL stores and 1.7-52.5× higher than OLTP databases and OLAP columnar stores.

HelpMe: Student Help Seeking using Office Hours and Email.

The Limitations of Data, Machine Learning and Us

The Hopsworks Feature Store for Machine Learning

Flexible Non-intrusive Dynamic Instrumentation for WebAssembly

PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation

Introduction to Responsible AI

Implications of Regulations on the Use of AI and Generative AI for Human-Centered Responsible Artificial Intelligence

Data Management for ML-Based Analytics and Beyond

EmbedDB: A High-Performance Database for Resource-Constrained Embedded Systems Too Small for SQLite.

14th Temporal Web Analytics Workshop (TempWeb)

Nonstandard Errors

In statistics, samples are drawn from a population in a data‐generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence‐generating process (EGP). We claim that EGP variation across researchers adds uncertainty—nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer‐review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Evolution of Linguistic Markers of Agency, Centrality and Content During Metacognitive Therapy for Psychosis: A Pilot Exploratory Study

Overview of the Ninth Dialog System Technology Challenge: DSTC9

Quantum Annotated AI Modelling of Digital Assets

This research investigates the practical application of artificial intelligence in digital asset trading to ensure secure transactions on public ledgers. The emphasis lies in predicting prices based on data from digital exchanges, utilizing algorithmic design and seamless automation through APIs across multiple platforms. Additionally, this study explores the integration of quantum computing by visualizing Interference Quantum Processor (IQP) circuits with digital asset prices as annotations. The evaluation methodology incorporates real-time exchange values of Digital Assets, leveraging the collaborative potential of AI, data science, and network capabilities for synchronized valuation. The presented findings underscore the practical application of these technologies in enhancing the efficiency and security of transactions within the dynamic landscape of digital exchanges.

Distributed Brain–Computer Interfacing With a Networked Multiaccelerator Architecture

SCALO is the first distributed brain–computer interface (BCI) consisting of multiple wireless-networked implants placed on different brain regions. SCALO unlocks new treatment options for debilitating neurological disorders and new research into brainwide network behavior. Achieving the fast and low-power communication necessary for real-time processing has historically restricted BCIs to single brain sites. SCALO also adheres to tight power constraints but enables fast distributed processing. Central to SCALO’s efficiency is its realization as a full stack distributed system of brain implants with accelerator-rich compute. SCALO balances modular system layering with aggressive cross-layer hardware–software co-design to integrate compute, networking, and storage. The result is a lesson in designing energy-efficient networked distributed systems with hardware accelerators from the ground up.

Responsible AI: An Urgent Mandate

Alignment Studio: Aligning Large Language Models to Particular Contextual Regulations

The alignment of large language models is usually done by model providers to add or control behaviors that are common or universally understood across use cases and contexts. In contrast, in this article, we present an approach and architecture that empowers application developers to tune a model to their particular values, social norms, laws and other regulations, and orchestrate between potentially conflicting requirements in context. We lay out three main components of such an Alignment Studio architecture: Framers, Instructors, and Auditors that work in concert to control the behavior of a language model. We illustrate this approach with a running example of aligning a company's internal-facing enterprise chatbot to its business conduct guidelines.

Mobility ChatBot: supporting decision making in mobility data with chatbots

This paper presents an innovative chatbot architecture designed to support decision-making in the context of mobility data, leveraging recent advancements in Large Language Models (LLMs). As transportation systems and location-based services produce more data, understanding human mobility patterns and providing relevant insights becomes critical for effective decision-making. The chatbot aims to offer a user-friendly tool that allows users to interact with mobility datasets through natural language, preventing the end user from writing complex SQL queries and allowing them to create data visualization on the fly.

Enhancing Transfer Learning Network Classification Accuracy on Degraded Images by Network Structural Modification

Transfer learning approach is widely employed in various applications. However, this approach on image classification has its challenges, one of them is that its performance would deteriorate when training image quality is not perfect. In this study, structural modifications on the transfer learning network are applied to mitigate the negative impact caused by image degeneration. We modify the structure via several methods, namely fine-tuning the last layers of the pre-trained model, adding fully-connected layers, and reducing the depth of the pre-trained model. Upon examining the impact of these modifications when common image corruption occurs, we have found that both fine-tuning and reducing the depth of the network improves performance while adding fully-connected layers does not lead to a significantly positive impact. These empirical results have demonstrated to what extent structural modifications can compensate for the transfer learning network performance deterioration due to image corruption. The results help to further broaden the scope of the transfer learning application to degraded image classification.

Toward a Holistic Performance Evaluation of Large Language Models Across Diverse AI Accelerators

Scalable Continuous Integration using Remote Execution

Continuous integration at scale is a common problem for any large project. Multiple dimensions need to be optimized in order to provide a consistent experience for a large set of users. In particular, cost and performance are the typical ends of the spectrum that a CI maintainer needs to balance.

Adaptive Recursive Query Optimization

Unveiling the Potential of Natural Language Processing in Collaborative Robots (Cobots): A Comprehensive Survey

This paper delves into the integration of natural language processing (NLP) within the domain of cyber-physical systems, with a specific focus on collaborative robots (cobots), aiming to unravel the potential enhancements in communication, decision-making, and collaboration between humans and cobots. The primary objective is to conduct a comprehensive survey of NLP applications within the context of AI-assisted networking and systems, shedding light on how NLP can effectively elevate cobot capabilities, encompassing real-time decision-making, collaborative interactions, and training processes. The study meticulously categorizes the existing body of NLP research implemented on cobots, considering factors such as keywords, the number of published papers, and categorizations, allowing for a comprehensive assessment of their significance and impact.

Large Language Models As A Proxy For Human Evaluation In Assessing The Comprehensibility Of Disordered Speech Transcription.

ML-driven design of 3’ UTRs for mRNA stability

Using mRNA as a therapeutic has received enormous attention in the last few years, but instability of the molecule remains a hurdle to achieving long-lasting therapeutic levels of protein expression. In this study, we describe our approach for designing stable mRNA molecules by combining machine learning-driven sequence design with high-throughput experimental assays. We developed a high-throughput massively parallel reporter assay (MPRA) that, in a single experiment, measures the half-life of tens of thousands of unique mRNA sequences containing designed 3’ UTRs. Over multiple design-build-test iterations, we have accumulated mRNA stability measurements for 180,000 unique genomic and synthetic 3’ UTRs, representing the largest such dataset of sequences. We trained highly-accurate machine learning models to map from 3’ UTR sequence to mRNA stability, and used them to guide the design of synthetic 3’ UTRs that increase mRNA stability in cell lines. Finally, we validated the function of several ML-designed 3’ UTRs in mouse models, resulting in up to 2-fold more protein production over time and 30–100-fold higher protein output at later time points compared to a commonly used benchmark. These results highlight the potential of ML-driven sequence design for mRNA therapeutics.

Automated, Objective Speech and Language Markers of Longitudinal Changes in Psychosis Symptoms

Inferring the Evolutionary Model of Community-Structuring Traits with Convolutional Kitchen Sinks

Abstract When communities are assembled through processes such as filtering or limiting similarity acting on phylogenetically conserved traits, the evolutionary signature of those traits may be reflected in patterns of community membership. We show how the model of trait evolution underlying community-structuring traits can be inferred from community membership data using both a variation of a traditional eco-phylogenetic metric—the mean pairwise phylogenetic distance (MPD) between taxa—and a recent machine learning tool, Convolutional Kitchen Sinks (CKS). Both methods perform well across a range of phylogenetically informative evolutionary models, but CKS outperforms MPD as tree size increases. We demonstrate CKS by inferring the evolutionary history of freeze tolerance in angiosperms. Our analysis is consistent with a late burst model, suggesting freeze tolerance evolved recently. We suggest that multiple data types that are ordered on phylogenies, such as trait values, species interactions, or community presence/absence, are good candidates for CKS modeling because the generative models produce structured differences between neighboring points that CKS is well-suited for. We introduce the R package kitchen to perform CKS for generic application of the technique.

Artificial intelligence as a second reader for screening mammography

Artificial intelligence (AI) has shown promise in mammography interpretation, and its use as a second reader in breast cancer screening may reduce burden on healthcare systems. To evaluate the performance differences between routine double read and an AI as a second reader workflow (AISR) where the second reader is replaced with AI. A cohort of patients undergoing routine breast cancer screening at a single center with mammography was retrospectively collected between 2005 and 2021. A model developed on US and UK data was fine tuned on Japanese data. We subsequently performed a reader study with ten qualified readers with varied experience (five reader pairs), comparing routine double read to an AISR workflow. A ‘test set’ of 4059 women (mean age 56 ± 14 years; 157 positive, 3902 negative) was collected, with 278 (mean age 55 ± 13 years; 90 positive, 188 negative) evaluated for the reader study. We demonstrate an AUC=.84 (95%CI: 0.805-0.881), on the test set, with no significant difference to decisions made in clinical practice (p=.32). Compared with routine double reading, in the AISR arm sensitivity improved by 7.6% (95%CI: 3.80-11.4, p=.00004) and specificity decreased 3.4% (1.42-5.43, p=.0016), with 71% (212/298) of scans no longer requiring input from a second reader. Variation in recall decision between reader pairs improved from a Cohen’s kappa of κ=.65 (96% CI, .61-.68) to κ=.74 (96% CI, .71-.77) in the AISR arm. AISR improves sensitivity, reduces variability and decreases workload compared to routine dual screening.

The Most Disruptive Near-Term Use of AI in Cancer Care: Patient Empowerment Through Software Agents

Cancer care often involves making complex medical decisions within a challenging environment: a balkanized medical system of many specialists, information overload and obsolescence, limited time with doctors, and siloed data. New AI tools can enable patients and caregivers to more actively participate in treatment decisions. For example, consider receiving a complex scan report in a patient portal. While today, a patient may face confusion in interpreting a complex diagnostic report, a personalized generative AI agent could help by translating the scan report into language a patient can understand, and even contextualizing it within a patient’s personal health history and clinical evidence/guidelines. By providing an understandable version of the report and the clinical context of their test results, patients and their caregivers can engage more fully in decision-making with their oncology team. As described by Clayton Christensen’s “The Innovator’s Dilemma,” industry incumbents typically do not adopt disruptive technologies for fear of cannibalizing existing revenue streams (such as the case of Kodak and digital photography). This leads incumbents to serve their existing customers with the same value proposition, while ignoring “disruptive innovations” that offer a new value proposition to underserved customers (such as the transistor radio, which enabled teenagers to take their music with them). This theory predicts that institutional health care will focus on adopting AI for incremental operational improvements (e.g., patient scheduling, scan interpretation, claims processing). In this review, we argue that a positive disruptive impact of AI in oncology can come if AI-enabled software agents are used to support patients and caregivers in seeking better outcomes through personalized care. We review existing gaps and challenges that patients face as they go through receiving a cancer diagnosis, selecting a treatment plan, and then living with cancer. We use this pr

Prospective Multi-Site Validation of AI to Detect Tuberculosis and Chest X-Ray Abnormalities

BACKGROUND Using artificial intelligence (AI) to interpret chest X-rays (CXRs) could support accessible triage tests for active pulmonary tuberculosis (TB) in resource-constrained settings. METHODS The performance of two cloud-based CXR AI systems — one to detect TB and the other to detect CXR abnormalities — in a population with a high TB and human immunodeficiency virus (HIV) burden was evaluated. We recruited 1978 adults who had TB symptoms, were close contacts of known TB patients, or were newly diagnosed with HIV at three clinical sites. The TB-detecting AI (TB AI) scores were converted to binary using two thresholds: a high-sensitivity threshold and an exploratory threshold designed to resemble radiologist performance. Ten radiologists reviewed images for signs of TB, blinded to the reference standard. Primary analysis measured AI detection noninferiority to radiologist performance. Secondary analysis evaluated AI detection as compared with the World Health Organization (WHO) targets (90% sensitivity, 70% specificity). Both used an absolute margin of 5%. The abnormality-detecting AI (abnormality AI) was evaluated for noninferiority to a high-sensitivity target suitable for triaging (90% sensitivity, 50% specificity). RESULTS Of the 1910 patients analyzed, 1827 (96%) had conclusive TB status, of which 649 (36%) were HIV positive and 192 (11%) were TB positive. The TB AI’s sensitivity and specificity were 87% and 70%, respectively, at the high-sensitivity threshold and 78% and 82%, respectively, at the balanced threshold. Radiologists’ mean sensitivity was 76% and mean specificity was 82%. At the high-sensitivity threshold, the TB AI was noninferior to average radiologist sensitivity (P<0.001) but not to average radiologist specificity (P=0.99) and was higher than the WHO target for specificity but not sensitivity. At the balanced threshold, the TB AI was comparable to radiologists. The abnormality AI’s sensitivity and specificity were 97% and 79%, respectively,

Ordered Magnetic Fields around the 3C 84 Central Black Hole

The Persistent Shadow of the Supermassive Black Hole of M 87. I. Observations, Calibration, Imaging, and Analysis

Automated Measures of Syntactic Complexity in Natural Speech Production: Older and Younger Adults as a Case Study

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

Large language models (LLMs) such as Open AI's GPT-4 (which power ChatGPT) and Google's Gemini, built on artificial intelligence, hold immense potential to support, augment, or even eventually automate psychotherapy. Enthusiasm about such applications is mounting in the field as well as industry. These developments promise to address insufficient mental healthcare system capacity and scale individual access to personalized treatments. However, clinical psychology is an uncommonly high stakes application domain for AI systems, as responsible and evidence-based therapy requires nuanced expertise. This paper provides a roadmap for the ambitious yet responsible application of clinical LLMs in psychotherapy. First, a technical overview of clinical LLMs is presented. Second, the stages of integration of LLMs into psychotherapy are discussed while highlighting parallels to the development of autonomous vehicle technology. Third, potential applications of LLMs in clinical care, training, and research are discussed, highlighting areas of risk given the complex nature of psychotherapy. Fourth, recommendations for the responsible development and evaluation of clinical LLMs are provided, which include centering clinical science, involving robust interdisciplinary collaboration, and attending to issues like assessment, risk detection, transparency, and bias. Lastly, a vision is outlined for how LLMs might enable a new generation of studies of evidence-based interventions at scale, and how these studies may challenge assumptions about psychotherapy.

Rethinking machine unlearning for large language models

We explore machine unlearning in the domain of large language models (LLMs), referred to as LLM unlearning. This initiative aims to eliminate undesirable data influence (for example, sensitive or illegal information) and the associated model capabilities, while maintaining the integrity of essential knowledge generation and not affecting causally unrelated information. We envision LLM unlearning becoming a pivotal element in the life-cycle management of LLMs, potentially standing as an essential foundation for developing generative artificial intelligence that is not only safe, secure and trustworthy but also resource-efficient without the need for full retraining. We navigate the unlearning landscape in LLMs from conceptual formulation, methodologies, metrics and applications. In particular, we highlight the often-overlooked aspects of existing LLM unlearning research, for example, unlearning scope, data–model interaction and multifaceted efficacy assessment. We also draw connections between LLM unlearning and related areas such as model editing, influence functions, model explanation, adversarial training and reinforcement learning. Furthermore, we outline an effective assessment framework for LLM unlearning and explore its applications in copyright and privacy safeguards and sociotechnical harm reduction. Machine unlearning techniques remove undesirable data and associated model capabilities while preserving essential knowledge, so that machine learning models can be updated without costly retraining. Liu et al. review recent advances and opportunities in machine unlearning in LLMs, revisiting methodologies and overlooked principles for future improvements and exploring emerging applications in copyright and privacy safeguards and in reducing sociotechnical harms.

LangTest: A comprehensive evaluation library for custom LLM and NLP models

The use of natural language processing (NLP) models, including the more recent large language models (LLM) in real-world applications obtained relevant success in the past years. To measure the performance of these systems, traditional performance metrics such as accuracy, precision, recall, and f1-score are used. Although it is important to measure the performance of the models in those terms, natural language often requires an holistic evaluation that consider other important aspects such as robustness, bias, accuracy, toxicity, fairness, safety, efficiency, clinical relevance, security, representation, disinformation, political orientation, sensitivity, factuality, legal concerns, and vulnerabilities. To address the gap, we introduce LangTest , an open source Python toolkit, aimed at reshaping the evaluation of LLMs and NLP models in real-world applications. The project aims to empower data scientists, enabling them to meet high standards in the ever-evolving landscape of AI model development. Specifically, it provides a comprehensive suite of more than 60 test types, ensuring a more comprehensive understanding of a model's behavior and responsible AI use. In this experiment, a Named Entity Recognition (NER) clinical model showed significant improvement in its capabilities to identify clinical entities in text after applying data augmentation for robustness.

Light-HIDRA: Scalable and decentralized resource orchestration in Fog-IoT environments

30 Single-cell DNA sequencing for transgene copy number in gene therapy for Artemis-deficient severe combined immunodeficiency

Human-AI coevolution

Correction: AI content detection in the emerging information ecosystem: new obligations for media and tech companies

AI content detection in the emerging information ecosystem: new obligations for media and tech companies

The Curve Fitting Problem, Data Validation, and Inductive Generalization in Machine Learning

Aris Spanos and Deborah Mayo’s error-statistical approach to statistical modeling and inference adopts the reliability of inductive inference as a primary criterion for statistical model and estimator selection (e.g., curve fitting). In this paper, we expand the error-statistical approach’s adoption of reliable inductive inference by scrutinizing the epistemic legitimacy of contemporary techniques leveraged in data science. We argue that data validation and testing potentially provides a direct, measurable method of evaluating evidence for reliable inductive inferences in cases where the error-statistical approach is not easily applied, and conclude with an exploration of core methodological foils to the reliability of inductive inference revealed by this argument.

Optimizing B-Trees for Memory-Constrained Flash Embedded Devices.

Subset-Optimized BLS Multi-signature with Key Aggregation

FARSum: Feedback-Aware Abstractive Product Review Summarization