Agenda Day 1: Trustworthiness of AI

Abstract:
In this talk, Prof. Herrera addresses the critical interplay between reliance on artificial intelligence (AI) and human intuition in AI-assisted decision-making processes. Explainable AI (XAI) is discussed as a means to bridge the gap between complex AI algorithms and human understanding. The presentation focuses on the role of intuition in interpreting AI outputs and making informed decisions. XAI ensures that AI-assisted decision-making processes are understandable, fostering reliance on and trust in AI systems. It also requires increasing individuals' cognitive motivation to engage analytically with the explanations provided. Thus, a key challenge is to develop XAI approaches that are useful for experts, ultimately enhancing human-AI collaboration and improving decision outcomes.

Abstract:
In this talk, Prof. Lopez de Prado will explain why investment efficiency cannot be achieved through associational or observational studies, emphasizing that the estimation of optimal portfolios requires causal analysis. The transition from associational to causal inference will necessitate a redesign of the quantitative protocols used in investing.

Abstract:
Learning from data may be aimed at description, prediction, or counterfactual prediction. The contrast of two or more counterfactual predictions, or causal inference, is the type of leaning from data that guides decision making. This talk focuses on causal inference for disease treatment and prevention based on healthcare databases. Specifically, the talk dissects the components of causal inference and proposes a framework to discuss causal AI technologies. An open question is whether AI technologies will soon be ready to make causal inferences from healthcare data that are as trustworthy as, or more trustworthy than, those made by human investigators.

Abstract:
As Large Language Models (LLMs) continue to be integrated into healthcare, evaluating their trustworthiness becomes essential, particularly in clinical settings where safety and explainability are critical. In this talk, Shadab will present the MEDIC framework, which provides a comprehensive evaluation of LLMs across five key dimensions: medical reasoning, ethics and bias, data and language understanding, in-context learning, and clinical safety. This framework allows for a systematic assessment of models, bridging the gap between theoretical potential and practical implementation expectations, and provides a necessary complementary approach to the real-world evaluation of LLMs. By examining case studies of LLM use in clinical decision support and medical documentation, he will discuss how the MEDIC framework ensures that LLMs meet the high standards of trust required in healthcare. The talk will also explore performance trade-offs and highlight areas for further work in building safer and more reliable LLMs for clinical applications.

Abstract:
Given the need for data, computational cost and technical expertise required to train machine learning models, users outsource the task of learning to AI expert companies. Prof. Goldwasser will discuss adversarial settings where outsourcing computations encounters unforeseen problems throughout the Machine Learning pipeline including risks to robustness, privacy and even correctness and discuss ways to mitigate them.

Abstract:
The recent NIST Zero Trust Architecture (publication SP 800-207) advocates a new cybersecurity paradigm that moves defenses from static, network-based perimeters to focus on users, assets, and resources. These resources include Enterprise environment data, compute capabilities, and AI-based decision-making algorithms. However, the Enterprise sensors and network elements that collect and analyze environment data are themselves subject to cyberattacks, leading to possible injection of false data into the data-stream used by AI-based systems that determine crucial day-to-day functions, such as employee access control to sensitive corporate information, workload migration across clouds, and others. In this presentation we discuss the notion of strong roots-of-trust based on the utilization of trusted hardware, software bill of materials (SBOM), vendor endorsements and the ecosystem needed to verify the state of the sensors and network elements in the Enterprise.

Abstract:
Privacy Preserving / Enhancing Technologies (PETs) are key, given the advancement and rise of Generative AI. Beyond traditional security methods and anonymization techniques, privacy for AI seems to be shifting towards novel technologies such as Fully Homomorphic Encryption (FHE) and secure Multi-Party Computation (MPC). This talk addresses TII’s work on PETs, the use of MPC and FHE for AI applications, and the new challenges and trade-offs related to their usage including considerations for Verifiable Computation, pros and cons of MPC interactive nature, performance costs, etc. It also covers specific attacks on Machine Learning (ML) models, e.g., model extraction and poisoning, training data extraction, amongst others, even when PETs are applied. A part of this talk goes over TII’s work related to the cryptanalytic problem of model extraction whereby the weights and architecture of a DNN are extracted or inferred using side-channel techniques and/or black box oracle access to a Deep Neural Network (DNN).

Abstract:
The size of current-day machine learning models has been growing by leaps and bounds. A lot of data continues to remain distributed across siloed organizations and smart edge devices. These distributed entities are steeply challenged in their ability to collaborate and engage with large models due to constraints of resource efficiency and privacy. This talk presents several variants of federated learning that incorporate a greater level of model parallelism, while ensuring privacy guarantees over the communicated payloads to cater to these demands.

Abstract:
In this talk, Dr. Kairouz will explore how Federated Learning (FL), a technique that allows multiple entities to collaborate on training machine learning models without sharing local data, has evolved over the past decade. FL has already been deployed at scale, with production systems from major organizations like Google, Apple, and Meta showing its real-world impact. However, key challenges remain — such as verifying server-side differential privacy (DP) guarantees and handling the complexity of coordinating training across millions of heterogeneous and resource-constrained devices — which still hinder broader adoption. Additionally, emerging trends such as large (multi-modal) models and blurred lines between training, inference, and personalization challenge traditional FL frameworks. In response to these challenges, Dr. Kairouz will propose a redefined approach to FL that emphasizes privacy principles over rigid definitions and discuss how trusted execution environments and open-source ecosystems can address these challenges.

Abstract:
This talk will focus on the opportunities and challenges of decentralized data in collaborative AI systems, especially when working with untrusted clients. Samuel Horvath will explore how federated learning can become more resilient through techniques that ensure Byzantine robustness, enabling reliable performance even in the presence of faulty or malicious clients. Additionally, the talk will cover how personalization can improve model adaptation to individual users while also enhancing overall robustness. Through these strategies, Samuel will propose methods to strengthen federated learning systems in decentralized environments, where collaboration and trust are not always guaranteed.