Sessions “Papers, please”
Il s’agit de sessions de partage d’articles pour les doctorants de PIRAT qui sont ouvertes à tous les membres de l’équipe. L’évènement a lieu au 5e étage de CS. Pour toute demande d’information, contacter pierre-francois.gimenez@inria.fr.
14/11/24 at 15:30: “PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing“, presented by Sébastien Kilian
Abstract: Penetration testing, a crucial industrial practice for ensuring system security, has traditionally resisted automation due to the extensive expertise required by human professionals. Large Language Models (LLMs) have shown significant advancements in various domains, and their emergent abilities suggest their potential to revolutionize industries. In this work, we establish a comprehensive benchmark using real-world penetration testing targets and further use it to explore the capabilities of LLMs in this domain. Our findings reveal that while LLMs demonstrate proficiency in specific sub-tasks within the penetration testing process, such as using testing tools, interpreting outputs, and proposing subsequent actions, they also encounter difficulties maintaining a whole context of the overall testing scenario. Based on these insights, we introduce PENTESTGPT, an LLM-empowered automated penetration testing framework that leverages the abundant domain knowledge inherent in LLMs. PENTESTGPT is meticulously designed with three self-interacting modules, each addressing individual sub-tasks of penetration testing, to mitigate the challenges related to context loss. Our evaluation shows that PENTESTGPT not only outperforms LLMs with a task-completion increase of 228.6% compared to the GPT-3.5 model among the benchmark targets, but also proves effective in tackling real-world penetration testing targets and CTF challenges. Having been open-sourced on GitHub, PENTESTGPT has garnered over 6,500 stars in 12 months and fostered active community engagement, attesting to its value and impact in both the academic and industrial spheres.
05/12/24 at 15:30: Natan Talon
09/01/25 at 15:30: Manuel Poisson
23/01/25 at 15:30: Patrick Zounon
06/02/25 at 15:30: Lucas Aubard
20/02/25 at 15:30: Pierre-François Gimenez
06/03/25 at 15:30: Lionel Hemmerlé
Past sessions
31/10/24 at 15:30: “Advanced Persistent Threat Attack Detection Systems: A Review of Approaches, Challenges, and Trends“, presented by Fanny Dijoud
Abstract: Advanced persistent threat (APT) attacks present a significant challenge for any organization, as they are difficult to detect due to their elusive nature and characteristics. In this paper, we conduct a comprehensive literature review to investigate the various APT attack detection systems and approaches and classify them based on their threat model and detection method. Our findings reveal common obstacles in APT attack detection, such as correctly attributing anomalous behavior to APT attack activities, limited availability of public datasets and inadequate evaluation methods, challenges with detection procedures, and misinterpretation of requirements. Based on our findings, we propose a reference architecture to enhance the comparability of existing systems and provide a framework for classifying detection systems. In addition, we look in detail at the problems encountered in current evaluations and other scientific gaps, such as a neglected consideration of integrating the systems into existing security architectures and their adaptability and durability. While no one-size-fits-all solution exists for APT attack detection, this review shows that graph-based approaches hold promising potential. However, further research is required for real-world usability, considering the systems’ adaptability and explainability.
17/10/24 at 15:30: “Benchopt: Reproducible, efficient and collaborative optimization benchmarks“, presented by Matthieu Mouzaoui
Abstract: Numerical validation is at the core of machine learning research as it allows to assess the actual impact of new methods, and to confirm the agreement between theory and practice. Yet, the rapid development of the field poses several challenges: researchers are confronted with a profusion of methods to compare, limited transparency and consensus on best practices, as well as tedious re-implementation work. As a result, validation is often very partial, which can lead to wrong conclusions that slow down the progress of research. We propose Benchopt, a collaborative framework to automate, reproduce and publish optimization benchmarks in machine learning across programming languages and hardware architectures. Benchopt simplifies benchmarking for the community by providing an off-the-shelf tool for running, sharing and extending experiments. To demonstrate its broad usability, we showcase benchmarks on three standard learning tasks: l2-regularized logistic regression, Lasso, and ResNet18 training for image classification. These benchmarks highlight key practical findings that give a more nuanced view of the state-of-the-art for these problems, showing that for practical evaluation, the devil is in the details. We hope that Benchopt will foster collaborative work in the community hence improving the reproducibility of research findings.
13/06/24 at 15:30: “The Ransomware-as-a-Service economy within the darknet“, presented by Anass Belarbi
Abstract: Ransomware is an epidemic that adversely affects the lives of both individuals and large companies, where criminals demand payments to release infected digital assets. In the wake of the ransomware success, Ransomware-as-a-Service (RaaS) has become a franchise offered through darknet marketplaces, allowing aspiring cybercriminals to take part in this dubious economy. We have studied contemporary darknet markets and forums over a period of two years using a netnographic research approach. Our findings show that RaaS currently seems like a modest threat relative to popular opinion. Compared to other types of illegal digital goods, there are rather few RaaS items offered for sale in darknet marketplaces, often with questionable authenticity. From our data we have created a value chain and descriptions of the actors involved in this economy.
30/05/24 at 14:00: “Cyber Digital Twin Simulator for Automatic Gathering and Prioritization of Security Controls’ Requirements“, presented by Manuel Poisson
Abstract: The scale and complexity of cyber threats in digital enterprises hamper operators’ ability to gather, prioritize and rationalize which security controls requirements should be handled first to achieve rapid risk reduction. This paper presents a cyber digital twin, based on attack graph analytics, that automatically gathers and prioritizes security controls requirements at scale over active networks. The first-of-a-kind twin collects information about the computer network, associates it with attack tactics, measures the efficiency of implemented security controls requirements and automatically detects missing security controls. The twin also evaluates a cyber risk value using the attack graph and proposes prioritization of the detected requirements to rapidly reduce risk within existing system constraints. The cyber digital twin simulator offers several new risk reduction methods for automatically selecting security controls requirements. The necessary data for constructing a contextual cyber digital twin is defined, including the relationship between security controls and attack tactics. The paper illustrates the calculations used for ranking security controls’ risk impact, the algorithm for security controls’ requirements prioritization, and finally demonstrates successful resultsusing a field experiment conducted via an active network.
16/05/24 at 15:30: “Hybrid cyber defense strategies using Honey-X: A survey“, presented by Yohann Morel
Abstract: The development and adoption of network technologies reshape our daily life; however, network-connected devices have become popular targets of cybercrimes. Honey-X-based cyber defense technologies, like honeypots, honeynets, and honeytokens, can provide cyber threat intelligence and protect network users against various cyberattacks by leveraging deception techniques. The hybrid defense strategies of defensive cyber deception (DCD) and moving target defense (MTD) approaches while combining the advances of Machine Learning (ML) or Game Theory (GT), which have the potential to offer cyber security enhancement against malicious adversaries. This survey paper aims to comprehensively understand the defensive approaches based on honey-X for cyber defense. It analyses and categorizes the honey-X-based defense strategies, presents the security quantification of cyber defense, and outlines challenges and future directions for honey-X-based cyber defense strategies.
17/04/24 at 10:30: “The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems“, presented by Fabien Pesquerel
Abstract: Reinforcement learning can provide a robust and natural means for agents to learn how to coordinate their action choices in multi agent systems. We examine some of the factors that can influence the dynamics of the learning process in such a setting. We first distinguish reinforcement learners that are unaware of (or ignore) the presence of other agents from those that explicitly attempt to learn the value of joint actions and the strategies of their counterparts. We study (a simple form of) Q-leaming in cooperative multi agent systems under these two perspectives, focusing on the influence of that game structure and exploration strategies on convergence to (optimal and suboptimal) Nash equilibria. We then propose alternative optimistic exploration strategies that increase the likelihood of convergence to an optimal equilibrium.
28/03/24 at 15:30: “On Autonomous Agents in a Cyber Defence Environment“, presented by Pierre Lledo
Abstract: Autonomous Cyber Defence is required to respond to high-tempo cyber-attacks. To facilitate the research in this challenging area, we explore the utility of the autonomous cyber operation environments presented as part of the Cyber Autonomy Gym for Experimentation (CAGE) Challenges, with a specific focus on CAGE Challenge 2. CAGE Challenge 2 required a defensive Blue agent to defend a network from an attacking Red agent. We provide a detailed description of the this challenge and describe the approaches taken by challenge participants. From the submitted agents, we identify four classes of algorithms, namely, Single- Agent Deep Reinforcement Learning (DRL), Hierarchical DRL, Ensembles, and Non-DRL approaches. Of these classes, we found that the hierarchical DRL approach was the most capable of learning an effective cyber defensive strategy. Our analysis of the agent policies identified that different algorithms within the same class produced diverse strategies and that the strategy used by the defensive Blue agent varied depending on the strategy used by the offensive Red agent. We conclude that DRL algorithms are a suitable candidate for autonomous cyber defence applications.
14/03/24 at 15:30: “Generating Practical Adversarial Network Traffic Flows Using NIDSGAN“, presented by Matthieu Mouzaoui
Abstract: Network intrusion detection systems (NIDS) are an essential defense for computer networks and the hosts within them. Machine learning (ML) nowadays predominantly serves as the basis for NIDS decision making, where models are tuned to reduce false alarms, increase detection rates, and detect known and unknown attacks. At the same time, ML models have been found to be vulnerable to adversarial examples that undermine the downstream task. In this work, we ask the practical question of whether real-world ML-based NIDS can be circumvented by crafted adversarial flows, and if so, how can they be created. We develop the generative adversarial network (GAN)-based attack algorithm NIDSGAN and evaluate its effectiveness against realistic ML-based NIDS. Two main challenges arise for generating adversarial network traffic flows: (1) the network features must obey the constraints of the domain (i.e., represent realistic network behavior), and (2) the adversary must learn the decision behavior of the target NIDS without knowing its model internals (e.g., architecture and meta-parameters) and training data. Despite these challenges, the NIDSGAN algorithm generates highly realistic adversarial traffic flows that evade ML-based NIDS. We evaluate our attack algorithm against two state-of-the-art DNN-based NIDS in whitebox, blackbox, and restricted-blackbox threat models and achieve success rates which are on average 99%, 85%, and 70%, respectively. We also show that our attack algorithm can evade NIDS based on classical ML models including logistic regression, SVM, decision trees and KNNs, with a success rate of 70% on average. Our results demonstrate that deploying ML-based NIDS without careful defensive strategies against adversarial flows may (and arguably likely will) lead to future compromises.
15/02/24 at 15:30: “Prime and Prejudice: Primality Testing Under Adversarial Conditions”, presented by Sébastien Kilian
Abtract: This work provides a systematic analysis of primality testing under adversarial conditions, where the numbers being tested for primality are not generated randomly, but instead provided by a possibly malicious party. Such a situation can arise in secure messaging protocols where a server supplies Diffie-Hellman parameters to the peers, or in a secure communications protocol like TLS where a developer can insert such a number to be able to later passively spy on client-server data. We study a broad range of cryptographic libraries and assess their performance in this adversarial setting. As examples of our findings, we are able to construct 2048-bit composites that are declared prime with probability (1/16) by OpenSSL’s primality testing in its default configuration; the advertised performance is (2-80). We can also construct 1024-bit composites that always pass the primality testing routine in GNU GMP when configured with the recommended minimum number of rounds. And, for a number of libraries (Cryptlib, LibTomCrypt, JavaScript Big Number, WolfSSL), we can construct composites that always pass the supplied primality tests. We explore the implications of these security failures in applications, focusing on the construction of malicious Diffie-Hellman parameters. We show that, unless careful primality testing is performed, an adversary can supply parameters (p,q,g) which on the surface look secure, but where the discrete logarithm problem in the subgroup of order q generated by g is easy. We close by making recommendations for users and developers. In particular, we promote the Baillie-PSW primality test which is both efficient and conjectured to be robust even in the adversarial setting for numbers up to a few thousand bits.
02/02/24 at 15:30: Tuto, please: Typst, an Alternative to Latex, par Jean-Marie Mineau
Typst is a new markup-based typesetting system that is designed to be as powerful as LaTeX while being much easier to learn and use
18/01/24 at 15:30: “Divergence-aware federated self-supervised learning” presented by Hélène Orsini
Abstract: Self-supervised learning (SSL) is capable of learning remarkable representations from centrally available data. Recent works further implement federated learning with SSL to learn from rapidly growing decentralized unlabeled images (e.g., from cameras and phones), often resulted from privacy constraints. Extensive attention has been paid to SSL approaches based on Siamese networks. However, such an effort has not yet revealed deep insights into various fundamental building blocks for the federated self-supervised learning (FedSSL) architecture. We aim to fill in this gap via in-depth empirical study and propose a new method to tackle the non-independently and identically distributed (non-IID) data problem of decentralized data. Firstly, we introduce a generalized FedSSL framework that embraces existing SSL methods based on Siamese networks and presents flexibility catering to future methods. In this framework, a server coordinates multiple clients to conduct SSL training and periodically updates local models of clients with the aggregated global model. Using the framework, our study uncovers unique insights of FedSSL: 1) stop-gradient operation, previously reported to be essential, is not always necessary in FedSSL; 2) retaining local knowledge of clients in FedSSL is particularly beneficial for non-IID data. Inspired by the insights, we then propose a new approach for model update, Federated Divergence-aware Exponential Moving Average update (FedEMA). FedEMA updates local models of clients adaptively using EMA of the global model, where the decay rate is dynamically measured by model divergence. Extensive experiments demonstrate that FedEMA outperforms existing methods by 3-4% on linear evaluation. We hope that this work will provide useful insights for future research.
11/01/24 at 15:30: “ILAB: An Interactive Labelling Strategy for Intrusion Detection” presented by Maxime Lanvin
Abstract: Acquiring a representative labelled dataset is a hurdle that has to be overcome to learn a supervised detection model. Labelling a dataset is particularly expensive in computer security as expert knowledge is required to perform the annotations. In this paper, we introduce ILAB, a novel interactive labelling strategy that helps experts label large datasets for intrusion detection with a reduced workload. First, we compare ILAB with two state-of-the-art labelling strategies on public labelled datasets and demonstrate it is both an effective and a scalable solution. Second, we show ILAB is workable with a real-world annotation project carried out on a large unlabelled NetFlow dataset originating from a production environment. We provide an open source implementation (https://github.com/ANSSI-FR/SecuML/) to allow security experts to label their own datasets and researchers to compare labelling strategies.
