{"id":8838,"date":"2024-01-19T15:34:33","date_gmt":"2024-01-19T14:34:33","guid":{"rendered":"https:\/\/sites.unica.it\/pralab\/rexlearn\/"},"modified":"2025-01-08T11:20:36","modified_gmt":"2025-01-08T10:20:36","slug":"trustml","status":"publish","type":"page","link":"https:\/\/sites.unica.it\/pralab\/trustml\/","title":{"rendered":"TrustML Project"},"content":{"rendered":"\n

TrustML project <\/strong><\/p>\n\n\n\n

<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n
<\/div>\n\n\n\n
\n
\n
<\/div><\/div>\n\n\n\n
\n
\n

Project coordinator<\/strong>
Prof. Battista Biggio<\/a>
PRA Lab – University of Cagliari<\/mark><\/p>\n<\/div>\n<\/div><\/div>\n<\/div>\n\n\n\n

\n
\n
<\/div><\/div>\n<\/div><\/div>\n\n\n\n

Duration<\/strong>
25 Months
September 2022 – October 2024<\/mark><\/p>\n<\/div>\n\n\n\n

\n
<\/div><\/div>\n\n\n\n

Budget<\/strong>
Total budget: 52777.78 \u20ac
FDS funded budget: 52777.78 \u20ac<\/mark><\/mark><\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
\n


People involved<\/mark><\/strong><\/p>\n\n\n\n

Prof. Giorgio Giacinto<\/a>
Prof. Giorgio Fumera<\/a>
Prof. Luca Didaci<\/a>
Prof. Matteo Fraschini<\/p>\n<\/div>\n\n\n\n

\n


Funding<\/mark><\/strong><\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

<\/p>\n<\/div>\n<\/div>\n\n\n\n

<\/div>\n\n\n\n

Project context<\/strong> and challenges<\/strong><\/p>\n\n\n\n

\n
\n

Machine learning<\/strong> (ML) and Artificial intelligence<\/strong> (AI) have recorded unprecedented success in different applications, including computer vision, speech recognition, and natural language processing. <\/p>\n\n\n\n

Despite providing accurate predictions, such models suffer from several limitations which prevent their applicability in security-sensitive and safety-critical domains<\/strong>. <\/p>\n\n\n\n

In this project, we aim to address three main challenges that are hindering the development of trustworthy AI\/ML<\/strong> models in these domains.<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure><\/div><\/div>\n<\/div>\n\n\n\n
<\/div>\n\n\n\n
\n
\n
\"\"<\/figure>
\n


1. The challenge of adversarial robustness

2. The challenge of model explainability

3. The challenge of uncertainty estimation<\/p>\n<\/div><\/div>\n<\/div>\n<\/div>\n\n\n\n

<\/div>\n\n\n\n

The first is the challenge of adversarial robustness.<\/strong><\/em> AI\/ML models have been shown to be vulnerable to adversarial examples, i.e., carefully optimized perturbations added to the input images, text, or audio. Evaluating robustness in other domains, like malware detection, remains an open challenge due to the lack of a proper formalization of the feasible, structural, and more complex input manipulations specific to such domains.
The second is the challenge of model explainability<\/strong><\/em>. AI\/ML models output predictions which are hardly interpretable; it is difficult to identify which features of the input samples contribute to predicting them within a given class, or which training samples support such a decision. Explainability is a desirable property not only to build users\u2019 trust in AI\/ML, but also to understand what models learn, enabling detection and mitigation of potential dataset\/model biases.
The third challenge is about evaluating uncertainty<\/strong><\/em>. It arises from the fact that AI\/ML models can output highly confident predictions even when the input data is outside the support of the training data distribution, as in the case of adversarial examples.<\/p>\n\n\n\n

These problems have been addressed by the development of appropriate scientific methodologies allowing a significant advancement of the state of the art<\/strong>. The methodological advancement concerned:<\/p>\n\n\n\n

– the development of techniques for evaluating the robustness\/safety of machine learning<\/strong> algorithms with respect to inputs disrupted to compromise their decisions, known as adversarial examples<\/strong>;
– the development of techniques for improving the robustness of algorithms<\/strong> against these inputs.
– the development of techniques to improve<\/strong> the interpretability of the decisions provided by these algorithms<\/strong>. <\/p>\n\n\n\n

A large collection of datasets<\/strong> for the experimentation on use cases<\/strong> was performed, followed by the development and testing of prototype systems <\/strong>on applications of interest, which included image recognition<\/strong> and automatic detection of computer viruses<\/strong>.<\/p>\n\n\n\n

<\/div>\n\n\n\n


Project results<\/strong> and impact<\/strong><\/p>\n\n\n\n

The project produced significant tangible results, including (i) scientific publications published in international journals and in the proceedings of conferences among the most important in the field of machine learning and artificial intelligence; (ii) the release of benchmark datasets and associated experimental prototypes, through the production of various projects with open code source; (iii) an intense dissemination and communication activity towards stakeholders and the scientific community.
The project has also fostered numerous scientific collaborations at national and international level<\/strong>, such as demonstrated by the number of publications with co-authors external to the project unit team. This led to the consolidation and formalization of agreements for carrying out some joint activities with companies and international institutions.<\/p>\n\n\n\n

<\/p>\n\n\n\n

\n
Publications<\/summary>
\n
    \n
  1. G. Floris, R. Mura, L. Scionis, G. Piras, M. Pintor, A. Demontis, and B. Biggio. Improving fast minimum-norm attacks with hyperparameter optimization. In ESANN, 2023.<\/li>\n\n\n\n
  2. R. Mura, G. Floris, L. Scionis, G. Piras, M. Pintor, A. Demontis, G. Giacinto, B. Biggio, and F. Roli. HO-FMN: Hyperparameter optimization for fast minimum-norm attacks. Neurocomputing, 616:128918, 2025.<\/li>\n\n\n\n
  3. A. E. Cin\u00e0, K. Grosse, A. Demontis, S. Vascon, W. Zellinger, B. A. Moser, A. Oprea, B. Biggio, M. Pelillo, and F. Roli. Wild patterns reloaded: A survey of machine learning security against training data poisoning. ACM Comput. Surv., 55(13s):294:1\u2013294:39, jul 2023.<\/li>\n\n\n\n
  4. K. Grosse, L. Bieringer, T. R. Besold, B. Biggio, and K. Krombholz. Machine learning security in industry: A quantitative survey. IEEE Transactions on Information Forensics and Security, 18:1749\u20131762, 2023.<\/li>\n\n\n\n
  5. H. Eghbal-Zadeh, W. Zellinger, M. Pintor, K. Grosse, K. Koutini, B. A. Moser, B. Biggio, and G. Widmer. Rethinking data augmentation for adversarial robustness. Information Sciences, 654:119838, 2024.<\/li>\n\n\n\n
  6. M. Pintor, L. Demetrio, A. Sotgiu, H.-Y. Lin, C. Fang, A. Demontis, and B. Biggio. Detecting attacks against deep reinforcement learning for autonomous driving. In International Conference on Machine Learning and Cybernetics, ICMLC. IEEE SMC, 2023.<\/li>\n\n\n\n
  7. B. Montaruli, L. Demetrio, M. Pintor, L. Compagna, D. Balzarotti, and B. Biggio. Raze to the ground: Query-efficient adversarial HTML attacks on machine-learning phishing webpage detectors. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, AISec \u201923, pages 233\u2013244, New York, NY, USA, 2023. ACM.<\/li>\n\n\n\n
  8. D. Trizna, L. Demetrio, B. Biggio, and F. Roli. Nebula: Self-attention for dynamic malware analysis. IEEE Transactions on Information Forensics and Security, 19:6155\u20136167, 2024.<\/li>\n\n\n\n
  9. L. Minnei, H. Eddoubi, A. Sotgiu, M. Pintor, A. Demontis, and B. Biggio. Data drift in android malware detection. In International Conference on Machine Learning and Cybernetics, ICMLC. IEEE, 2024 (in press).<\/li>\n\n\n\n
  10. W. Guo, A. Demontis, M. Pintor, P. P. K. Chan, and B. Biggio. LFPD: Local-feature-powered defense against adaptive backdoor attacks. In International Conference on Machine Learning and Cybernetics, ICMLC. IEEE, 2024 (in press).<\/li>\n\n\n\n
  11. E. Ledda, D. Angioni, G. Piras, G. Fumera, B. Biggio, and F. Roli. Adversarial attacks against uncertainty quantification. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV) Workshops, pages 4599\u20134608, October 2023.<\/li>\n\n\n\n
  12. G. Faa, M. Castagnola, L. Didaci, F. Coghe, M. Scartozzi, L. Saba, and M. Fraschini. The Quest for the Application of Artificial Intelligence to Whole Slide Imaging: Unique Prospective from New Advanced Tools. Algorithms. 2024; 17(6):254. <\/li>\n\n\n\n
  13. M. Fraschini, M. Castagnola, L. Barberini, R. Sanfilippo, F. Coghe, L. Didaci, R. Cau, C. Frongia, M. Scartozzi, L. Saba, and G. Faa. An Unsupervised Learning Tool for Plaque Tissue Characterization in Histopathological Images. Sensors. 2024; 24(16), 5383. <\/li>\n\n\n\n
  14. L. Didaci, S. M. Pani, C. Frongia, and M. Fraschini. How Time Window Influences Biometrics Performance: An EEG-Based Fingerprint Connectivity Study. Signals 2024; 5, 597-604. <\/li>\n\n\n\n
  15. Silvia Lucia Sanna, Diego Soi, Davide Maiorca, Giorgio Fumera, Giorgio Giacinto, A risk estimation study of native code vulnerabilities in Android applications, Journal of Cybersecurity<\/em>, Volume 10, Issue 1, 2024, tyae015.<\/li>\n<\/ol>\n\n\n\n

    Preprints (under review)<\/strong><\/p>\n\n\n\n

      \n
    1. D. Gibert, L. Demetrio, G. Zizzo, Q. Le, J. Planes, and B. Biggio. Certified adversarial robustness of machine learning-based malware detectors via (de)randomized smoothing. arXiv e-prints, 2024.<\/li>\n\n\n\n
    2. L. Demetrio and B. Biggio. secml-malware: Pentesting Windows malware classifiers with adversarial EXEmples in Python. arXiv e-prints, 2024.<\/li>\n\n\n\n
    3. A. E. Cin\u00e0, F. Villani, M. Pintor, L. Sch\u00f6nherr, B. Biggio, and M. Pelillo. \u03c3-zero: Gradient-based optimization of L0-norm adversarial examples. ArXiv e-prints, 2024.<\/li>\n\n\n\n
    4. G. Piras, M. Pintor, A. Demontis, B. Biggio, G. Giacinto, and F. Roli. Adversarial pruning: A survey and benchmark of pruning methods for adversarial robustness. arXiv e-prints, 2024.<\/li>\n\n\n\n
    5. E. Ledda, G. Scodeller, D. Angioni, G. Piras, A. E. Cin\u00e0, G. Fumera, B. Biggio, F. Roli. On the Robustness of Adversarial Training Against Uncertainty Attacks. arXiv e-prints, 2024.<\/li>\n<\/ol>\n\n\n\n

      <\/p>\n<\/div><\/details><\/div>\n<\/div>\n\n\n\n

      \n
      Open source software & Prototypes<\/summary>
      \n