Introduction

Building upon our foundation in classical machine learning techniques, which often rely on manually crafted feature vectors or descriptors derived from material composition and structure, we now venture into more sophisticated methodologies. While methods like linear regression, support vector machines, and even standard neural networks have proven valuable, they can struggle to fully leverage the intricate relationships inherent in atomic structures or to generate entirely novel material candidates. This chapter introduces two powerful classes of advanced machine learning models that directly address these challenges: Graph Neural Networks (GNNs), which excel at learning from structured data like atomic arrangements, and Diffusion Models, a state-of-the-art approach for generative tasks, including the design of new materials.

Limitations of Traditional ML and the Need for Structure-Aware Models¶

Recall from previous lectures that materials properties are fundamentally determined by the type of atoms present and their specific spatial arrangement, including bonding, coordination environments, and crystal symmetry. Classical machine learning approaches often represent a material using a fixed-length vector of features. These features might include stoichiometric information, average electronegativity, packing density, or perhaps statistics derived from diffraction patterns or radial distribution functions.

While useful, this “flattening” of complex structural information into a feature vector can lead to information loss. Critically, standard models like neural networks are generally not permutation invariant. This means that arbitrarily re-ordering the atoms in the input representation (e.g., changing their index in a list) would typically change the output prediction, which is physically nonsensical – the properties of a crystal do not depend on how we choose to label its constituent atoms. Furthermore, these models require fixed-size inputs, posing challenges when dealing with systems containing varying numbers of atoms (e.g., different unit cells, defects, or interfaces).

To overcome these limitations, we need machine learning frameworks capable of directly processing and learning from the graph-like nature of atomic structures, recognizing that atoms (nodes) are connected by bonds or proximity relationships (edges). Similarly, for the ambitious goal of inverse design – generating novel materials with targeted properties – we require generative models sophisticated enough to propose chemically plausible and structurally realistic candidates.