Formats

In materials informatics, data can be stored in various formats depending on the type of data, its structure, and the intended use. Different formats have different advantages and are suited to different types of data and applications. Common formats include CSV (Comma-Separated Values) and JSON (JavaScript Object Notation), among others. These formats are used to store and exchange data in a structured and organized manner. Here are some common formats used in materials informatics:

Human-Readable Formats¶

Human-readable formats are designed to be easily read and understood by humans. These formats are typically text-based and can be opened and viewed using a text editor or spreadsheet program. Human-readable formats are commonly used for sharing data between researchers.

CSV (Comma-Separated Values)¶

CSV files store data in plain text, where each line represents a record and fields within the record are separated by delimiters such as commas, tabs, or spaces. This format is simple and easy to read, but it can be inefficient when the database is large. For example, a CSV file might look like this:

ID,Name,Department,Salary
1,John Doe,Engineering,75000
2,Jane Smith,Marketing,68000
3,Alice Johnson,HR,72000

In this example, each line represents an employee record, with fields for ID, Name, Department, and Salary separated by commas.

JSON (JavaScript Object Notation)¶

JSON (JavaScript Object Notation) is a lightweight, standard text-based format for representing structured data. It follows a hierarchical structure of key-value pairs, making it easy to represent complex data structures. JSON is widely used in application programming interfaces (APIs) for data exchange. For example, a JSON object might look like this:

{
    "crystal_structure": {
        "atomic_positions": [
            {
                "wyckoff": "8a",
                "element": "Si",
                "x": "3/4",
                "y": "3/4",
                "z": "1/4"
            }
        ],
        "symmetry": {
            "crystal_system": "Cubic",
            "lattice_system": "Cubic",
            "hall_number": "F 4d 2 3 -1d",
            "international_number": 227,
            "symbol": "Fd3̅m",
            "point_group": "m3̅m"
        },
        "structure_meta": {
            "number_of_atoms": 8,
            "density": "2.31 g·cm⁻³",
            "dimensionality": "",
            "possible_oxidation_states": "Si⁰⁺"
        }
    }
}

In this example, the JSON object contains detailed information about the crystal structure of silicon, including atomic positions, symmetry, and metadata. Each section is organized in a hierarchical manner, making it easy to access specific pieces of information. For instance, to access the crystal system, you would use data["crystal_structure"]["symmetry"]["crystal_system"], which would return “Cubic”. This format is particularly useful for representing complex data structures in a clear and organized way.

However, JSON is not as efficient as binary formats for large datasets, as it can be verbose and redundant. For large datasets, binary formats like HDF5 or NetCDF are more suitable.

Other Formats¶

Other formats related to crystal structure including XYZ file, protein data bank (PDB), crystallographic information file (CIF), and extended XYZ (ExtXYZ) are also widely used in materials informatics and will be discussed in the lecture later.

Machine-Readable Formats¶

Machine-readable formats are designed to be easily processed by computers. These formats are optimized for efficient storage, retrieval, and manipulation of data.

HDF5 (Hierarchical Data Format version 5)¶

One such format is HDF5 (Hierarchical Data Format version 5). HDF5 is a versatile, high-performance data format that supports large, complex, and hierarchical datasets. It is widely used in scientific computing and materials informatics for storing and exchanging data. HDF5 files can store a wide range of data types and structures, making them suitable for diverse applications. The hierarchical structure of HDF5 allows for efficient organization and retrieval of data. For example, an HDF5 file might contain datasets representing material properties, simulation results, or experimental measurements.

NetCDF (Network Common Data Form)¶

Another machine-readable format is NetCDF (Network Common Data Form). NetCDF is a machine-independent, self-describing data format used for storing array-oriented scientific data. It is commonly used in atmospheric and oceanographic research, climate modeling, and geospatial data analysis. NetCDF files can store multidimensional arrays, metadata, and attributes, making them suitable for representing complex scientific data. The self-describing nature of NetCDF files allows users to store and exchange data along with relevant information about the data structure and content.