`mscompress.metadata`¶

Croissant metadata builders for ML dataset publication.

Builders¶

MetadataBuilder ¶

Bases: ABC

Abstract base class for building Croissant metadata from data sources.

Subclasses implement specific metadata extraction for different file types (MSZ files, search results, etc.).

name `abstractmethod` `property` ¶

name: str

Unique identifier for this metadata source.

description `abstractmethod` `property` ¶

description: str

Human-readable description of this metadata source.

join_key `property` ¶

join_key: Optional[str]

The field name to use for joining with other metadata sources. Override in subclasses to enable joining.

get_record_set `abstractmethod` ¶

get_record_set() -> RecordSetDefinition

Build the RecordSet definition for this metadata source.

get_distribution `abstractmethod` ¶

get_distribution(base_url: str) -> FileDistribution

Build the file distribution definition.

get_data_collection_info `abstractmethod` ¶

get_data_collection_info() -> DataCollectionInfo

Build the data collection info.

MSZMetadataBuilder ¶

MSZMetadataBuilder(
    msz: MSZFile, source_name: Optional[str] = None
)

Bases: MetadataBuilder

Builds Croissant metadata from MSZ (compressed mzML) files.

Extracts spectrum-level metadata including m/z arrays, intensities, retention times, and scan information.

Initialize the MSZ metadata builder.

Parameters:

Name	Type	Description	Default
`msz`	`MSZFile`	MSZFile to extract metadata from.	required
`source_name`	`Optional[str]`	Optional name override for the file source.	`None`

get_record_set ¶

get_record_set() -> RecordSetDefinition

Build RecordSet with all spectrum fields.

SearchResultsMetadataBuilder ¶

SearchResultsMetadataBuilder(
    file_path: Union[str, Path],
    source_name: Optional[str] = None,
)

Bases: MetadataBuilder

Abstract base class for search results metadata (PSMs, peptide IDs, etc.).

Search results typically contain peptide-spectrum matches (PSMs) with scores, sequences, and other identification data.

Initialize the search results metadata builder.

Parameters:

Name	Type	Description	Default
`file_path`	`Union[str, Path]`	Path to the search results file.	required
`source_name`	`Optional[str]`	Optional name override for the file source.	`None`

join_key `property` ¶

join_key: str

Search results join on scan_number by default.

PercolatorMetadataBuilder ¶

PercolatorMetadataBuilder(
    file_path: Union[str, Path],
    source_name: Optional[str] = None,
    num_psms: Optional[int] = None,
)

Bases: SearchResultsMetadataBuilder

Builds metadata from Percolator .pin or .pout files.

Percolator files contain peptide-spectrum match scores and features used for FDR control in proteomics.

Initialize the Percolator metadata builder.

Parameters:

Name	Type	Description	Default
`file_path`	`Union[str, Path]`	Path to .pin or .pout file.	required
`source_name`	`Optional[str]`	Optional name for the source.	`None`
`num_psms`	`Optional[int]`	Number of PSMs (if known, otherwise computed lazily).	`None`

get_record_set ¶

get_record_set() -> RecordSetDefinition

Build RecordSet with Percolator-specific fields.

PepXMLMetadataBuilder ¶

PepXMLMetadataBuilder(
    file_path: Union[str, Path],
    source_name: Optional[str] = None,
    num_psms: Optional[int] = None,
)

Bases: SearchResultsMetadataBuilder

Builds metadata from pepXML search result files.

pepXML is a standardized format for peptide identification results from database search engines.

Initialize the pepXML metadata builder.

Parameters:

Name	Type	Description	Default
`file_path`	`Union[str, Path]`	Path to .pepXML file.	required
`source_name`	`Optional[str]`	Optional name for the source.	`None`
`num_psms`	`Optional[int]`	Number of PSMs (if known).	`None`

get_record_set ¶

get_record_set() -> RecordSetDefinition

Build RecordSet with pepXML-specific fields.

CompositeMetadataBuilder ¶

CompositeMetadataBuilder(
    dataset_name: str,
    description: str,
    repo_id: str,
    license: str = "unknown",
)

Combines multiple metadata builders into a unified Croissant dataset.

Allows building datasets that include both spectrum data and search results, with defined relationships between them.

Example

msz = mscompress.read("sample.msz")
msz_builder = MSZMetadataBuilder(msz)
pin_builder = PercolatorMetadataBuilder("sample.pin")

composite = CompositeMetadataBuilder(
    dataset_name="Proteomics Dataset",
    description="Mass spec data with search results",
    repo_id="username/dataset",
)
composite.add_builder(msz_builder, primary=True)
composite.add_builder(pin_builder)
composite.add_join(msz_builder, pin_builder, "scan_number")

metadata = composite.build()

Initialize the composite metadata builder.

Parameters:

Name	Type	Description	Default
`dataset_name`	`str`	Human-readable name for the dataset.	required
`description`	`str`	Description of the dataset.	required
`repo_id`	`str`	HuggingFace repository ID.	required
`license`	`str`	License identifier (e.g., "CC-BY-4.0").	`'unknown'`

base_url `property` ¶

base_url: str

Base URL for the HuggingFace dataset.

add_builder ¶

add_builder(
    builder: MetadataBuilder, primary: bool = False
) -> CompositeMetadataBuilder

Add a metadata builder to the composite.

Parameters:

Name	Type	Description	Default
`builder`	`MetadataBuilder`	MetadataBuilder instance to add.	required
`primary`	`bool`	If True, this is the primary data source.	`False`

Returns:

Type	Description
`CompositeMetadataBuilder`	Self for method chaining.

add_join ¶

add_join(
    left: MetadataBuilder,
    right: MetadataBuilder,
    join_key: Optional[str] = None,
    join_type: str = "left",
) -> CompositeMetadataBuilder

Define a join relationship between two metadata sources.

Parameters:

Name	Type	Description	Default
`left`	`MetadataBuilder`	Left side of the join.	required
`right`	`MetadataBuilder`	Right side of the join.	required
`join_key`	`Optional[str]`	Field to join on (auto-detected if not specified).	`None`
`join_type`	`str`	Type of join (left, right, inner, outer).	`'left'`

Returns:

Type	Description
`CompositeMetadataBuilder`	Self for method chaining.

build ¶

build() -> Dict[str, Any]

Build the complete Croissant metadata dictionary.

Returns:

Type	Description
`Dict[str, Any]`	Complete Croissant metadata following the ML Croissant standard.

Convenience functions¶

build_msz_metadata ¶

build_msz_metadata(
    msz: MSZFile,
    dataset_name: str,
    description: str,
    repo_id: str,
) -> Dict[str, Any]

Generate Croissant metadata for an MSZ dataset.

This is a convenience function that wraps the class-based API for simple use cases with a single MSZ file.

Croissant is a high-level format for machine learning datasets that provides standardized metadata about dataset structure, fields, and semantics.

Parameters:

Name	Type	Description	Default
`msz`	`MSZFile`	MSZFile object to extract metadata from.	required
`dataset_name`	`str`	Human-readable name for the dataset.	required
`description`	`str`	Description of the dataset.	required
`repo_id`	`str`	HuggingFace repository ID.	required

Returns:

Type	Description
`Dict[str, Any]`	Dict containing Croissant metadata following the ML Croissant standard.

Example

import mscompress
from mscompress.metadata import build_msz_metadata

msz = mscompress.read("sample.msz")
metadata = build_msz_metadata(
    msz,
    dataset_name="My Dataset",
    description="Sample proteomics data",
    repo_id="username/my-dataset",
)

build_composite_metadata ¶

build_composite_metadata(
    msz: MSZFile,
    search_results: Optional[Union[str, Path]] = None,
    search_format: str = "pin",
    dataset_name: str = "Dataset",
    description: str = "",
    repo_id: str = "",
    join_key: str = "scan_number",
) -> Dict[str, Any]

Generate Croissant metadata for an MSZ dataset with optional search results.

Convenience function for creating composite metadata that includes both spectrum data and search results (PSMs).

Parameters:

Name	Type	Description	Default
`msz`	`MSZFile`	MSZFile object.	required
`search_results`	`Optional[Union[str, Path]]`	Path to search results file (.pin or .pepXML).	`None`
`search_format`	`str`	Format of search results ("pin" or "pepxml").	`'pin'`
`dataset_name`	`str`	Human-readable name for the dataset.	`'Dataset'`
`description`	`str`	Description of the dataset.	`''`
`repo_id`	`str`	HuggingFace repository ID.	`''`
`join_key`	`str`	Field to join spectrum and search data on.	`'scan_number'`

Returns:

Type	Description
`Dict[str, Any]`	Dict containing Croissant metadata.

Example

import mscompress
from mscompress.metadata import build_composite_metadata

msz = mscompress.read("sample.msz")
metadata = build_composite_metadata(
    msz,
    search_results="sample.pin",
    search_format="pin",
    dataset_name="Annotated Dataset",
    description="Proteomics with PSM annotations",
    repo_id="username/annotated-data",
)

Type definitions¶

FieldDefinition `dataclass` ¶

FieldDefinition(
    name: str,
    description: str,
    data_type: Union[str, List[str]],
    source_file_object: str,
    is_array: bool = False,
)

Definition of a field in a Croissant RecordSet.

to_croissant ¶

to_croissant() -> Dict[str, Any]

Convert to Croissant field format.

RecordSetDefinition `dataclass` ¶

RecordSetDefinition(
    name: str,
    description: str,
    fields: List[FieldDefinition] = list(),
    join_key: Optional[str] = None,
)

Definition of a RecordSet in Croissant metadata.

to_croissant ¶

to_croissant() -> Dict[str, Any]

Convert to Croissant RecordSet format.

FileDistribution `dataclass` ¶

FileDistribution(
    name: str,
    content_url: str,
    encoding_format: str = "application/octet-stream",
    sha256: str = "unknown",
)

Definition of a file distribution in Croissant metadata.

to_croissant ¶

to_croissant() -> Dict[str, Any]

Convert to Croissant FileObject format.

DataCollectionInfo `dataclass` ¶

DataCollectionInfo(
    size_description: str,
    data_type: str,
    format_description: str,
)

Information about the data collection.

to_croissant ¶

to_croissant() -> Dict[str, Any]

Convert to Croissant DataCollection format.

JoinDefinition `dataclass` ¶

JoinDefinition(
    left_record_set: str,
    right_record_set: str,
    join_key: str,
    join_type: str = "left",
)

Defines how two record sets should be joined.

to_croissant ¶

to_croissant() -> Dict[str, Any]

Convert to Croissant join definition format.

JoinStrategy ¶

Bases: Enum

Strategy for joining metadata from multiple sources.

mscompress.metadata¶

Builders¶

MetadataBuilder ¶

name abstractmethod property ¶

description abstractmethod property ¶

join_key property ¶

get_record_set abstractmethod ¶

get_distribution abstractmethod ¶

get_data_collection_info abstractmethod ¶

MSZMetadataBuilder ¶

get_record_set ¶

SearchResultsMetadataBuilder ¶

join_key property ¶

PercolatorMetadataBuilder ¶

get_record_set ¶

PepXMLMetadataBuilder ¶

get_record_set ¶

CompositeMetadataBuilder ¶

base_url property ¶

add_builder ¶

add_join ¶

build ¶

Convenience functions¶

build_msz_metadata ¶

build_composite_metadata ¶

Type definitions¶

FieldDefinition dataclass ¶

to_croissant ¶

RecordSetDefinition dataclass ¶

to_croissant ¶

FileDistribution dataclass ¶

to_croissant ¶

DataCollectionInfo dataclass ¶

to_croissant ¶

JoinDefinition dataclass ¶

to_croissant ¶

JoinStrategy ¶

`mscompress.metadata`¶

name `abstractmethod` `property` ¶

description `abstractmethod` `property` ¶

join_key `property` ¶

get_record_set `abstractmethod` ¶

get_distribution `abstractmethod` ¶

get_data_collection_info `abstractmethod` ¶

join_key `property` ¶

base_url `property` ¶

FieldDefinition `dataclass` ¶

RecordSetDefinition `dataclass` ¶

FileDistribution `dataclass` ¶

DataCollectionInfo `dataclass` ¶

JoinDefinition `dataclass` ¶