OutputDatasetConfig Class

Represent how to copy the output of a job to a remote storage location and be promoted to a Dataset.

This is the base class used to represent how to copy the output of a job to a remote storage location, whether to register it as a named and versioned Dataset, and whether to apply any additional transformations to the Dataset that was created.

You should not be creating instances of this class directly but instead should use the appropriate subclass.

Initialize a OutputDatasetConfig.

Inheritance
builtins.object
OutputDatasetConfig

Constructor

OutputDatasetConfig(mode, name=None, **kwargs)

Parameters

Name Description
mode
Required
str

The mode in which to copy the output to the remote storage.

name
Required
str

The name of the output specific to the run it will be produced in.

Methods

as_input

Specify how to consume the output as an input in subsequent pipeline steps.

register_on_complete

Register the output as a new version of a named Dataset after the run has ran.

If there are no datasets registered under the specified name, a new Dataset with the specified name will be registered. If there is a dataset registered under the specified name, then a new version will be added to this dataset.

as_input

Specify how to consume the output as an input in subsequent pipeline steps.

as_input(name=None)

Parameters

Name Description
name
str

The name of the input specific to the run.

Default value: None

Returns

Type Description

A DatasetConsumptionConfig instance describing how to deliver the input data.

register_on_complete

Register the output as a new version of a named Dataset after the run has ran.

If there are no datasets registered under the specified name, a new Dataset with the specified name will be registered. If there is a dataset registered under the specified name, then a new version will be added to this dataset.

register_on_complete(name, description=None, tags=None)

Parameters

Name Description
name
Required
str

The Dataset name to register the output under.

description
Required
str

The description for the Dataset.

tags
Required

A list of tags to be assigned to the Dataset.

Returns

Type Description

A new OutputDatasetConfig instance with the registration information.

Attributes

name

Name of the output.

Returns

Type Description

Name of the output.