Interface PipelineOutputContainer

Method Details

• createSinkForStep

  ResultOrProblems<Sink> createSinkForStep(SinkParameters parameters)

  Create a Sink for the given terminalStep.

  Parameters:

  parameters - to tailor the created sink to

  Returns:

  sink or problems preventing one from being created

• close

  void close()

  Finish writing to the output container. At this point all Sinks should have been closed.

  If the store supports writing the progress metrics now is the time to write them.

  Specified by:

  close in interface AutoCloseable

• getPipelineOutputs

  Map<String,URI> getPipelineOutputs()

  Returns:

  a map of output names to storage locations. TODO we might want to replace URI with something a bit more informative, such as number of rows, size, checksum etc

• getStore

  PipelineOutputStore getStore()

  Returns:

  the store that created this container.

• getStoredAt

  URI getStoredAt()

  Returns a URI that describes where the outputs are stored. Note this relates to the pipeline output URIs in that this URI is typically a 'container' URI, but how these relate will typically depend on the format or type of storage

  Note that at this stage we're not making any guarantees that this will be generally useful beyond being informational. Ideally, other components could reason about these URIs based on their structure and do something useful with them, e.g. if it looks like a local directory, you could open it. If it's a database URL, you could attempt to connect to it.

  Note also that this URI might be different to the one the user gave, e.g. we might add a bunch of directories to the beginning of a file, or resolve/follow a URI to some other location, so this should be used in preference to the one the user gave when presenting info back to the user