Package nz.org.riskscape.engine.pipeline

Class Pipeline

java.lang.Object
nz.org.riskscape.engine.pipeline.Pipeline
public class Pipeline extends Object

A user-configurable data processing pipeline for performing hazard analysis. A pipeline gives the basic shape of some possible data transformation and computation, with parameters for adjusting the specifics of the modeling, such as setting the source data, filtering and output options.

  • Field Details

  • Constructor Details

  • Method Details

    • usingAst

      public static Pipeline usingAst(PipelineDeclaration ast)

      Create a pipeline with the given ast as metadata - it won't necessarily have been built from the ast yet.

    • empty

      public static Pipeline empty()

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull Edge from, @NonNull @NonNull NamedStep to)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull String from, @NonNull @NonNull String to)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull String from, @NonNull @NonNull String to, @NonNull @NonNull String namedInput)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull String from, @NonNull @NonNull String to, Optional<String> namedInput)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull String from, @NonNull @NonNull NamedStep to)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull NamedStep from, @NonNull @NonNull NamedStep to)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull NamedStep from, @NonNull @NonNull NamedStep to, @NonNull @NonNull String namedInput)

    • addEdge

      public Pipeline addEdge(@NonNull @NonNull NamedStep from, @NonNull @NonNull NamedStep to, @NonNull @NonNull Optional<String> namedInput) throws PipelineDefinitionException

      Adds an edge with the given parameters to the pipeline. In terms of pipeline execution, this will mean data will move from the from step to the to step. If a named input is given, data will move to this named input in particular. See Step.getInputNames() for more infprmation about named inputs.

      Parameters:
      from - a named step that may or may not already exist in this pipeline.
      to - a named step taht may or may not already exist in this pipeline.
      namedInput - an optional named input to target with this edge. Some steps declare named inputs, which tend to have a particular purpose. This needs to be present if the target step declares named inputs (NamedStep.requiresNamedInputs() will return true).
      Returns:
      this pipeline, for fluid builder-style use
      Throws:
      PipelineDefinitionException - if adding this edge would invalidate the pipeline

    • addStep

      public Pipeline addStep(@NonNull @NonNull String name, @NonNull @NonNull Step step)

    • addStep

      public Pipeline addStep(@NonNull @NonNull NamedStep step)

    • getStep

      public Optional<NamedStep> getStep(@NonNull @NonNull String name)

    • getEdgesFrom

      public List<Edge> getEdgesFrom(@NonNull @NonNull NamedStep fromStep)

    • getEdgesTo

      public List<Edge> getEdgesTo(@NonNull @NonNull NamedStep toStep)

    • getSteps

      public Set<NamedStep> getSteps()

    • getDependencies

      public List<NamedStep> getDependencies(@NonNull @NonNull NamedStep step)

      Get a list of dependencies that are required to provide the input(s) to step in the same order as the inputs are defined in step parameters.

      Parameters:
      step - to get dependencies of
      Returns:
      dependencies in input order

    • getEdges

      public Set<Edge> getEdges()

    • getParameterNames

      public List<String> getParameterNames()

    • getParameters

      public List<Parameter> getParameters()

    • addParameter

      public Pipeline addParameter(NamedStep step, String paramName, Object paramValue)

      Convenience version of that will construct the parameterKey to be <step-name>.<paramName></p>

    • addParameter

      public Pipeline addParameter(String paramKey, Object paramValue)

      Add a parameter value to parameters.

      If the parameter map already contains an entry for this parameter then paramValue is appended to the value list. Otherwise a new list is inserted containing only paramValue.

      Parameters:
      paramKey - fully qualified parameter key
      paramValue - parameter value
      Returns:
      pipeline

    • setParameter

      public Pipeline setParameter(NamedStep step, String paramName, List<?> paramValues)

      Convenience version of that will construct the parameterKey to be <step-name>.<paramName></p>

    • setParameter

      public Pipeline setParameter(String paramKey, List<?> paramValues)

      Sets the parameter values, discarding any values that already existed for that parameter.

      Parameters:
      paramKey - fully qualified parameter key
      paramValues - values to set
      Returns:
      pipeline

    • visit

      public <T> List<T> visit(BiFunction<NamedStep,List<T>,T> visitor) throws PipelineCycleException

      Visit all steps in the DAG in dependency order.

      Parameters:
      visitor - a function that gets given all the steps, along with any built things from dependent steps
      Returns:
      a list of all the elements returned from the visitor function, in the order they were created
      Throws:
      PipelineCycleException - if the pipeline contains a cycle - this will be picked up before the visitor is called

    • clone

      public Pipeline clone()
      Overrides:
      clone in class Object

    • addAll

      public void addAll(Pipeline copyPipeline)

    • getParameter

      public Optional<Parameter> getParameter(String key)

    • hasStep

      public boolean hasStep(String lookForStepNamed)
      Returns:
      true if the pipeline contains a step with the given name

    • getStepsRequiringInput

      public List<NamedStep> getStepsRequiringInput()
      Returns:
      a list of NamedSteps that do not yet have Edges for all of their required inputs.

    • getStartSteps

      public List<NamedStep> getStartSteps()

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • canEqual

      protected boolean canEqual(Object other)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • getAst

      public PipelineDeclaration getAst()

      The AST that this pipeline was built from.

    • getUnboundParameters

      public Map<String,List<?>> getUnboundParameters()

      Unbound parameters, can be set with pipeline default values.