Package nz.org.riskscape.engine.sched

Class PageAllocator

java.lang.Object

nz.org.riskscape.engine.sched.PageAllocator

public class PageAllocator
extends Object

Handles all the complicated logic of how many tuples should fit into a Page. These characteristics can vary from task to task, so this class allows different parts of the pipeline to fine-tune that behaviour, e.g. slower tasks should use smaller pages and more of them.

Constructor Summary

Constructors

Constructor	Description
PageAllocator(int fixedSize)	Returns a basic page allocator that will always allocate the same size page.
PageAllocator(int startAt, int maxSize, Optional<Integer> incrementPeriod, Optional<Predicate<PageAllocator>> backoff)

Method Summary

Modifier and Type	Method	Description
PageAllocator	doubleEvery(int nTimes)	Returns a new PageAllocator that gradually increases the size of the page, every nTimes.
int	getMaxPageSize()	The maximum number of tuples that can potentially fit into a single Page.
protected int	getPageSize()
boolean	isBackoffNeeded()
Page	newPage()
PageAllocator	withBackoff(Predicate<PageAllocator> backoffWhen)	Returns a new PageAllocator that can backoff and reduce the size of the pages it allocates under certain conditions, i.e.
PageAllocator	withMaxPageSize(int maxSize)	Returns a new PageAllocator with a fixed maximum size, but with all other characteristics the same.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PageAllocator

public PageAllocator(int startAt,
 int maxSize,
 Optional<Integer> incrementPeriod,
 Optional<Predicate<PageAllocator>> backoff)

PageAllocator

public PageAllocator(int fixedSize)

Returns a basic page allocator that will always allocate the same size page.

Method Details

withMaxPageSize

public PageAllocator withMaxPageSize(int maxSize)

Returns a new PageAllocator with a fixed maximum size, but with all other characteristics the same.

withBackoff

public PageAllocator withBackoff(Predicate<PageAllocator> backoffWhen)

Returns a new PageAllocator that can backoff and reduce the size of the pages it allocates under certain conditions, i.e. memory pressure. The allocator will go back to the initial page size, so this only will work in combination with doubleEvery(int).

doubleEvery

public PageAllocator doubleEvery(int nTimes)

Returns a new PageAllocator that gradually increases the size of the page, every nTimes. This allows pages to start off small (which works better for a small number of total tuples that require intensive processing), and then grow up to their maximum size (which works better for large datasets).

isBackoffNeeded

public boolean isBackoffNeeded()

Returns:

true if we should reduce the size of the pages we're allocating (or the number of tuples queued in the buffer itself)

getPageSize

protected int getPageSize()

Returns:

the size (number of tuples) that the next page should use

newPage

public Page newPage()

Returns:

a new Page that can then be filled with Tuples

getMaxPageSize

public int getMaxPageSize()

The maximum number of tuples that can potentially fit into a single Page.