.. _python-functions: Python functions ====================== ## Basic function Let's start off with the simplest possible example Python function: a 'Hello World' example: ```python def function(name): return 'Hello, ' + name ``` Python functions must be defined in a `.py` file (e.g. `hello.py`). The function in this file that RiskScape will execute should always be called `function`. You then need to add a section to your `project.ini` file that tells RiskScape how your function can be used. For example, the following sections loads the `hello.py` function and names it `hello`. ```ini [function hello] description = Simple 'Hello world' function example location = hello.py argument-types = [ text ] return-type = text ``` Once you have defined your function, you should see it successfully displayed by the `riskscape function list` command. For example: ```none riskscape function list +-----+-------------------------------------+---------+-----------+----------+ |id |description |arguments|return-type|category | +-----+-------------------------------------+---------+-----------+----------+ |hello|Simple 'Hello world' function example|[Text] |Text |UNASSIGNED| +-----+-------------------------------------+---------+-----------+----------+ ``` You can then use (or 'call') your function from any RiskScape expression. For example: ```none riskscape expression evaluate "hello('world')" Hello, world riskscape expression evaluate "hello('Ronnie')" Hello, Ronnie ``` ## Types All the input data that RiskScape processes has :ref:`type information ` associated with it, such as `text`, `integer`, `floating`, `geometry`, etc. The same also applies to a function's arguments and return value - you need to define what data types your function accepts before you can use it. The INI file definition tells RiskScape what types the function expects. For example, our `hello` function expects a single `text` string argument. This means we will get an error if we call it with different arguments, such as an integer or multiple text strings. ```none riskscape expression evaluate "hello(1)" Failed to evaluate expression - Could not find function hello with arguments [Integer] - function 'hello' only supports arguments [Text] riskscape expression evaluate "hello('Ronnie', 'Janice')" Failed to evaluate expression - Could not find function hello with arguments [Text, Text] - function 'hello' only supports arguments [Text] ``` ### Anything type RiskScape functions support an `anything` argument-type. This basically acts like a 'type wildcard' and means your function will accept _any_ data type as an argument. .. tip:: Using the ``anything`` type as a function argument can make life easier if you are getting RiskScape errors complaining about '*Could not find function with arguments...*'. Let's try using the `anything` type with our `hello` function. Update your `project.ini` file so that it looks like this: ```ini [function hello] description = Simple 'Hello world' function example location = hello.py argument-types = [ anything ] return-type = text ``` We can still pass our function a `text` argument and it behaves the same way. ```none riskscape expr eval "hello('Ronnie')" Hello, Ronnie ``` Let's check what error we get when we pass the function an `integer` type, like we did in the previous section. ```none riskscape expr eval "hello(1)" Failed to evaluate `hello(1)` - Problems found with 'hello' function (from source file:/C:/RiskScape_Projects/function-demo/hello.py) - Your CPython function raised an error: TypeError: can only concatenate str (not "int") to str File "/C:/RiskScape_Projects/function-demo/hello.py", line 2, in function return 'Hello, ' + name ``` Notice that the error is now coming directly from our Python code, rather than RiskScape complaining that the types do not match. This is the main drawback of using the `anything` type. RiskScape no longer provides type-safety checks for you, nor guarantees that the attributes in the input data will match what your Python function expects. .. note:: Using the ``anything`` type may reduce performance slightly, as it adds some additional processing overhead for RiskScape. This would be most noticeable when your input data contains complex line or polygon geometry. .. _hello_world: ### Multiple arguments The `argument-types` is a comma-separated list of types, surrounded by `[ ]`s. Let's expand our example so that we can pass our function a name _and_ a greeting. Our Python code will look like: ```python def function(greeting, name): return greeting + ', ' + name ``` We then need to update the `argument-types` in our INI file definition so that it matches. Our function now takes _two_ `text` string arguments. ```ini [function hello] description = Simple 'Hello world' function example that takes multiple arguments location = hello.py argument-types = [ text, text ] return-type = text ``` We can then call our function, passing it _two_ arguments this time. ```none riskscape expression evaluate "hello('Kia ora', 'Ronnie')" Kia ora, Ronnie ``` .. note:: RiskScape does not inspect your Python code for you. Any errors in your Python code will result in an error when you try to call your function. If the number of arguments in your Python code *differs* to your INI file definition, then this will result in an error. Here's an example of what an error looks like when our Python code has _two_ function arguments, but our INI definition only declares _one_ argument-type. ```none riskscape expr eval "hello('foo')" Failed to evaluate `hello('foo')` - A problem occurred while executing the function 'hello'. Please check your Python code carefully for the likely cause. - TypeError: function() takes exactly 2 arguments (1 given) (no traceback available) ``` ### Keyword arguments You can optionally specify keyword names for each of your argument-types, to make things clearer. These keywords get used on the _RiskScape_ side of things, rather than on the Python side. For example, update the `argument-types` in your INI file definition to the following: ```ini [function hello] description = 'Hello world' function example that takes keyword arguments location = hello.py argument-types = [ greeting: text, name: text ] return-type = text ``` The new keyword arguments will be displayed in `riskscape function list`. The keywords can also _optionally_ be used in any RiskScape expression, e.g. ```none riskscape expression evaluate "hello(greeting: 'Kia ora', name: 'Ronnie')" Kia ora, Ronnie riskscape expression evaluate "hello('Greetings', name: 'Janice')" Greetings, Janice ``` .. tip:: Keywords help add clarity and avoid confusion when your function takes several arguments of the same type. For example, ``do_some_maths(mean: 0.22, stddev: 0.74)`` is easier to read than ``do_some_maths(0.22, 0.74)``. ### Structs So far, our example function's argument and return types have been _simple_ types, like `text` or `floating`. They can also be a complex type, called a _Struct_. A `struct` can represent a set of several different attributes from your input data bundled together. We can define a struct as a _type definition_ using the following format: ```none struct(attribute-name: simple-type, ...) ``` As an example, let's change our `hello` function to take a basic struct containing two attributes: a greeting and a name. ```ini [function hello] description = 'Hello world' function example that takes a struct location = hello.py argument-types = [ struct(greeting: text, name: text) ] return-type = text ``` This looks very similar to our previous example, but instead of having _two_ keyword `text` arguments, we now have a _single_ `struct` argument. In our Python code, we can access the `struct` in a similar way to a Python dictionary: `struct.get('attribute-name')`. For example: ```python def function(details): return details.get('greeting') + ', ' + details.get('name') ``` We can now try calling our function and pass it a struct. We can define a struct in a RiskScape _expression_ using `{ }`s, and specify the actual _value_ rather than the _type_. For example: ```none riskscape expression evaluate "hello({greeting: 'Kia ora', name: 'Ronnie'})" Kia ora, Ronnie riskscape expr eval "hello({greeting: 'Greetings', name: 'Janice'})" Greetings, Janice ``` ### Struct coercion When the data supplied to a function does not match the expected argument types exactly, RiskScape can often _coerce_ the argument types, or 'make do with what it has got'. This means you can pass a function a struct with _more_ attributes than the function expects, as long as the attributes are a _subset_ of the expected struct. For example, we can call our function with an extra `time` attribute that our function will just ignore. ```none riskscape expression evaluate "hello({greeting: 'Kia ora', name: 'Ronnie', time: '11:00am'})" Kia ora, Ronnie ``` .. tip:: Try to limit your function's ``argument-types`` to just the bare essentials, i.e. only include the attributes that the function actually needs to use. Your function's ``argument-types`` do not have to match your input data *exactly*. This will lead to functions you can more easily reuse with different input data files. ### User-defined types The `struct()` type expressions can get complicated when you have many different attributes you are interested in. To make life easier, you can also define your own types in the INI file, and then _reference_ these in your function definition. For example, we can define our own `greet` type and then use that in our function. ```ini [type greet] type.greeting = text type.name = text [function hello] description = 'Hello world' function example that takes a user-defined type location = hello.py argument-types = [ greet ] return-type = text ``` This behaves exactly the same as our previous example. However, it can make it simpler to follow what your function does when dealing with complicated types, or the same types that you end up using a lot. .. note:: If you want to use a user-defined type *within* another ``struct`` definition, then you will have to use ``lookup()`` to resolve the type reference. For example, to use our ``greet`` type as an attribute in another struct, we have to do: ``struct(the_greeting: lookup('greet'))``. ### Optional arguments You can define a function that takes an optional argument. This means the argument value may or may not be specified. In your Python code, the argument will be `None` if it is omitted. E.g. ```python def function(details): if details is None: return 'Hello, world' return details.get('greeting') + ', ' + details.get('name') ``` In order for this to work you have to also tell RiskScape that the argument may be `None` or 'nothing'. In RiskScape terminology, the type is called :ref:`nullable `. Your function definition in your `project.ini` file should look like this: ```ini [type greet] type.greeting = text type.name = text [function hello] description = 'Hello world' function example with optional argument location = hello.py # note that we now surround the argument type in 'nullable' (this means we need 'lookup()' as well) argument-types = [ nullable(lookup('greet')) ] return-type = text ``` Our function will now work if we completely omit the arguments. ```none riskscape expr eval "hello()" Hello, world riskscape expr eval "hello({greeting: 'Kia ora', name: 'Ronnie'})" Kia ora, Ronnie ``` .. note:: The hazard intensity measure in your model will always be nullable, i.e. if the hazard-layer did not intersect with an asset-at-risk, then the hazard value will be null. If you do not define the hazard argument as nullable, then RiskScape will simply not call your function when the hazard value is null. ### Return types Defining the function's `return-type` in the INI file is similar to defining the `argument-types`. Let's make our function's return value a little more complicated. As well as returning a greeting, let's say we also want to return a 'diary entry' of when we met the person. ```ini [function hello] description = 'Hello world' function example with struct return-value location = hello.py argument-types = [ text, text ] return-type = struct(greeting: text, diary_entry: text) ``` To return a struct in our Python code, we just define it as a Python dictionary. For example: ```python import time def function(greeting, name): return { 'greeting': greeting + ', ' + name, 'diary_entry': 'Met with ' + name + ' at ' + time.asctime() } ``` When we call our function it now looks like this: ```none riskscape expr eval "hello('Kia ora', 'Ronnie')" {greeting=Kia ora, Ronnie, diary_entry=Met with Ronnie at Fri Aug 20 16:06:24 2021} ``` ### Multiple functions in the same file You may have noticed this example contains _two_ Python functions in the same file. Your Python file can contain several other smaller functions that help your primary Python function to do its job. Remember that the primary Python function that RiskScape executes should always be called `function`. .. tip:: Splitting the work up into several functions can help to make your Python code easier to follow and to test. ### Reusing functions for different input files Ideally, you should be able to reuse the same function across multiple different input data files, as long as the input data is essentially the same and the function is doing the same job. RiskScape will try to _coerce_ the input data as best it can to make it fit your function. However, one common problem is if the same attribute is called different things in different files. This is particularly a problem with shapefiles, which often have truncated attribute names. For example, a common problem when trying to build or run your model is getting an error like: ```none - Could not find function quake with arguments [{REPLACE_1=>Integer, CONSTRUCT_1=>Integer}, Floating] - function 'quake' only supports arguments [building, Nullable[Floating]] ``` In this case the attribute names in the input data (`REPLACE_1`, `CONSTRUCT_1`) don't match what our function expects (`replace`, `construct`). The best way to fix the problem is to simply change the attribute names in the input data to match what our function expects. We can do this on the fly in the :ref:`bookmark ` definition without needing to modify our input data file at all. For example: ```ini [bookmark buildings] location = buildings.shp set-attribute.replace = REPLACE_1 set-attribute.construct = CONSTRUCT_1 ``` ## Function categories All built-in RiskScape functions are assigned to a category. Categories are mostly useful for grouping functions in `riskscape function list --category`. Using categories be helpful in organizing your functions if you start to add a lot of them. You can assign categories to your own functions in the INI definition, e.g. ```ini [function hello] description = 'Hello world' function example with category location = hello.py argument-types = [text, text] return-type = text # assign our function to the 'language' category category = language ``` .. _jython_vs_cpython: ## Jython vs CPython RiskScape will use _Jython_ to execute your Python function by default. Jython support is included as part of the RiskScape software image. Refer to :ref:`jython-impl` for more specifics on using Jython functions in RiskScape. However, Jython is a little different to the Python you would normally use outside of RiskScape, which is called _CPython_. Experienced Python users may prefer to use CPython (i.e. regular Python) instead of Jython. However, CPython is _not_ distributed as part of the RiskScape software. So you have to manually configure RiskScape to use same Python executable that you use locally. Refer to :ref:`cpython-impl` for more specifics on using CPython functions in RiskScape. .. tip:: If you are not sure what Python implementation you should use, then the Jython default is probably good enough for most simple Python functions. If you need to import other Python packages, such as for maths operations, then we recommend using CPython. .