ArcGIS Online, ArcGIS Enterprise, and ArcGIS Hub

The simplest way to use ArcGIS data in your RiskScape model is to download the data, rather than using the HTTP APIs to access the data. You can then use the downloaded file in your model as per normal.

Manually downloading the data has a few downsides:

  • Your local copy might be out of date if the data in ArcGIS changes.

  • It makes it harder to share your model with others, as they have to download the exact same set of files manually.

RiskScape can download data directly from any HTTP link. However, it is not always easy to find the right URL to access ArcGIS data. The following page describes how to read data directly from sources like ArcGIS Online and ArcGIS Enterprise.

Tip

Generally the formats that work best with RiskScape over HTTP are WFS, GeoJSON, and KML.

WFS

WFS is a standard way of accessing geospatial data across the internet, and is generally the simplest way to download data from an ArcGIS source. You can read more about WFS here.

Here is an example bookmark that accesses an ArcGIS Online WFS server. This example was found through the https://data.govt.nz website, and accesses a dataset containing the recreation centres in Canterbury.

[bookmark Canterbury_rec_centres]
location = https://gis.ecan.govt.nz/arcgis/services/Public/Canterbury_Maps/MapServer/WFSServer
format = wfs
layer = Canterbury_Maps:Recreation_Centre
crs-name = EPSG:2193
crs-longitude-first = true

Note that we have to set crs-longitude-first, as this particular WFS server returns the coordinates in long, lat order.

The WFS address for an ArcGIS Online server will typically look something like:

https://WEBSITE/arcgis/services/DATASET_CONTAINER/MapServer/WFSServer?request=GetCapabilities&service=WFS

Note that it typically has WFSServer after MapServer in the URL. The ?request=GetCapabilities&service=WFS bit is optional in your RiskScape bookmark, but can be useful for testing in your web browser.

ArcGIS Online usually also provides a REST API that can help you navigate the datasets in your web browser. For our example bookmark, you can view the ArcGIS Online REST API here.

When you view the REST API, at the top of the page there will be links for (optionally) JSON, SOAP, WFS, and WMS. Click on the WFS link - that is the URL you should use in your RiskScape bookmark.

Tip

Typically, the WFS link will only appear on REST API pages that end with MapServer, i.e. if you click on the individual datasets, you will no longer see the WFS link at the top of the page.

Accessing private data

The previous WFS example accessed data that was publicly available. If the data you want to use is not public, then you will need to use an access token in your URL.

Typically, this is just a matter of including token=YOUR_TOKEN in your bookmark URL, e.g.

https://WEBSITE/arcgis/services/DATASET_CONTAINER/MapServer/WFSServer?request=GetCapabilities&service=WFS&token=YOUR_TOKEN

The hard part is obtaining a suitable ArcGIS access token to use (i.e. replacing YOUR_TOKEN with a valid token). The simplest way is to use generate-token in the ArcGIS REST API.

For example, the following URL should generate an access token valid for one week, if you replace WEBSITE, USERNAME, and PASSWORD with your actual details.

https://WEBSITE/server/rest/generateToken?username=USERNAME&password=PASSWORD&client=requestip&ip=&referer=&expiration=10080&f=json

Note

Be careful to keep your password details secure here. For example, delete the URL from your browser history once you are done.

Access token limitations

Access tokens will last for up to two weeks. So you will regularly need to generate a new token, and update your bookmarks.

In order to use the generateToken REST API, you must use a built-in account in ArcGIS. Otherwise you will likely get an error code response, e.g. 405 (not allowed).

The generateToken API is not supported if your organization uses OAuth to authenticate ArcGIS users, i.e. you login to ArcGIS with the same LDAP username and password you use to access other systems within your organization.

The ArcGIS administrator for your organization may be able to create a new built-in account for you to use. However, this may incur additional licensing costs depending on your ArcGIS setup, as it will require an extra ‘Viewer’ license.

If your organization uses OAuth and you cannot easily create new built-in accounts, then you may still be able to retrieve an access token via OAuth. For example, you may be able to use the ‘Web Developer’ tools in your browser to view your token when you login to ArcGIS normally.

Try contacting your ArcGIS support person for more details on obtaining an access token.

ArcGIS Hub

Anyone can access data that is available in ArcGIS Hub. If your organization uses ArcGIS Online, then it can be configured to make public data available in ArcGIS Hub.

The URL to load ArcGIS Hub data into RiskScape will look something like this:

https://opendata.arcgis.com/api/v3/datasets/DATASET_ID/downloads/data?format=geojson&spatialRefId=4326

However, you need to replace DATASET_ID with the actual ID of the data you are interested in. The hard part is finding out the correct dataset ID to use. You can follow these steps to find a suitable download link:

  1. Go to https://hub.arcgis.com/

  2. Search for the data you are interested in, and then click on the correct search result.

  3. Click on the ‘Download’ icon on the left-hand side of the map. Then click on either the GeoJSON or KML buttons.

  4. Use your web browser’s ‘downloads’ manager to find the URL that was used. E.g. in Firefox you can right-click on the downloaded file and select ‘Copy Download Link’.

  5. You can then use the copied link as the location in a RiskScape bookmark. You will also need to specify either format = geojson or format = kml in your bookmark, based on the download button you used.

Warning

It is unclear whether these ArcGIS Hub download links will pull in the latest data if the underlying dataset is updated. In other words, updates to the underlying dataset may result in a new DATASET_ID, and so your bookmark may continue pointing to an older copy of the data.

Here is an example of a bookmark that uses data downloaded from ArcGIS Hub.

[bookmark Canterbury_stormwater_pipes]
description = Over 400,000 pipes managed by Canterbury regional Council. Refer \
    https://hub.arcgis.com/datasets/ecan::canterbury-stormwater-pipelines
location = https://opendata.arcgis.com/api/v3/datasets/268050a3ebee4fc3861b88122e5b6771_3/downloads/data?format=geojson&spatialRefId=4326
format = geojson

REST API

The final option available is to load data into RiskScape directly from the ArcGIS Online REST API.

Warning

The ArcGIS REST API should only be used in RiskScape for small datasets, i.e. 1000-2000 records or fewer. ArcGIS feature services have a maxRecordCount configured, which defaults to 1000 or 2000, and only that many records will be returned. When using the REST API, you should always check that your bookmark returns all the expected features. You can use riskscape bookmark info or riskscape bookmark evaluate to check this.

Typically, an ArcGIS Online REST API address will look something like:

https://WEBSITE/arcgis/rest/services/DATASET_CONTAINER/MapServer/LAYER_ID

For example, Greater Wellington Regional Council has a dataset containing segments of the Te Araroa trail. You can view this dataset in the ArcGIS REST API here.

We can download this data in GeoJSON format by using the REST ‘query’ API as the bookmark’s location, e.g.

[bookmark Te_Araroa]
location = https://mapping1.gw.govt.nz/arcgis/rest/services/Tracks/Regional_Tracks/MapServer/2/query?where=1%3D1&\
    outFields=*&f=geojson
format = geojson

All we have done here is add query?where=1%3D1&outFields=*&f=geojson onto the end of the ArcGIS REST API for the layer we are interested in. Breaking the URL down, it uses the query REST API to return all attributes (outFields=*) for all records (where=1%3D1, or 1=1) in GeoJSON format (f=geojson).