climada_petals.entity.exposures.openstreetmap package#

climada_petals.entity.exposures.openstreetmap.osm_dataloader module#

class climada_petals.entity.exposures.openstreetmap.osm_dataloader.OSMRaw[source]#

Bases: object

functions to obtain entire raw datasets from OSM, from different sources

__init__()[source]#
get_data_geofabrik(iso3, file_format='pbf', save_path=PosixPath('/home/docs/climada/data/openstreetmap'))[source]#

Download country files with all OSM map info from the provider Geofabrik.de, if doesn’t exist, yet.

Parameters:

  • iso3 (str) – ISO3 code of country to download Exceptions: Russia is divided into European and Asian part (‘RUS-E’, ‘RUS-A’), Canary Islands are ‘IC’.

  • file_format (str) – Format in which file should be downloaded; options are ESRI Shapefiles (shp), which can easily be loaded into gdfs, or osm-Protocolbuffer Binary Format (pbf), which is smaller in size, but has a more complicated query syntax to load (functions are provided in the OSMFileQuery class).

  • save_path (str or pathlib.Path) – Folder in which to save the file

Returns:

File is downloaded and stored under save_path + the Geofabrik filename

Return type:

None

See also

DICT_GEOFABRIK

get_data_planet(save_path=PosixPath('/home/docs/climada/data/openstreetmap/planet-latest.osm.pbf'))[source]#

Download the entire planet file from the OSM server (ca. 60 GB).

Parameters:

save_path (str or pathlib.Path)

get_data_planetextract(shape, path_extract, path_planet=PosixPath('/home/docs/climada/data/openstreetmap/planet-latest.osm.pbf'), overwrite=False)[source]#

get OSM raw data from a custom shape / bounding-box, which is extracted from the entire OSM planet file. Accepts bbox lists or .poly files for non-rectangular shapes.

Parameters:

  • shape (list or str) – bounding box [xmin, ymin, xmax, ymax] or file path to a .poly file

  • path_extract (str or pathlib.Path) – file path (incl. name & ending) under which extract will be stored

  • path_planet (str or pathlib.Path) – file path to planet-latest.osm.pbf. Will download & store it as indicated, if doesn`t yet exist. Default is DATA_DIR/planet-latest.osm.pbf

Note

For more info on what .poly files are (incl. several tools for creating them), see https://wiki.openstreetmap.org/wiki/Osmosis/Polygon_Filter_File_Format

For creating .poly files on admin0 to admin3 levels of any place on the globe, see the GitHub repo ElcoK/osm_clipper (especially the function make_poly_file())

Note

This function uses the command line tool osmosis to cut out new osm.pbf files from the original ones. Installation instructions (windows, linux, apple) - see https://wiki.openstreetmap.org/wiki/Osmosis/Installation

get_data_fileextract(shape, path_extract, path_parentfile, overwrite=False)[source]#

Extract a geographic sub-set from a raw osm-pbf file.

Note

The shape must be entirely contained within the file to extract from, else it will yield weird results.

Parameters:

  • shape (list or str) – bounding box [xmin, ymin, xmax, ymax] or file path to a .poly file

  • path_extract (str or pathlib.Path) – file path (incl. name & ending) under which extract will be stored

  • path_parentfile (str or pathlib.Path) – file path to parentfile.osm.pbf from which the shape will be cut out

  • overwrite (bool) – default is False. Whether to overwrite files if they already exist.

Note

This function uses the command line tool osmosis to cut out new osm.pbf files from the original ones. Installation instructions (windows, linux, apple) - see https://wiki.openstreetmap.org/wiki/Osmosis/Installation

class climada_petals.entity.exposures.openstreetmap.osm_dataloader.OSMFileQuery(osm_path)[source]#

Bases: object

Load features from raw osm.pbf files.

__init__(osm_path)[source]#
Parameters:

osm_path (str or pathlib.Path) – file path to the .osm.pbf file to extract info from.

Raises:

ValueError – if the given path is not a file

retrieve(geo_type, osm_keys, osm_query)[source]#

Function to extract geometries and tag info for entires in the OSM file matching certain OSM key-value constraints. from an OpenStreetMap osm.pbf file. adapted from BenDickens/trails repo (BenDickens/trails.git, see extract.py)

Parameters:

  • geo_type (str) – Type of geometry to retrieve. One of [points, lines, multipolygons]

  • osm_keys (list) – a list with all the osm keys that should be reported as columns in the output gdf.

  • osm_query (str) – query string of the syntax “key(=’value’) (and/or further queries)”. See examples in DICT_CIS_OSM in case of doubt.

Returns:

A gdf with all results from the osm.pbf file matching the specified constraints.

Return type:

gpd.GeoDataFrame

Note

1) The keys that are searchable are specified in the osmconf.ini file. Make sure that they exist in the attributes=… paragraph under the respective geometry section. For example, to retrieve multipolygons with building=’yes’, building must be in the attributes under the [multipolygons] section of the file. You can find it in the same folder as the osm_dataloader.py module is located. 2) OSM keys that have : in their name must be changed to _ in the search dict, but not in the osmconf.ini E.g. tower:type is called tower_type, since it would interfere with the SQL syntax otherwise, but still tower:type in the osmconf.ini

See also

https

//taginfo.openstreetmap.org/ to check what keys and key/value

pairs

https

//overpass-turbo.eu/ for a direct visual output of the query,

and, correct

retrieve_cis(ci_type)[source]#

A wrapper around retrieve() to conveniently retrieve map info for a selection of critical infrastructure types from the given osm.pbf file. No need to search for osm key/value tags and relevant geometry types.

Parameters:

ci_type (str) – one of DICT_CIS_OSM.keys(), i.e. ‘education’, ‘healthcare’, ‘water’, ‘telecom’, ‘road’, ‘rail’, ‘air’, ‘gas’, ‘oil’, ‘power’, ‘wastewater’, ‘food’

See also

DICT_CIS_OSM, CIs.

class climada_petals.entity.exposures.openstreetmap.osm_dataloader.OSMApiQuery(area, condition)[source]#

Bases: object

Queries features directly via the overpass turbo API.

area: tuple (xmin, ymin, xmax, ymax), list [xmin, ymin, xmax, ymax]

or shapely.geometry.Polygon

query: str

must be of format ‘[“key”]’ or ‘[“key”=”value”]’, etc.

__init__(area, condition)[source]#
get_data_overpass(closed_lines_are_polys=True)[source]#

wrapper for all helper funcs to get & assemble data