here.platform.catalog module#

HERE platform catalog abstraction.

class here.platform.catalog.Catalog(hrn: str, platform: Platform, adapter: Adapter | None = None, billing_tag: str | None = None)[source]#

Bases: object

HERE platform catalog abstraction.

add_layer(id: str, layer_type: str, name: str, summary: str, description: str, content_type: str, **details) → Layer[source]#

Adds a layer to the catalog and returns the object of the created layer.

Parameters:

id – ID of the layer to be added
layer_type – layer_type of the layer to be added
name – name of the layer to be added
summary – summary of the layer to be added
description – description of the layer to be added
content_type – content_type of the layer to be added
details – other details of the layer to be added

Returns:

the object of added layer

Raises:

CatalogConfigurationException – if the catalog already contains a layer with ID :param:`id`

compatible_versions(catalog_dependencies: dict, limit: int | None = 1) → Iterator[List[Version]][source]#: Return compatible versions iterator which returns list of Versions. Each Version object contains version and shared_dependencies attribute. :param catalog_dependencies: The catalog dependencies we want to search for :param limit: number of records in api response. :yields: List of class: Version objects.

property configuration: CatalogConfiguration#: The configuration of the catalog

delete_layer(layer_id: str)[source]#

Delete the catalog layer. Supported layer-types: volatile, stream, index.

Parameters:

layer_id – the identifier of the layer

Raises:

CatalogConfigurationException – in case the catalog configuration is not accepted
LayerConfigurationException – in case the layer configuration is not accepted

Usage:

>>> from here.platform import Platform
>>> platform = Platform()
>>> catalog = platform.get_catalog("hrn:here:data::olp-here:example") 
>>> catalog.delete_layer("layer-id") 

first_version() → int | None[source]#

Return first version of the catalog available on the platform.

Returns:: first version number of the catalog if present, None otherwise

get_details() → dict[source]#

Fetch and buffer catalog details from the platform.

Returns:: catalog details

get_layer(layer_id: str) → Layer[source]#

Create and return a Layer object for the given layer ID.

Parameters:: layer_id – layer ID
Returns:: the layer requested
Raises:: ValueError – Unknown layer id

get_layers_version(catalog_version: int) → Dict[str, int][source]#

Return list of all available layers and layer version for a given catalog version. Layer version is always less or equal than the catalog version and it represent the version when the layer was last updated.

Parameters:: catalog_version – An int to represent a catalog version
Returns:: all available layers with their version

get_version(version: int) → Version[source]#

Return details of a single version which is specified.

Parameters:: version – An int to specify the specific version

for which catalog details are needed :return: details of the version specified :raises ValueError:The value of the version is not an integer or when the value is greater than the latest catalog version or less than 0

grant_access(entity_type: str, entity_id: str, action: str)[source]#

Grants access to a catalog to an entity.

Parameters:

entity_type – Type of entity app, user or group.
entity_id – a unique identifier for the given entity type.
action – an action type to grant access - read, write, manage.

has_layer(layer_id: str) → bool[source]#

Check if a layer ID exists in the catalog.

Parameters:: layer_id – layer ID in the catalog.
Returns:: a boolean value indicating if a layer exists

Ingest sensor data in SDII format in a stream layer.

Parameters:

layer_id – the id of the stream layer
partition_id – A unique id to specify the partition
path_or_data – Data to be uploaded in the partition. It must be less than 20 MB
checksum – A base64 encoded SHA-256 hash of the data
trace_id – A unique id to track your request and identify the message in a catalog

Returns:

Trace and message IDs returned by the service

Raises:

ValueError – Data size is greater than the permissible limit of 20 MB

Example:

>>> from here.platform import Platform
>>> platform = Platform()
>>> catalog = platform.get_catalog("catalog_hrn") 
>>> layer_id = "layer_id" 
>>> partition_id = "partition_id" 
>>> data = b'Some valid data in SDII format' 
>>> catalog.ingest_sdii(layer_id, partition_id, data) 

init_publication(layers: List[str] | List[Layer], dependencies: List[VersionDependency] | None = []) → Publication[source]#

Initialize a new publication, indicating which layers will be affected.

Pass the returned Publication to the functions that write to the layers. Call Publication.complete() to complete the work.

Dependencies with other catalogs and relative versions can be published together with the data. Submitting a new publication that affects one or more versioned layer determines the creation of a new catalog version with the dependencies specified.

Parameters:

layers – which layers are affected by the publication
dependencies – list of dependencies to be use in case a new version is created

Returns:

a new publication valid for the layers specified # noqa

latest_version() → int | None[source]#

Return latest version of the catalog available on the platform.

Returns:: latest version number of the catalog if present, None otherwise

list_layers() → List[Layer][source]#

Return a list of all the layers present in the catalog.

Returns:: a list of layers

list_versions(start: int | None = None, end: int | None = None) → List[Version][source]#

Return list of all catalog versions available on the platform, with one dict each.

Parameters:

start – An int to represent start version of catalog, None if unspecified
end – An int to represent end version of catalog, None if unspecified

Returns:

list of all catalog versions beginning at start + 1``and continuing through (and including) ``end

lookup_apis(region: str | None = None) → dict[source]#

Lookup implemented APIs for the given HRN.

Parameters:: region – an Optional param to look up a specific region for a given resource
Returns:: dictionary with API descriptions for the HRN

modify_layer(layer_id: str, **details)[source]#

Modify the catalog layer details.

Parameters:

layer_id – the identifier of the layer
details – Details of the catalog to be modified

The documentation for the API endpoint used can be found here: Update a catalog layer # noqa E501

Usage:

>>> from here.platform import Platform
>>> platform = Platform()
>>> catalog = platform.get_catalog("hrn:here:data::olp-here:example") 
>>> catalog.modify_layer('layer-id', name='updated-name', description='updated-description') 

open_in_portal()[source]#: Open the catalog page on the HERE platform portal.

retrieve_publication(publication_id: str) → Publication[source]#

Retrieves an existing publication based on its ID.

Parameters:: publication_id – the ID for an already active publication.
Returns:: a publication for the ID.
Raises:: ValueError – the publication isn’t in an initialized (or open) state or a layer from the publication isn’t present in the current catalog.

revoke_access(entity_type, entity_id, action)[source]#

Revokes access to a catalog from an entity.

Parameters:

entity_type – Type of entity app, user or group.
entity_id – a unique identifier for the given entity type.
action – an action type to grant access - read, write, manage.

share(entity_type: str, entity_id: str)[source]#

Share a catalog with an entity.

Parameters:

entity_type – Type of entity app, user or a group.
entity_id – a unique identifier for the given entity type.

update_layer(layer_id: str, layer_type: str, name: str, summary: str, description: str, content_type: str, **details)[source]#

Updates the layer with :param:`layer_id` with given layer details.

Parameters:

layer_id – ID of the layer to be updated
layer_type – layer_type of the layer to be updated
name – name of the layer to be updated
summary – summary of the layer to be updated
description – description of the layer to be updated
content_type – content_type of the layer to be updated
details – other details of the layer to be updated

Raises:

CatalogConfigurationException – if the catalog has no layer with the given ID :param:`layer_id`

write_index_layer(layer: str | Layer, path_or_data: str | Path | bytes | pd.DataFrame, timestamp: int | None = None, fields: Dict[str, str | int | bool] = {}, additional_metadata: Dict[str, str] = {}, encode: bool = True, adapter: Adapter | None = None, **kwargs)[source]#

Write data to an index layer.

Parameters:

layer – the layer or ID of the layer
path_or_data – a file name or path or bytes with encoded content. Alternatively, DataFrame is supported only for layers with content-type application/x-parquet when using a GeoPandasAdapter.
timestamp – timestamp
fields – a dict representing the fields of index record for data being uploaded
additional_metadata – a dict of additional metadata about data being uploaded
encode – whether to encode the data through an adapter or store raw bytes
adapter – the Adapter to transform the input data. None to use the default adapter of the catalog.
kwargs – adapter-specific, please consult the documentation of the specific adapter to for the parameters and types it supports

Raises:

ValueError – in case the specified layer is of the wrong type

Example:

>>> from here.platform import Platform
>>> platform = Platform()
>>> catalog = platform.get_catalog(hrn="hrn:here:data::olp-here:write-function-test")
>>> layer_id = "sample_index_layer"
>>> file_path = "~/sample_data.parquet"
>>> fields = {"ingestionTime": 1114, "partition_id": 343433}
>>> catalog.write_index_layer(layer_id, path_or_data=file_path, 
>>>                           fields=fields, encode=False) 

Write data to stream layers of a catalog.

Data for each layer must be encoded according to the content type configured for the layer.

Parameters:

layers_write_info – a nested dict containing layer_ids, partition_ids and encoded data in the form of bytes or file names or paths which will be uploaded.
timestamp – optional timestamp for all the messages: in milliseconds since Unix epoch (1970-01-01T00:00:00 UTC)
inline_data_limit – threshold data size in bytes to decide if inline stream data field should be populated, if data size is less than the inline_data_limit then the data would be added to StreamPartition.data field or else blob would be uploaded and its data_handle will be added to StreamPartition.data_handle field. Default is 819200 bytes.
encode – whether to encode the data or upload raw bytes
adapter – the Adapter to transform the input data. None to use the default adapter of the catalog.

Raises:

ValueError – in case one the layers specified is of the wrong type

Write data to versioned layers of a catalog, creating one single new version.

Data for each layer must be encoded according to the content type configured for the layer.

Parameters:

layers_write_info – a nested dict containing layer_ids, partition_ids and encoded data in the form of bytes or file names or paths which will be uploaded
version_dependencies – A list of version dependencies
encode – whether to encode the data or upload raw bytes
adapter – the Adapter to transform the input data. None to use the default adapter of the catalog.

Raises:

ValueError – in case one the layers specified is of the wrong type

write_volatile_layers(layers_write_info: Mapping[str, Mapping[str | int, str | Path | bytes]])[source]#

Write data to volatile layers of a catalog.

Data for each layer must be encoded according to the content type configured for the layer.

Parameters:: layers_write_info – a nested dict containing layer_ids, partition_ids and encoded data in the form of bytes or file names or paths which will be uploaded.
Raises:: ValueError – in case one the layers specified is of the wrong type

class here.platform.catalog.CatalogConfiguration(json_dict: Dict[str, Any])[source]#

Bases: JsonDictDocument

The configuration of a catalog, including its most significant properties

property automatic_version_deletion: AutomaticVersionDeletion | None#: Specifies the number of versions to keep for the catalog

property billing_tag: List[str] | None#: List of billing tags used for grouping billing records together for the catalog

property coverage: Coverage | None#: The geographic area that this catalog covers

property created: datetime#: Timestamp, in ISO 8601 format, when the catalog was initially created

property description: str | None#: A longer description of the catalog

property hrn: str#: The HERE Resource Name (HRN) of the catalog

property id: str#: The ID of the catalog

property layers: List[LayerConfiguration]#: The layers in the catalog

property name: str#: The name of the catalog

property notifications: Notifications | None#: Indicates whether or not to notify each time the version of the catalog changes

property owner: Owner | None#: The owner of the catalog

property replication: Replication | None#: The replication set for the catalog

property summary: str | None#: The summary of the catalog

property tags: List[str]#: List of user-defined tags applied to the catalog

property version: int#: The version of the catalog configuration

class here.platform.catalog.Dependency(*args, **kwargs)[source]#

Bases: VersionDependency

This class is deprecated. Please use here.platform.model.VersionDependency.

class here.platform.catalog.Version(json_dict: Dict[str, Any])[source]#

Bases: JsonDictDocument

Represents a catalog version with its details.

property dependencies: List[VersionDependency]#: The list of dependencies on external catalogs and their versions

property shared_dependencies: List[VersionDependency]#: The list of dependencies on external catalogs and their versions

property version: int#: The version number