here.platform.adapter_default module

HERE platform, content adaptation, encoding and decoding to standard Python classes such as list, dictionaries, and protocol buffers Message types.

class here.platform.adapter_default.DefaultAdapter

Bases: Adapter

This adapter transforms data from and to standard Python classes, such as list, dictionaries, and protocol buffers Message types.

An adapter controls the encoding and decoding process of platform data. It transforms data from and to adapter-specific data structure and supports reading, writing, encoding and decoding a variety of MIME content types.

For the list of MIME content types supported when reading and writing a layer with read_* and write_* functions of the Layer and its subclasses, please see documentation of DefaultDecoder and DefaultEncoder.

All the operations involving content passes through an adapter when the parameters encode or decode are True, their default value. These are parameters of the read_* and write_* functions. If a content type is not supported, or if reading or writing raw content is preferred, pass False to skip encoding or decoding and deal with raw bytes instead.

property content_adapter: ContentAdapter

The adapter specialized for content.

property decoder: Decoder

The decoder associated with the adapter.

property encoder: Encoder

The encoder associated with the adapter.

from_feature_ids(feature_ids: Iterator[str], **kwargs) → List[str]

Adapt a sequence of feature identifiers to a list.

Parameters:

• feature_ids – sequence of feature identifiers

• kwargs – unused

Returns:

a list of feature identifiers

from_geo_features(features: Iterator[Feature], **kwargs) → FeatureCollection

Adapt a sequence of geographic features to a GeoJSON FeatureCollection.

Parameters:

• features – sequence of geographic features

• kwargs – unused

Returns:

a single GeoJSON FeatureCollection containing the features

from_index_data(partitions_data: Iterator[Tuple[IndexPartition, bytes]], content_type: str, schema: Schema | None, **kwargs) → Iterator[Tuple[IndexPartition, Any]]

Adapt versioned partition metadata and data to a sequence of class:IndexPartition and decoded data.

Parameters:

• partitions_data – sequence of partition metadata and data from an index layer

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultDecoder

Returns:

sequence of :class:IndexPartition and decoded data

from_index_metadata(partitions: Iterator[IndexPartition], **kwargs) → Iterator[IndexPartition]

Adapt index partition metadata to a sequence of :class:IndexPartition.

Parameters:

• partitions – sequence of partition metadata from an index layer

• kwargs – unused

Returns:

partition metadata as sequence of :class:IndexPartition

from_stream_data(partitions_data: Iterator[Tuple[StreamPartition, bytes]], content_type: str, schema: Schema | None, **kwargs) → Iterator[Tuple[StreamPartition, Any]]

Adapt versioned partition metadata and data to a sequence of class:StreamPartition and decoded data.

Parameters:

• partitions_data – sequence of partition metadata and data from a stream layer

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultDecoder

Returns:

sequence of :class:StreamPartition and decoded data

from_stream_metadata(partitions: Iterator[StreamPartition], **kwargs) → Iterator[StreamPartition]

Adapt stream partition metadata to a sequence of :class:StreamPartition.

Parameters:

• partitions – sequence of partition metadata from a stream layer

• kwargs – unused

Returns:

partition metadata as sequence of :class:StreamPartition

from_versioned_data(partitions_data: Iterator[Tuple[VersionedPartition, bytes]], content_type: str, schema: Schema | None, **kwargs) → Iterator[Tuple[VersionedPartition, Any]]

Adapt versioned partition metadata and data to a sequence of class:VersionedPartition and decoded data.

Parameters:

• partitions_data – sequence of partition metadata and data from a versioned layer

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultDecoder

Returns:

sequence of :class:VersionedPartition and decoded data

from_versioned_metadata(partitions: Iterator[VersionedPartition], **kwargs) → Iterator[VersionedPartition]

Adapt versioned partition metadata to a sequence of :class:VersionedPartition.

Parameters:

• partitions – sequence of partition metadata from a versioned layer

• kwargs – unused

Returns:

partition metadata as sequence of :class:VersionedPartition

from_volatile_data(partitions_data: Iterator[Tuple[VolatilePartition, bytes]], content_type: str, schema: Schema | None, **kwargs) → Iterator[Tuple[VolatilePartition, Any]]

Adapt volatile partition metadata and data to a sequence of class:VolatilePartition and decoded data.

Parameters:

• partitions_data – sequence of partition metadata and data from a volatile layer

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultDecoder

Returns:

sequence of :class:VolatilePartition and decoded data

from_volatile_metadata(partitions: Iterator[VolatilePartition], **kwargs) → Iterator[VolatilePartition]

Adapt volatile partition metadata to a sequence of :class:VolatilePartition.

Parameters:

• partitions – sequence of partition metadata from a volatile layer

• kwargs – unused

Returns:

partition metadata as sequence of :class:VolatilePartition

to_feature_ids(data: Iterable[str], **kwargs) → Iterator[str]

Adapt data to a sequence of feature identifiers.

Parameters:

• data – sequence, list or iterator of feature identifiers

• kwargs – unused

Returns:

sequence of feature identifiers

to_geo_features(data: FeatureCollection | dict | Any | Iterable[Feature | dict | Any], **kwargs) → Iterator[Feature]

Adapt data in a supported format to a sequence of geographic features.

This adapter supports the following formats: - A :class:CollectionFeature or GeoJSON FeatureCollection dictionary or

any object implementing the __geo_interface__ with type="FeatureCollection" as described in https://gist.github.com/sgillies/2217756

• An iterable of :class:Feature or GeoJSON Feature dictionary or any object implementing the __geo_interface__ with type="Feature" as described in https://gist.github.com/sgillies/2217756

Parameters:

• data – a collection of features in one of the supported formats

• kwargs – unused

Returns:

sequence of geographic features

Raises:

ValueError – if data doesn’t match any of the supported formats

to_index_single_data(data: None, content_type: str, schema: Schema | None, **kwargs) → bytes

Adapt data to be stored in an index layer.

Parameters:

• data – data (nothing supported so far)

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultEncoder

Returns:

data encoded for an index layer

Raises:

ValueError – in case the content type is not supported by the adapter # noqa

to_stream_data(layer: StreamLayer, data: Iterable[Tuple[str | int, Any] | Tuple[str | int, Any, int | None]] | Mapping[str | int, Any], content_type: str, schema: Schema | None, timestamp: int | None, **kwargs) → Iterator[Tuple[str | int, bytes, int | None]]

Adapt data from the target format to stream partition metadata and data.

Parameters:

• layer – the layer all the metadata and data belong to

• data – sequence of partition id, content and optional timestamp to adapt

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• timestamp – optional timestamp for all the messages, if none is specified in data: in milliseconds since Unix epoch (1970-01-01T00:00:00 UTC)

• kwargs – adapter-specific, please consult the documentation of the specific adapter to for the parameters and types it supports

Returns:

sequence of partition id, data and timestamp for the stream layer

to_stream_metadata(layer: StreamLayer, partitions: Iterator[StreamPartition], **kwargs) → Iterator[StreamPartition]

Adapt what to publish from the target format to stream partition metadata.

Parameters:

• layer – the layer all the metadata and data belong to

• partitions – sequence of partitions metadata to publish

• kwargs – unused

Returns:

Iterator with the StreamPartition that are adapted

to_versioned_data(layer: VersionedLayer, data: Iterator[Tuple[str | int, Any]] | Mapping[str | int, Any], content_type: str, schema: Schema | None, **kwargs) → Iterator[Tuple[str | int, bytes]]

Adapt data from sequence of partition ids and data to versioned partition id and data.

Parameters:

• layer – the layer all the metadata and data belong to

• data – dictionary or sequence of partition id and content to adapt

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultEncoder

Returns:

sequence of partition id and data for the versioned layer

to_versioned_metadata(layer: VersionedLayer, partitions_update: Iterator[VersionedPartition] | None, partitions_delete: Iterator[str | int] | None, **kwargs) → Tuple[Iterator[VersionedPartition], Iterator[str | int]]

Adapt sequences of VersionedPartition and partition ids to versioned partition metadata and partition ids to update and delete.

Parameters:

• layer – the layer all the metadata and data belong to

• partitions_update – the sequence of partitions metadata to update, if any

• partitions_delete – the sequence of partitions ids to delete, if any

• kwargs – unused

Returns:

tuple of Iterator, the first with the VersionedPartition that have to be updated, the second with the partition ids to delete

to_volatile_data(layer: VolatileLayer, data: Iterator[Tuple[str | int, Any]] | Mapping[str | int, Any], content_type: str, schema: Schema | None, **kwargs) → Iterator[Tuple[str | int, bytes]]

Adapt data from sequence of partition ids and data to volatile partition id and data.

Parameters:

• layer – the layer all the metadata and data belong to

• data – dictionary or sequence of partition id and content to adapt

• content_type – the MIME content type of the layer

• schema – optional Schema of the layer

• kwargs – additional, content-type-specific parameters, see DefaultEncoder

Returns:

sequence of partition id and data for the volatile layer

to_volatile_metadata(layer: VolatileLayer, partitions_update: Iterator[VolatilePartition] | None, partitions_delete: Iterator[str | int] | None, **kwargs) → Tuple[Iterator[VolatilePartition], Iterator[str | int]]

Adapt sequences of VolatilePartition and partition ids to volatile partition metadata and partition ids to update and delete.

Parameters:

• layer – the layer all the metadata and data belong to

• partitions_update – the sequence of partitions metadata to update, if any

• partitions_delete – the sequence of partitions ids to delete, if any

• kwargs – unused

Returns:

tuple of Iterator, the first with the VolatilePartition that have to be updated, the second with the partition ids to delete

class here.platform.adapter_default.DefaultContentAdapter

Bases: ContentAdapter

The default content adapter, that exposes content via dictionaries and lists.

from_objects(fields: type, data: Iterator[object], single_element: bool = False, index_partition: None | str | Callable[[object], Partition] = None, index_id: None | str | Callable[[object], Identifier] = None, index_ref: None | str | Callable[[object], Ref | Iterable[Ref]] = None) → object | List[object] | Dict[Partition, object] | Dict[Identifier, object] | Dict[Partition, Dict[Identifier, object]] | Dict[Ref, List[object]]

Adapt content form an object representation to the target format.

It can optionally perform indexing of objects, based on their partition, identifier and set of references to other objects. Indexing is specified by naming the field of the object that contains the value to index, or by passing a function that calculates that value from the object.

If indexing by object id is enabled, object are indexed by their identifier, nesting them into an outer dictionary with the object identifier used as key:

{ o1_id: o1, o2_id: o2, o3_id: o3, … }

When both partition and object indexing is enabled, object are indexed by partition identifier, nesting them into an outer dictionary with the partition identifiers used as key. This wraps also the dictionary that index objects by their identifiers:

{ p1_id: { o1_id: o1, o2_id: o2, … }, p2_id: { o3_id: o3, … } }

When only indexing by partition is enabled, objects are returned in one single dict: { p1_id: [ o1, o2, … ], p2_id: [ o3, … ] }

When no indexing is enabled, objects are returned in a list: [ o1, o2, o3, … ]

When indexing by reference is enabled, objects are grouped and indexed by the zero, one or more reference they contain. Result is indexed by reference. This may result in the same object present more than once:

{ r1: [ o1, o3, … ], r2: [ o2, o3, … ], r3: [ o1, o2, o4, … ] }

Parameters:

• fields – the fields to extract, as specified by a dataclass. Field names are looked up among the attributes of each object via getattr`. When missing, ``None or equivalent is used. Each field has a type that describes its semantic: it is used to adapt the value to the most appropriate representation for the output format. TypeError is raised in case this is not possible.

• data – the objects to adapt to the target format. Fields not mentioned in fields are discarded. Expected but missing fields and identifiers are considered None. Field values may be of any type compatible with the type declared for the field. Partition ids don’t have to be unique, but they have to be contiguous: all the objects with a given partition identifier must be returned in sequence. Object identifiers, when present, must be unique across the whole content.

• single_element – the data contains exactly one element, the content adapter case use this information to optimize or return a specialized representation

• index_partition – index the content by partition, using the field specified

• index_id – index the content by object identifier, using the field specified

• index_ref – index the content by references, using the field specified. Each object can contain zero, one or more references, and references can be shared among multiple objects.

Returns:

objects indexed as requested. Indexing is implemented with Dict.

Raises:

ValueError: if the fields are not described by a dataclass KeyError: in case partition id, object id or reference is needed but not present TypeError: in case partition or object id is not of type int or string. Also raised in case field values are not of the type declared for the field, or if they can’t be converted to it.

from_tabular(columns, data, geometry_column='geometry')

Convert the given tabular data.

Parameters:

• columns – column names

• data – tabular data

• geometry_column – geometry column string

Returns:

data representation is function of the adapter # noqa

Raises:

NotImplementedError – so far.

class here.platform.adapter_default.DefaultDecoder

Bases: Decoder

The decoder provided by default in case no Adapter is configured for the platform or for a specific catalog.

decode_blob(data: bytes, content_type: str, schema: Schema | None = None, **kwargs)

Decode one single blob of data.

Parameters:

• data – the encoded data

• content_type – the MIME content type to be decoded

• schema – the schema, if the content type requires one

• kwargs – additional, content-type-specific parameters for the decoder:

For JSON (application/json):

For additional info, please see: https://docs.python.org/3/library/json.html?highlight=json#json.loads

For CSV (text/csv):

For additional info, please see: https://docs.python.org/3/library/csv.html#csv.DictReader

For GeoJSON (application/geo+json or application/vnd.geo+json):

For additional info, please see: jazzband/geojson

Returns:

the decoded blob, its type correspond to the type declared in the property decodable_content_types for the content type

Raises:

• UnsupportedContentTypeDecodeException – in case the content type is not decodable

• ValueError – if the schema is mandatory for the content type but missing

• DecodeException – in case the blob can’t be properly decoded # noqa

• SchemaException – in case the schema can’t be used to decode the content # noqa

property supported_content_types: Dict[str, type | Tuple[type, ...]]

Returns:

the dictionary of MIME content types supported when decoding single blobs

with the decode_blob function of this decoder, each with the type of the decoded data.

class here.platform.adapter_default.DefaultEncoder

Bases: Encoder

A content encoder and its capabilities.

encode_blob(data, content_type: str, schema: Schema | None = None, **kwargs) → bytes

Encode one single blob of data.

Parameters:

• data – the data to be encoded, its type corresponds to the type declared in the property supported_content_types for the content type

• content_type – the MIME content type to be encoded

• schema – the schema, if the content type requires one

• kwargs – additional, content-type-specific parameters for the encoder:

For JSON (application/json):

For additional info, please see: https://docs.python.org/3/library/json.html?highlight=json#json.dumps

For CSV (text/csv):

For additional info, please see: https://docs.python.org/3/library/csv.html#csv.DictWriter

For GeoJSON (application/geo+json or application/vnd.geo+json):

For additional info, please see: jazzband/geojson

Returns:

the encoded data

Raises:

• UnsupportedContentTypeEncodeException – in case the content type is not encodable

• ValueError – if the schema is mandatory for the content type but missing

• EncodeException – in case the blob can’t be properly encoded # noqa

• SchemaException – in case the schema can’t be used to encode the content # noqa

property supported_content_types: Dict[str, type | Tuple[type, ...]]

Returns:

the dictionary of MIME content types supported when encoding single blobs

with the encode_blob function of this encoder, each with the type of the encoded data.