Decorators

ratelimit

Apply rate limiting at a threshold per second

ratelimit(max_per_second)

Parameters:max_per_second (float) – Number of permitted hits per second

schema_transform

Apply a field name transformation to a data output from the wrapped function, such that specified field names are transformed and unspecified fields are dropped. A valid file would be formatted as shown:

[{“tier_0”: “bad_col”, “tier_1”: “good_col”}, {“tier_0”: “another_bad_col”, “tier_1”: “another_good_col”}, …]

where tier_0 and tier_1 correspond to from_key and to_key in the below documentation.

load_transformer(filename, from_key, to_key)

schema_transform(filename, from_key, to_key)

Parameters:

• filename (str) – A record-oriented JSON file path mapping field names denoted by from from_key and to_key.
• from_key (str) – The key in file indicated by filename which indicates the field name to transform.
• to_key (str) – The key in file indicated by filename which what the field name indicated by from_key will be transformed to.

Returns:

Data in the format it was originally passed to the wrapper in, with specified field names transformed and unspecified fields dropped.

schema_transformer(data, *, filename, from_key, to_key, ignore=[])

Function version of the schema_transformer wrapper. :param data: the data requiring the schama transformation :type data: dataframe OR list of dicts :param filename: the path to the schema json file :type filename: str :param from_key: tier level of the data :type from_key: str :param to_key: tier level to be applied to the data :type to_key: str :param ignore: optional list of fields, eg ids or keys which shouldn’t be dropped :type ignore: list

Returns:supplied data with schema applied