Catalogue Streaming plays S3 bucket logger

The plays logger is a mechanism that allows clients to send catalogue play logs via a PARQUET file format and deliver them using a MassiveMusic S3 bucket.

By adhering to these specifications, you'll ensure that your data integrates seamlessly with our system.

If you have any questions or need further assistance, please reach out to the Client Success team.

Prerequisites

Before getting started with the Logger, please ensure that you have the following:

Access credentials provided by us (Access Key and Secret Key) to access our S3 bucket.
Familiarity with the PARQUET file format.
MassiveMusic Partner ID (to be provided by MassiveMusic)

Log file

To ensure seamless integration, pay attention to three critical aspects of your data:

Fields (Columns): The naming, case, spelling, and ordering of the columns in your file must match the exact specifications provided below.
Data Types (Rows): Ensure that the data types in each cell match the specified types listed below.
Values (Rows): The values in your data must adhere to the acceptable and valid formats detailed in this documentation.

Schema and Column Specifications

Your Parquet file should follow this schema with precise column names and data types:

|-- id: BYTE_ARRAY uuid (required)

A globally unique identifier string, produced programmatically by a FUNCTION.

|-- datetimelogged: BYTE_ARRAY (required)

Not empty or NULL.
Date permitted period = time now - 61 days.

|-- userid: BYTE_ARRAY (required)

Not empty, but can be NULL for all records.

|-- trackid: BYTE_ARRAY (required)

Not empty or NULL.
Greater than 0.

|-- releaseid: BYTE_ARRAY (required)

Not empty or NULL.
Greater than 0.

|-- formatid: BYTE_ARRAY (required)

Not empty or NULL.
Greater than 0.

|-- datetimeplayed: BYTE_ARRAY (required)

Not empty or NULL.
Date permitted period = time now - 61 days.

|-- totaltimeplayed: INT64 (required)

Not empty or NULL.
Greater than or equal to 0.

|-- useragent: BYTE_ARRAY (required)

Not empty or NULL.
Maximum 255 characters.

|-- transactionid: BYTE_ARRAY (required)

Not empty, but can be NULL for all records.
Greater than or equal to 0.

|-- partnerid: BYTE_ARRAY (required)

Not empty or NULL.
Greater than or equal to 0.

|-- shopid: BYTE_ARRAY (required)

Not empty or NULL.
Permitted shop ID: Seehere

|-- endreason: BYTE_ARRAY (required)

Not empty or NULL.
Permitted values: "naturalEnd", "userSkipped", or "other".

|-- country: BYTE_ARRAY (required)

Not empty or NULL.
Permitted country code: Seehere

Example Row Representation

Here's an example of a row in your log file, adhering to the specified schema and requirements

{
  "id": "e9fcaaa9-854b-44b7-bbd8-6a5072f286e2",
  "datetimelogged": "2024-02-22T18:16:33.9293255Z",
  "userid": "12345-abcd-6789-efghijk",
  "trackid": "145213707",
  "releaseid": "19699577",
  "formatid": "55",
  "datetimeplayed": "2024-02-21T13:13:15.674",
  "totaltimeplayed": 194,
  "useragent": "Mozilla/5.0 (Linux; Android 7.0; SM)”,
  "transactionid": "null"
  "partnerid": "12345"
  "shopid": "480",
  "endreason": "naturalEnd",
  "country": "us"
}

Delivery to Our S3 Bucket

To deliver the logs to our S3 bucket, please follow these steps:

Access Credentials: Use the provided Access Key and Secret Key to authenticate your requests to our S3 bucket.
Delivery Path: Specify the designated delivery path within our S3 bucket where the logs should be uploaded.

PARQUET files containing logs should be written here:
s3://7digital-glue-data-prod/receipts/radio/(partnerID)/

File Naming Convention:
YYYY-MM-DD_YOURPARTNERIDHERE_NONSUBSTREAM_FILENUMBERINSERIES.parquet
Example: 2024-01-12_12345_NONSUBSTREAM_000000000001.parquet
Frequency: Daily usage

Troubleshooting

If there's an error in one of the files you submit we will notify you.

We will notify you should you be required to resubmit logs, please use the designated delivery paths to submit the revised logs once the error has been fixed.

PARQUET files containing resubmitted logs should be written here:
s3://7digital-glue-data-prod/failures/nonsubstream/(partnerID)/YYYY-MM-DD/

File Naming Convention:
YYYY-MM-DD_NUMBERINSERIES.parquet
Example: 2023-06-12_000000000001.parquet