Overview
This document explains how to structure and deliver your data exports to Ktrl's S3 bucket. Once set up, your system will drop files here on a regular cadence and Ktrl will ingest them automatically.
Before starting, make sure your data matches the schema defined in CLF Specification.
How it works
Each export is a complete snapshot of your data - not just changes since the last export. This prevents data drift as values evolve over time, and ensures forecasts are always based on the full picture.
Folder structure
Export files to your S3 bucket using this path format:
<YOUR_PATH>/YYYY_MM_DD_hh_mm/product_name*.parquet
The bucket path itself can be anything, but the timestamped folder and filename pattern must match exactly.
GCS integrations
If your games data isn't stored in the AWS and you would prefer to build the CLF data in GCS, then you can do so! Creating a BigQuery table that has the CLF data in and then create a scheduled query in the following format
EXPORT DATA
OPTIONS ( uri = CONCAT('gs://<INSERT_YOUR_S3_PATH>',
FORMAT_DATE('%Y_%m_%d_%H_%M', CURRENT_DATETIME()),
'/*.parquet'),
format='PARQUET',
compression='SNAPPY',
overwrite=TRUE ) AS
select * FROM <CLF_DATA>
Where CLF_DATA is the table that you have stored the data in.
Connecting your S3 bucket to Ktrl
Once your CLF data is in your S3 bucket and structured correctly, you're ready to connect it to Ktrl.
Choose your data source - In the Ktrl setup wizard, select Bespoke Data via S3.
Enter your S3 credentials - You'll be prompted for the following:
Access key ID and secret access key - from the AWS user with read access to your bucket
S3 directory path - the folder path to your CLF data, without the
s3://prefixRegion - the AWS region your bucket is in (e.g.
eu-west-1)
Validate and continue - Ktrl will test the connection and confirm it can read your data. Once validated, continue through the rest of onboarding as normal.