Connections and Tables¶
[Internal] List connections by Organization¶
-
GET
connections/
¶ Note
Authorized Roles: root, manager, backend
Return list of connections created by organization
- Response JSON Object
warnings (list-of-string) – Optional. List of warnings
connections (list-of-object) –
Inner fields:
warnings (list-of-string) – List of warnings
connection (enum) – choices: [
GCS
,S3
,SALESFORCE
,SAILTHRU
,ITERABLE
,BIGQUERY
,KLAVIYO
,SENDLANE
,M_PARTICLE
,SEGMENT
,MOVABLE_INK
,ITERABLE_FEED
,SHOPIFY
,BRAZE
] Connection namecredentials (object) – Credentials to connect
metadata (object) – Meta data for real-time connections
{ "connections": [ { "connection": "S3", "credentials": { "aws_access_key_id": "my-user@test.com", "aws_secret_access_key": "PasSW0rD" }, "metadata": {} } ] }
[Internal] Get connection¶
-
GET
connections/<str:connection>/
¶ Note
Authorized Roles: root, manager, backend
Return connection
- Response JSON Object
warnings (list-of-string) – Optional. List of warnings
connection (enum) – choices: [
GCS
,S3
,SALESFORCE
,SAILTHRU
,ITERABLE
,BIGQUERY
,KLAVIYO
,SENDLANE
,M_PARTICLE
,SEGMENT
,MOVABLE_INK
,ITERABLE_FEED
,SHOPIFY
,BRAZE
] Connection namecredentials (object) – Credentials to connect
metadata (object) – Meta data for real-time connections
{ "connection": "S3", "credentials": { "aws_access_key_id": "my-user@test.com", "aws_secret_access_key": "PasSW0rD" }, "metadata": {} }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if connection id does not exist or if connection is not allowedNotFoundError with error name
CONNECTION_NOT_FOUND
if the connection does not exist
[Internal] Create or Replace a Connection¶
-
PUT
connections/<str:connection>/
¶ Note
Authorized Roles: root, manager, backend
Create or replace a connection
- Request JSON Object
credentials (object) – Credentials to connect
metadata (object) – Optional. Meta data for real-time connections
{ "credentials": { "domain": "fake-domain", "client_id": "fake-client_id", "client_secret": "fake-client_secret" } }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if connection id does not exist or if connection is not allowed or if there are wrong parametersNotFoundError with error name
CONNECTION_NOT_FOUND
if the connection does not exist
[Internal] Delete connection¶
-
DELETE
connections/<str:connection>/
¶ Note
Authorized Roles: root, manager, backend
Delete connection
Errors:
WrongData with error name
WRONG_DATA_TYPE
if connection id does not exist or if connection is not allowedNotFoundError with error name
CONNECTION_NOT_FOUND
if the connection does not exist
[Internal] Test Credentials¶
-
POST
connections/<str:connection>/test/
¶ Note
Authorized Roles: root, manager, backend
Test credentials
- Request JSON Object
credentials (object) – Credentials to connect
{ "credentials": { "domain": "fake-domain", "client_id": "fake-client_id", "client_secret": "fake-client_secret" } }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if the credentials are wrong or if there are wrong parameters
[Internal] List tables by Database and Connection [Deprecated]¶
-
GET
connections/<str:connection>/tables/
¶ Note
Authorized Roles: root, manager, backend
Return list of tables created by Database and connection
Warning
Endpoint Deprecated
Please use
Filter Tables
.- Response JSON Object
warnings (list-of-string) – Optional. List of warnings
tables (list-of-object) –
Inner fields:
warnings (list-of-string) – List of warnings
created_timestamp (int) – Creation timestamp
modified_timestamp (int) – modification timestamp
status (enum) – choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table statusname (string) – Table name
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table represents
{ "tables": [ { "name": "table_name_test", "resource": "user", "status": "available", "created_timestamp": 123456789, "modified_timestamp": 123456789, } ] }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if connection id does not exist
[Internal] Create or Partial Update a Table¶
-
PATCH
connections/<str:connection>/tables/<str:name>/
¶ Note
Authorized Roles: root, manager, backend
Creates a new table if it doesn’t exist, otherwise it will do a shallow partial update, merging the stored table with the new params.
Before creating or partial updating a table, it is recommended to call the
Preview and validate updating a Table
endpoint to validate the table specifications and test the credentials of the table.Note: for
LOCAL
connections the fieldurl
will be created and added bygateway-b2b
.- Request JSON Object
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representsresource_params (object) – Optional. Resource parameters
connection_table_params (object) – Optional. Connection table parameters
connection_table_metadata (object) – Optional. Connection table metadata
status (enum) – Optional. choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table status
{ "resource": "user", "connection_table_params": { "file_format": "CSV", "options": { "compression": "zip" } }, "resource_params": { "columns_mapping": { "user_id": "test_field_user_id" } }, "connection_table_metadata": { "description": "list of users properties in a csv file" } }
- Request JSON Object
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representsresource_params (object) – Optional. Resource parameters
connection_table_params (object) – Optional. Connection table parameters
connection_table_metadata (object) – Optional. Connection table metadata
status (enum) – Optional. choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table status
{ "resource": "analytics_report", "connection_table_params": { "file_format": "dat", "options": {} }, "resource_params": { "columns_mapping": {} }, "connection_table_metadata": { "description": "segmentation with ab-test01" "segmentation_id": "segm123", "start_timestamp": 1634000000.01, "end_timestamp": 1635000000.02 } }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if connection id does not exist or if the table has wrong paramsNotFoundError with error name
CONNECTION_NOT_FOUND
if the connection does not exist
[Internal] Get a Table¶
-
GET
connections/<str:connection>/tables/<str:name>/
¶ Note
Authorized Roles: root, manager, backend
Return table
- Response JSON Object
warnings (list-of-string) – Optional. List of warnings
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representsresource_params (object) – Optional. Resource parameters
connection_table_params (object) – Optional. Connection table parameters
connection_table_metadata (object) – Optional. Connection table metadata
status (enum) – choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table statusname (string) – Table name
connection (enum) – choices: [
LOCAL
,GCS
,S3
,SALESFORCE
,SAILTHRU
,ITERABLE
,BIGQUERY
,KLAVIYO
,SENDLANE
,M_PARTICLE
,SEGMENT
,MOVABLE_INK
,ITERABLE_FEED
,SHOPIFY
,BRAZE
] Connection namecreated_timestamp (int) – Creation timestamp
modified_timestamp (int) – modification timestamp
{ "name": "table_name_test", "resource": "user", "created_timestamp": 123456789, "modified_timestamp": 123456789, "connection": "S3", "status": "available", "connection_table_params": { "file_format": "PARQUET", "bucket": "s3-bucket-test", "options": { "compression": "zip" } }, "resource_params": { "columns_mapping": { "age": "customer.age" } }, "connection_table_metadata": { "description": "list of users properties imported from AWS S3" } }
Errors:
NotFoundError with error name
TABLE_NOT_FOUND
if the table name does not exist
[Internal] Delete a Table¶
-
DELETE
connections/<str:connection>/tables/<str:name>/
¶ Note
Authorized Roles: root, manager, backend
Delete table
Errors:
NotFoundError with error name
TABLE_NOT_FOUND
if the table name does not exist
[Internal] Preview and validate updating a Table¶
-
POST
connections/<str:connection>/tables/<str:name>/test/
¶ Note
Authorized Roles: root, manager, backend
Run specifications and test the credentials. If the tables exist previously, it will merge both tables first and then validate/set default values/open the table.
Note: for
LOCAL
connections the fieldurl
will be created and added bygateway-b2b
.- Request JSON Object
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representsresource_params (object) – Optional. Resource parameters
connection_table_params (object) – Optional. Connection table parameters
connection_table_metadata (object) – Optional. Connection table metadata
status (enum) – Optional. choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table status
{ "resource": "item", "connection_table_params": { "file_format": "CSV", "bucket": "my-test-bucket", "options": { "compression": "zip" } }, "resource_params": { "columns_mapping": {} }, "connection_table_metadata": { "description": "list of items properties in a csv file" } }
- Response JSON Object
warnings (list-of-string) – Optional. List of warnings
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representsresource_params (object) – Optional. Resource parameters
connection_table_params (object) – Optional. Connection table parameters
connection_table_metadata (object) – Optional. Connection table metadata
status (enum) – Optional. choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table statusname (string) – Table name
connection (enum) – choices: [
LOCAL
,GCS
,S3
,SALESFORCE
,SAILTHRU
,ITERABLE
,BIGQUERY
,KLAVIYO
,SENDLANE
,M_PARTICLE
,SEGMENT
,MOVABLE_INK
,ITERABLE_FEED
,SHOPIFY
,BRAZE
] Connection name
{ "name": "table name_test.csv", "resource": "item", "connection": "GCS", "connection_table_params": { "file_format": "CSV", "bucket": "my-test-bucket", "options": { "compression": "zip", "delimiter": ",", "escape": "\\", } }, "resource_params": { "columns_mapping": { "item_id": "item_id" } }, "connection_table_metadata": { "description": "list of items properties in a csv file in google storage" } }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if the table has wrong params or if there is any problem with the connectionNotFoundError with error name
RESOURCE_NOT_FOUND
if the remote resource does not exist
[Internal] Test a Table¶
-
GET
connections/<str:connection>/tables/<str:name>/test/
¶ Note
Authorized Roles: root, manager, backend
Validate a table stored and test the credentials.
Errors:
WrongData with error name
WRONG_DATA_TYPE
if the table has wrong params or if there is any problem with the connectionNotFoundError with error name
TABLE_NOT_FOUND
if the table name does not existNotFoundError with error name
RESOURCE_NOT_FOUND
if the remote resource does not exist
[Internal] Filter Tables¶
-
GET
tables/
¶ Note
Authorized Roles: root, manager, backend
Return list of tables created by pages.
Using the query params connection and resource to filter the tables.
- Query Parameters
connection (enum) – Optional. choices: [
LOCAL
,GCS
,S3
,SALESFORCE
,SAILTHRU
,ITERABLE
,BIGQUERY
,KLAVIYO
,SENDLANE
,M_PARTICLE
,SEGMENT
,MOVABLE_INK
,ITERABLE_FEED
,SHOPIFY
,BRAZE
] Connection nameresource (enum) – Optional. choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representspage (int) – Optional. [min: 1] Page to be listed (default 1)
amt (int) – Optional. [min: 1 max: 64] Amount of tables to return (default 10)
?page=2&amt=10&connection=S3&resource=user_id
- Response JSON Object
warnings (list-of-string) – Optional. List of warnings
has_next (bool) – Indicates whether or not there are more pages to request
next_page (int) – Next page to request
tables (list-of-object) –
Inner fields:
warnings (list-of-string) – List of warnings
resource (enum) – choices: [
item
,user
,rating
,interaction
,user_id
,user_to_items_reco
,item_to_items_reco
,analytics_report
] Resource that the table representsresource_params (object) – Resource parameters
connection_table_params (object) – Connection table parameters
connection_table_metadata (object) – Connection table metadata
status (enum) – choices: [
available
,uploading
,wrong_credentials
,resource_not_found
,wrong_specification
,unknown_error
] Table statusname (string) – Table name
connection (enum) – choices: [
LOCAL
,GCS
,S3
,SALESFORCE
,SAILTHRU
,ITERABLE
,BIGQUERY
,KLAVIYO
,SENDLANE
,M_PARTICLE
,SEGMENT
,MOVABLE_INK
,ITERABLE_FEED
,SHOPIFY
,BRAZE
] Connection namecreated_timestamp (int) – Creation timestamp
modified_timestamp (int) – modification timestamp
{ "has_next": false, "next_page": 3, "tables": [ { "name": "user_whitelist_test", "connection": "S3", "resource": "user_id", "status": "available", "created_timestamp": 123456789, "modified_timestamp": 123456789, "connection_table_params": { "file_format": "PARQUET", "bucket": "s3-bucket-test", "options": { "compression": "zip" } }, "resource_params": { "columns_mapping": { "user_id": "customerID" }, "whitelist_params": {} }, "connection_table_metadata": { "description": "List of users imported from AWS S3" } } ] }
Errors:
WrongData with error name
WRONG_DATA_TYPE
if connection id does not exist