Connections and Tables

[Internal] List connections by Organization

GET connections/

Note

Authorized Roles: root, manager, backend

Return list of connections created by organization

Response JSON Object
  • warnings (list-of-string) – Optional. List of warnings

  • connections (list-of-object) –

    Inner fields:

    • warnings (list-of-string) – List of warnings

    • connection (enum) – choices: [GCS, S3, SALESFORCE, SAILTHRU, ITERABLE, BIGQUERY, KLAVIYO, SENDLANE, M_PARTICLE, SEGMENT, MOVABLE_INK, ITERABLE_FEED, SHOPIFY, BRAZE] Connection name

    • credentials (object) – Credentials to connect

    • metadata (object) – Meta data for real-time connections

EXAMPLE RESPONSE
  {
      "connections": [
          {
              "connection": "S3",
              "credentials": {
                  "aws_access_key_id": "my-user@test.com",
                  "aws_secret_access_key": "PasSW0rD"
              },
              "metadata": {}
          }
      ]
  }

[Internal] Get connection

GET connections/<str:connection>/

Note

Authorized Roles: root, manager, backend

Return connection

Response JSON Object
  • warnings (list-of-string) – Optional. List of warnings

  • connection (enum) – choices: [GCS, S3, SALESFORCE, SAILTHRU, ITERABLE, BIGQUERY, KLAVIYO, SENDLANE, M_PARTICLE, SEGMENT, MOVABLE_INK, ITERABLE_FEED, SHOPIFY, BRAZE] Connection name

  • credentials (object) – Credentials to connect

  • metadata (object) – Meta data for real-time connections

EXAMPLE RESPONSE
  {
      "connection": "S3",
      "credentials": {
          "aws_access_key_id": "my-user@test.com",
          "aws_secret_access_key": "PasSW0rD"
      },
      "metadata": {}
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if connection id does not exist or if connection is not allowed

  • NotFoundError with error name CONNECTION_NOT_FOUND if the connection does not exist

[Internal] Create or Replace a Connection

PUT connections/<str:connection>/

Note

Authorized Roles: root, manager, backend

Create or replace a connection

Request JSON Object
  • credentials (object) – Credentials to connect

  • metadata (object) – Optional. Meta data for real-time connections

EXAMPLE QUERY BODY
  {
      "credentials": {
          "domain": "fake-domain",
          "client_id": "fake-client_id",
          "client_secret": "fake-client_secret"
      }
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if connection id does not exist or if connection is not allowed or if there are wrong parameters

  • NotFoundError with error name CONNECTION_NOT_FOUND if the connection does not exist

[Internal] Delete connection

DELETE connections/<str:connection>/

Note

Authorized Roles: root, manager, backend

Delete connection

Errors:

  • WrongData with error name WRONG_DATA_TYPE if connection id does not exist or if connection is not allowed

  • NotFoundError with error name CONNECTION_NOT_FOUND if the connection does not exist

[Internal] Test Credentials

POST connections/<str:connection>/test/

Note

Authorized Roles: root, manager, backend

Test credentials

Request JSON Object
  • credentials (object) – Credentials to connect

EXAMPLE QUERY BODY
  {
      "credentials": {
          "domain": "fake-domain",
          "client_id": "fake-client_id",
          "client_secret": "fake-client_secret"
      }
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the credentials are wrong or if there are wrong parameters

[Internal] List tables by Database and Connection [Deprecated]

GET connections/<str:connection>/tables/

Note

Authorized Roles: root, manager, backend

Return list of tables created by Database and connection

Warning

Endpoint Deprecated

Please use Filter Tables.

Response JSON Object
  • warnings (list-of-string) – Optional. List of warnings

  • tables (list-of-object) –

    Inner fields:

    • warnings (list-of-string) – List of warnings

    • created_timestamp (int) – Creation timestamp

    • modified_timestamp (int) – modification timestamp

    • status (enum) – choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

    • name (string) – Table name

    • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

EXAMPLE RESPONSE
  {
      "tables": [
          {
              "name": "table_name_test",
              "resource": "user",
              "status": "available",
              "created_timestamp": 123456789,
              "modified_timestamp": 123456789,
          }
      ]
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if connection id does not exist

[Internal] Create or Partial Update a Table

PATCH connections/<str:connection>/tables/<str:name>/

Note

Authorized Roles: root, manager, backend

Creates a new table if it doesn’t exist, otherwise it will do a shallow partial update, merging the stored table with the new params.

Before creating or partial updating a table, it is recommended to call the Preview and validate updating a Table endpoint to validate the table specifications and test the credentials of the table.

Note: for LOCAL connections the field url will be created and added by gateway-b2b.

Request JSON Object
  • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

  • resource_params (object) – Optional. Resource parameters

  • connection_table_params (object) – Optional. Connection table parameters

  • connection_table_metadata (object) – Optional. Connection table metadata

  • status (enum) – Optional. choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

EXAMPLE QUERY BODY
  {
      "resource": "user",
      "connection_table_params": {
          "file_format": "CSV",
          "options": {
              "compression": "zip"
          }
      },
      "resource_params": {
          "columns_mapping": {
              "user_id": "test_field_user_id"
          }
      },
      "connection_table_metadata": {
          "description": "list of users properties in a csv file"
      }
  }
Request JSON Object
  • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

  • resource_params (object) – Optional. Resource parameters

  • connection_table_params (object) – Optional. Connection table parameters

  • connection_table_metadata (object) – Optional. Connection table metadata

  • status (enum) – Optional. choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

EXAMPLE QUERY BODY FOR ANALYTICS_REPORT RESOURCE
  {
      "resource": "analytics_report",
      "connection_table_params": {
          "file_format": "dat",
          "options": {}
      },
      "resource_params": {
          "columns_mapping": {}
      },
      "connection_table_metadata": {
          "description": "segmentation with ab-test01"
          "segmentation_id": "segm123",
          "start_timestamp": 1634000000.01,
          "end_timestamp": 1635000000.02
      }
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if connection id does not exist or if the table has wrong params

  • NotFoundError with error name CONNECTION_NOT_FOUND if the connection does not exist

[Internal] Get a Table

GET connections/<str:connection>/tables/<str:name>/

Note

Authorized Roles: root, manager, backend

Return table

Response JSON Object
  • warnings (list-of-string) – Optional. List of warnings

  • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

  • resource_params (object) – Optional. Resource parameters

  • connection_table_params (object) – Optional. Connection table parameters

  • connection_table_metadata (object) – Optional. Connection table metadata

  • status (enum) – choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

  • name (string) – Table name

  • connection (enum) – choices: [LOCAL, GCS, S3, SALESFORCE, SAILTHRU, ITERABLE, BIGQUERY, KLAVIYO, SENDLANE, M_PARTICLE, SEGMENT, MOVABLE_INK, ITERABLE_FEED, SHOPIFY, BRAZE] Connection name

  • created_timestamp (int) – Creation timestamp

  • modified_timestamp (int) – modification timestamp

EXAMPLE RESPONSE
  {
      "name": "table_name_test",
      "resource": "user",
      "created_timestamp": 123456789,
      "modified_timestamp": 123456789,
      "connection": "S3",
      "status": "available",
      "connection_table_params": {
          "file_format": "PARQUET",
          "bucket": "s3-bucket-test",
          "options": {
              "compression": "zip"
          }
      },
      "resource_params": {
          "columns_mapping": {
              "age": "customer.age"
          }
      },
      "connection_table_metadata": {
          "description": "list of users properties imported from AWS S3"
      }
  }

Errors:

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

[Internal] Delete a Table

DELETE connections/<str:connection>/tables/<str:name>/

Note

Authorized Roles: root, manager, backend

Delete table

Errors:

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

[Internal] Preview and validate updating a Table

POST connections/<str:connection>/tables/<str:name>/test/

Note

Authorized Roles: root, manager, backend

Run specifications and test the credentials. If the tables exist previously, it will merge both tables first and then validate/set default values/open the table.

Note: for LOCAL connections the field url will be created and added by gateway-b2b.

Request JSON Object
  • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

  • resource_params (object) – Optional. Resource parameters

  • connection_table_params (object) – Optional. Connection table parameters

  • connection_table_metadata (object) – Optional. Connection table metadata

  • status (enum) – Optional. choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

EXAMPLE QUERY BODY
  {
      "resource": "item",
      "connection_table_params": {
          "file_format": "CSV",
          "bucket": "my-test-bucket",
          "options": {
              "compression": "zip"
          }
      },
      "resource_params": {
          "columns_mapping": {}
      },
      "connection_table_metadata": {
          "description": "list of items properties in a csv file"
      }
  }
Response JSON Object
  • warnings (list-of-string) – Optional. List of warnings

  • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

  • resource_params (object) – Optional. Resource parameters

  • connection_table_params (object) – Optional. Connection table parameters

  • connection_table_metadata (object) – Optional. Connection table metadata

  • status (enum) – Optional. choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

  • name (string) – Table name

  • connection (enum) – choices: [LOCAL, GCS, S3, SALESFORCE, SAILTHRU, ITERABLE, BIGQUERY, KLAVIYO, SENDLANE, M_PARTICLE, SEGMENT, MOVABLE_INK, ITERABLE_FEED, SHOPIFY, BRAZE] Connection name

EXAMPLE RESPONSE
  {
      "name": "table name_test.csv",
      "resource": "item",
      "connection": "GCS",
      "connection_table_params": {
          "file_format": "CSV",
          "bucket": "my-test-bucket",
          "options": {
              "compression": "zip",
              "delimiter": ",",
              "escape": "\\",
          }
      },
      "resource_params": {
          "columns_mapping": {
              "item_id": "item_id"
          }
      },
      "connection_table_metadata": {
          "description": "list of items properties in a csv file in google storage"
      }
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the table has wrong params or if there is any problem with the connection

  • NotFoundError with error name RESOURCE_NOT_FOUND if the remote resource does not exist

[Internal] Test a Table

GET connections/<str:connection>/tables/<str:name>/test/

Note

Authorized Roles: root, manager, backend

Validate a table stored and test the credentials.

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the table has wrong params or if there is any problem with the connection

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

  • NotFoundError with error name RESOURCE_NOT_FOUND if the remote resource does not exist

[Internal] Filter Tables

GET tables/

Note

Authorized Roles: root, manager, backend

Return list of tables created by pages.

Using the query params connection and resource to filter the tables.

Query Parameters
  • connection (enum) – Optional. choices: [LOCAL, GCS, S3, SALESFORCE, SAILTHRU, ITERABLE, BIGQUERY, KLAVIYO, SENDLANE, M_PARTICLE, SEGMENT, MOVABLE_INK, ITERABLE_FEED, SHOPIFY, BRAZE] Connection name

  • resource (enum) – Optional. choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

  • page (int) – Optional. [min: 1] Page to be listed (default 1)

  • amt (int) – Optional. [min: 1 max: 64] Amount of tables to return (default 10)

EXAMPLE QUERY PARAMS
  ?page=2&amt=10&connection=S3&resource=user_id
Response JSON Object
  • warnings (list-of-string) – Optional. List of warnings

  • has_next (bool) – Indicates whether or not there are more pages to request

  • next_page (int) – Next page to request

  • tables (list-of-object) –

    Inner fields:

    • warnings (list-of-string) – List of warnings

    • resource (enum) – choices: [item, user, rating, interaction, user_id, user_to_items_reco, item_to_items_reco, analytics_report] Resource that the table represents

    • resource_params (object) – Resource parameters

    • connection_table_params (object) – Connection table parameters

    • connection_table_metadata (object) – Connection table metadata

    • status (enum) – choices: [available, uploading, wrong_credentials, resource_not_found, wrong_specification, unknown_error] Table status

    • name (string) – Table name

    • connection (enum) – choices: [LOCAL, GCS, S3, SALESFORCE, SAILTHRU, ITERABLE, BIGQUERY, KLAVIYO, SENDLANE, M_PARTICLE, SEGMENT, MOVABLE_INK, ITERABLE_FEED, SHOPIFY, BRAZE] Connection name

    • created_timestamp (int) – Creation timestamp

    • modified_timestamp (int) – modification timestamp

EXAMPLE RESPONSE
  {
      "has_next": false,
      "next_page": 3,
      "tables": [
          {
              "name": "user_whitelist_test",
              "connection": "S3",
              "resource": "user_id",
              "status": "available",
              "created_timestamp": 123456789,
              "modified_timestamp": 123456789,
              "connection_table_params": {
                  "file_format": "PARQUET",
                  "bucket": "s3-bucket-test",
                  "options": {
                      "compression": "zip"
                  }
              },
              "resource_params": {
                  "columns_mapping": {
                      "user_id": "customerID"
                  },
                  "whitelist_params": {}
              },
              "connection_table_metadata": {
                  "description": "List of users imported from AWS S3"
              }
          }
      ]
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if connection id does not exist