Background Tasks

Trigger Background Task

POST tasks/<str:task_name>/

Note

Authorized Roles: root, manager, backend

Manually trigger a background task.

Warning

DO NOT OVER-USE

All tasks are triggered automatically including the full retraining of all ML models. You should not use this endpoint regularly.

However the API will not trigger a full update of the models when you update only a tiny portion of the ratings. Use this endpoint if you want to be sure the models are using 100% of the ratings.

Allowed task names: ml_model_retrain, item_popularity_score_recalibrate, ml_new_users, ml_new_items.

Errors:

  • DuplicatedError with error name TASK_ALREADY_RUNNING if this task is already running

  • WrongData with error name WRONG_DATA_TYPE if the task name is wrong

List Recent Background Tasks

GET tasks/<str:task_name>/recents/

Note

Authorized Roles: root, manager, backend

List recent background tasks.

Allowed task names: ml_model_retrain, item_popularity_score_recalibrate, ml_new_users, ml_new_items

Query Parameters

  • page (int) – Optional. [min: 1] Page to be listed (default 1)

  • amt (int) – Optional. [min: 1 max: 64] Amount of tasks to return (default 50)

EXAMPLE QUERY PARAMS
  ?page=2&amt=10
Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • has_next (bool) – Indicates whether or not there are more pages to request

  • next_page (int) – Next page to request

  • tasks (list-of-object) –

    Inner fields

    • warnings (list-of-string) – List of warnings

    • task_id (string) – Background Task ID

    • name (string) – Task name

    • start_time (float64) – [min: -150000000000.0 (year -2786) max: 3500000000.0 (year 2080)] Start timestamp

    • status (enum) – choices: [REQUEST_SENT, RUNNING, FAILED, CANCELLED_REQUESTED, CANCELLED, COMPLETED] Execution status

    • progress (string) – Execution progress message

EXAMPLE RESPONSE
  {
      "has_next": false,
      "next_page": 3,
      "tasks": [
          {
              "task_id": "4xBBB6qkqDbGdlyYRAjF",
              "name": "ml_model_retrain",
              "start_time": 123456789,
              "status": "RUNNING",
              "progress": "trained model 38c6744c"
          }
      ]
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the task name is wrong

[Internal] Trigger Reco Export Task

POST tasks/private/reco_export/

Note

Authorized Roles: root, manager, backend

Manually trigger a reco_export background task.

Request JSON Object

  • context_task (object) – Context task

  • refresh_token (string) – Refresh token

EXAMPLE QUERY BODY
  {
      "context_task": {
          "input_table": {
              "connection": "GCS",
              "name": "input-myfile.csv.gz"
          },
          "reco_params": {
              "amt": 7,
              "filters": ["price:eq:10", "tags:in:family,fiction"]
          },
          "output_table": {
              "connection": "LOCAL",
              "name": "output-reco.parquet"
          }
      },
      "refresh_token": "mW+k/K..."
  }
Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • task_id (string) – Background Task ID

EXAMPLE RESPONSE
  {
      "task_id": "4xBBB6qkqDbGdlyYRAjF",
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the table has wrong params or if there is any problem with the connection

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

  • NotFoundError with error name RESOURCE_NOT_FOUND if the remote resource does not exist

[Internal] Trigger User Id Whitelist Export Task

POST tasks/private/user_id_whitelist_export/

Note

Authorized Roles: root, manager, backend

Manually trigger a user_id_whitelist_export background task.

Request JSON Object

  • context_task (object) – Context task

  • refresh_token (string) – Refresh token

EXAMPLE QUERY BODY
  {
      "context_task": {
          "output_table": {
              "connection": "LOCAL",
              "name": "output-whitelist.parquet"
          }
      },
      "refresh_token": "mW+k/K..."
  }
Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • task_id (string) – Background Task ID

EXAMPLE RESPONSE
  {
      "task_id": "4xBBB6qkqDbGdlyYRAjF"
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the table has wrong params or if there is any problem with the connection

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

  • NotFoundError with error name RESOURCE_NOT_FOUND if the remote resource does not exist

[Internal] Trigger Data Import Task

POST tasks/private/data_import/

Note

Authorized Roles: root, manager, backend

Manually trigger a data_import background task.

Request JSON Object

  • context_task (object) – Context task

  • refresh_token (string) – Refresh token

EXAMPLE QUERY BODY
  {
      "context_task": {
          "connection": "GCS",
          "table_name": "myfile.csv.gz"
      },
      "refresh_token": "mW+k/K..."
  }
Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • task_id (string) – Background Task ID

EXAMPLE RESPONSE
  {
      "task_id": "4xBBB6qkqDbGdlyYRAjF"
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the table has wrong params or if there is any problem with the connection

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

  • NotFoundError with error name RESOURCE_NOT_FOUND if the remote resource does not exist

[Internal] Trigger Analytics Report Export Task

POST tasks/private/analytics_report_export/

Note

Authorized Roles: root, manager, backend

Manually trigger a ‘analytics’ background task. This generates a report, a json document, and exports it. To access the stored document, use the GET storage-signed-urls/local-tables/<str:resource>/download/ endpoint with the analytics_report resource.

Read about analytics here: Analytics (a.k.a. KPIs)

Request JSON Object

  • context_task (object) – Context task

  • refresh_token (string) – Optional. Refresh token [WIP: will become required]

EXAMPLE QUERY BODY
  {
      "context_task": {
          "output_table": {
              "connection": "LOCAL",
              "name": "analytics_report_123xyz"
          }
      }
  }
Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • task_id (string) – Background Task ID

EXAMPLE RESPONSE
  {
      "task_id": "4xBBB6qkqDbGdlyYRAjF"
  }

Errors:

  • NotFoundError with error name TABLE_NOT_FOUND if the table name does not exist

[Internal] List Recent Background Private or Public Tasks

GET tasks/private/<str:task_name>/recents/

Note

Authorized Roles: root, manager, backend

List recent background tasks. Just return the payload within the details parameter.

Allowed task names: ml_model_retrain, item_popularity_score_recalibrate, ml_new_users, ml_new_items, data_import, reco_export, analytics_report_export, 'user_id_whitelist_export.

Query Parameters

  • page (int) – Optional. [min: 1] Page to be listed (default 1)

  • amt (int) – Optional. [min: 1 max: 64] Amount of tasks to return (default 50)

EXAMPLE QUERY PARAMS
  ?page=2&amt=10
Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • has_next (bool) – Indicates whether or not there are more pages to request

  • next_page (int) – Next page to request

  • tasks (list-of-object) –

    Inner fields

    • warnings (list-of-string) – List of warnings

    • task_id (string) – Background Task ID

    • name (string) – Task name

    • start_time (float64) – [min: -150000000000.0 (year -2786) max: 3500000000.0 (year 2080)] Start timestamp

    • status (enum) – choices: [REQUEST_SENT, RUNNING, FAILED, CANCELLED_REQUESTED, CANCELLED, COMPLETED] Execution status

    • progress (string) – Execution progress message

    • details (object) – Task details

EXAMPLE RESPONSE 1
  {
      "has_next": false,
      "next_page": 3,
      "tasks": [
          {
              "task_id": "4xBBB6qkqDbGdlyYRAjF",
              "name": "ml_model_retrain",
              "start_time": 123456789,
              "status": "RUNNING",
              "progress": "trained model 38c6744c",
              "details":{}
          }
      ]
  }
EXAMPLE RESPONSE 2
  {
      "has_next": false,
      "next_page": 2,
      "tasks": [
          {
              "task_id": "7xBBB6qkaDbGdlyYRAjF",
              "name": "data_import",
              "start_time": 123456789,
              "status": "RUNNING",
              "progress": "100 rows uploaded",
              "details": {
                  "payload": {
                      "connection": "LOCAL",
                      "name": "input-myfile.csv"
                  }
              }
          }
      ]
  }
EXAMPLE RESPONSE 3
  {
      "has_next": false,
      "next_page": 2,
      "tasks": [
          {
              "task_id": "4xBBB6qkaDbGdlyYRAjF",
              "name": "reco_export",
              "start_time": 123456789,
              "status": "RUNNING",
              "progress": "uploading 95%",
              "details": {
                  "payload": {
                      "input_table": {
                          "connection": "S3",
                          "name": "input-myfile.csv",
                      },
                      "reco_params": {
                          "amt": 7,
                          "filters": ["price:eq:10", "tags:in:family,fiction"],
                      },
                      "output_table": {
                          "connection": "LOCAL",
                          "name": "output-reco.csv",
                      }
                  }
              }
          }
      ]
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the task name is wrong

[Internal] Get Task

GET tasks/private/<str:task_name>/<str:task_id>/

Note

Authorized Roles: root, manager, backend

Get a background task.

Allowed task names: ml_model_retrain, item_popularity_score_recalibrate, ml_new_users, ml_new_items, data_import, reco_export, analytics_report_export, user_id_whitelist_export.

Response JSON Object

  • warnings (list-of-string) – Optional. List of warnings

  • task_id (string) – Background Task ID

  • name (string) – Task name

  • start_time (float64) – [min: -150000000000.0 (year -2786) max: 3500000000.0 (year 2080)] Start timestamp

  • status (enum) – choices: [REQUEST_SENT, RUNNING, FAILED, CANCELLED_REQUESTED, CANCELLED, COMPLETED] Execution status

  • details (object) – Task details

EXAMPLE RESPONSE 1
  {
      "task_id": "4xBBB6qkqDbGdlyYRAjF",
      "name": "data_import",
      "start_time": 123456789,
      "status": "RUNNING",
      "details": {
          "payload": {
              "connection": "LOCAL",
              "name": "input-myfile.csv"
          },
          "progress": {
              "rows": 40,
              "columns": ["item_id", "prop_float", "prop_int", "prop_tags_id"],
              "message": "40 rows uploaded",
              "resource": "ITEM",
              "extension": "csv",
              "connection": "LOCAL",
              "table_name": "input-myfile.csv",
              "compression": "N/A",
              "init_time_stamp": 1626057538
          },
          "timeout_at": 1626143934.2895942,
          "create_reason": "manually triggered by endpoint"
      }
  }
EXAMPLE RESPONSE 2
  {
      "task_id": "4xBBB6qkqDbGdlyYRAjF",
      "name": "reco_export",
      "start_time": 123456789,
      "status": "RUNNING",
      "details": {
          "payload": {
              "input_table": {
                  "name": "users-whitelist-test.csv",
                  "connection": "LOCAL"
              },
              "reco_params": {
                  "amt": 8
              },
              "output_table": {
                  "name": "out-recos.csv",
                  "connection": "LOCAL"
              }
          },
          "progress": {
              "message": "Computing and uploading recommendations",
              "n_total": 44,
              "n_failed": 0,
              "has_counts": true,
              "n_warnings": 0,
              "n_processed": 44
          },
          "timeout_at": 1628182939.94221,
          "create_reason": "manually triggered by endpoint"
      }
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the task name is wrong

  • NotFoundError with error name TASK_NOT_FOUND if no task with the given task_id can be found

[Internal] Update Task

PATCH tasks/private/<str:task_name>/<str:task_id>/

Note

Authorized Roles: root, manager, backend

Update a background task.

Allowed task names: ml_model_retrain, item_popularity_score_recalibrate, ml_new_users, ml_new_items, data_import, reco_export, analytics_report_export, user_id_whitelist_export.

Request JSON Object

  • status (enum) – Optional. choices: [FAILED, CANCELLED_REQUESTED, CANCELLED, COMPLETED]

  • details (object) – Optional. Task details

  • completed (bool) – Optional. Flag to enter that the task was completed

  • failed (bool) – Optional. Flag to enter that the task was failed

EXAMPLE QUERY BODY
  {
      "status": "RUNNING",
      "details": {
          "progress": {
              "message": "uploading 95%"
          }
      }
  }

or

EXAMPLE QUERY BODY
  {
      "completed": true
  }

Errors:

  • WrongData with error name WRONG_DATA_TYPE or LOWER_STATUS_NOT_ALLOWED if the task name or status are wrong

  • NotFoundError with error name TASK_NOT_FOUND if no task with the given task_id can be found

[Internal] Cancel Task

DELETE tasks/private/<str:task_name>/<str:task_id>/

Note

Authorized Roles: root, manager, backend

Request to cancel a background task.

Allowed task names: ml_model_retrain, item_popularity_score_recalibrate, ml_new_users, ml_new_items, data_import, reco_export, analytics_report_export, user_id_whitelist_export.

Errors:

  • WrongData with error name WRONG_DATA_TYPE if the task name is wrong

  • NotFoundError with error name TASK_NOT_FOUND if no task with the given task_id can be found