Language Pair Adaptation
Retrieve adapted language pairs
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs" \
-X GET \
-u u_jsmith@example.com_u0VmztKJrwqf:
Example Response:
{
"languagePairs": [
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "training",
"progress": 54
}
],
"options": {
"sourceLanguageId": "",
"targetLanguageId": "",
"state": "",
"sortBy": ""
},
"page": 1,
"perPage": 10,
"totalPages": 1,
"totalItems": 1
}
Retrieve the list of adapted language pairs
HTTP Request
GET /api/v2/adapted-language-pairs
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
sourceLanguageId (optional) |
string | Filters adapted language pairs to those that share the same source language ID | |
targetLanguageId (optional) |
string | Filters adapted language pairs to those that share the same target language ID | |
baselineModel (optional) |
string | Filters adapted language pairs to those that have the given baseline model | |
username (optional) |
string | Filters adapted language pairs to those that were submitted by a given user | |
domain (optional) |
string | Filters adapted language pairs to those that have the given domain | |
model (optional) |
string | Filters adapted language pairs to those that have the given model | |
state (optional) |
string | Filters adapted language pairs to those that share the same state | |
sortBy (optional) |
string | Sorts the returning list by either startDate (descending by queued time) or languagePairId | languagePairId |
Returns
Returns a list of all adapted language pairs filtered by the given parameters
| Name | Type | Description |
|---|---|---|
languagePairs |
array of objects | List of adapted language pairs as an array of AdaptedLanguagePair objects |
page |
int | Page number returned |
perPage |
int | Number of adapted language pairs returned per page |
totalPages |
int | Total number of pages available |
totalItems |
int | Total number of adapted language pairs |
Training Job States
The status of a job is represented by a state. There are twelve possible states.
| State | Description |
|---|---|
dispatching |
job is being dispatched to a training engine |
initializing |
job is beginning training |
training |
job is being trained |
computingBleu |
job is having its bleu score computed |
readyToDeploy |
job has finished training and can be deployed |
deleting |
job is being deleted |
deleted |
job has been deleted |
canceled |
job has been canceled |
failed |
job has failed |
Create an adapted language pair
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs" \
-X POST \
-u u_jsmith@example.com_u0VmztKJrwqf: \
-H "Content-Type: multipart/form-data" \
-F baselineLanguagePairId="EngFra_Generic_SRV_TNMV_8_4_x_1"
-F comments="This is my adapted LP."
-F hostname="controller-host"
-F name="myAdaptedLP"
-F testData="@<file>"
-F trainingData="@<file>"
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "dispatching"
}
Begin training a language-pair for adaptation
HTTP Request
POST /api/v2/adapted-language-pairs
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
name |
string | Alphanumeric, unique name of adapted language pair | |
baselineLanguagePairId |
string | Language pair being adapted | |
trainingData |
file | A single TMX document or a zip file containing multiple TMX documents to use during training | |
testData (optional) |
file | TMX document to use when testing the baseline and adapted LPs to measure the BLEU scores | |
hostname |
string | Name of host to train the language pair on. Host must have the baseline language pair on it. | |
comments (optional) |
string | Optional comments to add to the adapted LP | |
verifySegmentation (optional) |
boolean | Optional flag to tell the trainer to verify that each line has one segment (and to normalize it otherwise) |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
| Name | Type | Description |
|---|---|---|
trainingId |
string | Identifier of training job |
languagePairId |
string | ID of adapted |
profile |
object | Description of training job in the queue (expanded below) |
→ sourceLanguageId |
string | 3-letter code of the source language of the adapted LP |
→ targetLanguageId |
string | 3-letter code of the target language of the adapted LP |
→ baselineLanguagePairId |
string | Language pair being adapted |
→ comments |
string | Optional comments added to the adapted LP |
→ username |
string | username of the creator of the adapted LP |
→ trainingHost |
string | host the language pair is trained on |
input |
object | Statistics of the data used to train the language pair (expanded below) |
→ training |
object | Statistics of the training data |
→ → characters |
object | Character count statistics |
→ → → source |
int | Character totals of source segments |
→ → → target |
int | Character totals of target segments |
→ → words |
object | Word count statistics |
→ → → source |
int | Word totals of source segments |
→ → → target |
int | Word totals of target segments |
→ → translationUnits |
int | Number of translation units used for training data |
→ test |
object | Statistics of the test data |
→ → characters |
object | Character count statistics |
→ → → source |
int | Character totals of source segments |
→ → → target |
int | Character totals of target segments |
→ → words |
object | Word count statistics |
→ → → source |
int | Word totals of source segments |
→ → → target |
int | Word totals of target segments |
→ → translationUnits |
int | Number of translation units used for test data |
→ → autoExtracted |
boolean | True if the test data was automatically extracted from the input training data. If false, test data was supplied. |
→ verifySegmentation |
boolean | True if verification of the one-segment-per-line rule for the input was requested. |
bleu |
object | The bleu score results of the adapted language pair versus the baseline language pair's result (expanded below) |
→ baseline |
float | The bleu score of the baseline language pair |
→ adapted |
float | The bleu score of the adapted language pair |
timestamps |
object | Timestamps of checkpoints in training job progress (expanded below) in UTC |
→ queued |
string | When the training job was created before being submitted to a training engine (when state became dispatching) |
→ trainingStart |
string | When the training job was submitted to a training engine (when state became initializing) |
→ completed |
string | When the training job result was returned from a training engine (when state first became readyToDeploy) |
errorMessage |
string | Error message if state is failed |
state |
string | Training job state (see table) |
hostsDeployedTo |
array of strings | A list of hosts this language pair is currently deployed to |
Analyze training input data
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/analyze" \
-X POST \
-u u_jsmith@example.com_u0VmztKJrwqf: \
-H "Content-Type: multipart/form-data" \
-F baselineLanguagePairId="EngFra_Generic_SRV_TNMV_8_4_x_1"
-F name="myAdaptedLP"
-F testData="@<file>"
-F trainingData="@<file>"
Example Response:
{
"processorErrors": [
"Error processing TMX file /opt/sdl/ets/data/training-queue/EngGer_Adapted-test_SRV_TNM_8_4_x_1/training-set-2.tmx: XML syntax error on line 141: unexpected EOF"
],
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
}
}
Analyze training input data for language-pair adaptation
HTTP Request
POST /api/v2/adapted-language-pairs/analyze
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
name |
string | Alphanumeric, unique name of adapted language pair | |
baselineLanguagePairId |
string | Language pair being adapted | |
trainingData |
file | A single TMX document or a zip file containing multiple TMX documents to use during training | |
testData (optional) |
file | TMX document to use when testing the baseline and adapted LPs to measure the BLEU scores |
Returns
Returns anlaysis on the input data as an AdaptedAnalysisResults object
| Name | Type | Description |
|---|---|---|
processorErrors |
array of objects | List of errors the training data processor encountered during analysis |
input |
object | Statistics of the data used to train the language pair (expanded below) |
→ training |
object | Statistics of the training data |
→ → characters |
object | Character count statistics |
→ → → source |
int | Character totals of source segments |
→ → → target |
int | Character totals of target segments |
→ → words |
object | Word count statistics |
→ → → source |
int | Word totals of source segments |
→ → → target |
int | Word totals of target segments |
→ → translationUnits |
int | Number of translation units used for training data |
→ test |
object | Statistics of the test data |
→ → characters |
object | Character count statistics |
→ → → source |
int | Character totals of source segments |
→ → → target |
int | Character totals of target segments |
→ → words |
object | Word count statistics |
→ → → source |
int | Word totals of source segments |
→ → → target |
int | Word totals of target segments |
→ → translationUnits |
int | Number of translation units used for test data |
→ → autoExtracted |
boolean | True if the test data was automatically extracted from the input training data. If false, test data was supplied. |
→ verifySegmentation |
boolean | True if verification of the one-segment-per-line rule for the input was requested. |
Retrieve information on adapted language pairs
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497" \
-X GET \
-u u_jsmith@example.com_u0VmztKJrwqf:
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "training",
"progress": 54
}
Retrieve information on a given adapted language pair
HTTP Request
GET /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
trainingId |
string | Training ID of the adaptation job |
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
includeTestData (optional) |
boolean | Whether to include test data in the response | false |
includeTrainingData (optional) |
boolean | Whether to include training data in the response | false |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
Update a language pair adaptation
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497" \
-X PUT \
-u u_jsmith@example.com_u0VmztKJrwqf: \
--data-urlencode comment="This is my new adapted LP comment."
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "readyToDeploy"
}
Update a given adapted language pair
HTTP Request
PUT /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
trainingId |
string | Training ID of adapted language pair |
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
comment |
string | The new comment on the given adapted language pair |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
Cancel a language pair adaptation
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497/cancel" \
-X PUT \
-u u_jsmith@example.com_u0VmztKJrwqf:
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "canceled"
}
Cancel the adaptation of a language pair
HTTP Request
PUT /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}/cancel
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
trainingId |
string | Training ID of adapted language pair |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
Deploy an adapted language pair
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497/deploy" \
-X PUT \
-u u_jsmith@example.com_u0VmztKJrwqf: \
-d hostname="controller-host"
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "readyToDeploy",
"hostsDeployedTo": [
"controller-host"
]
}
Deploys an adapted language pair to a given host
HTTP Request
PUT /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}/deploy
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
trainingId |
string | Training ID of adapted language pair |
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
hostname |
string | Name of host |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
Undeploy an adapted language pair
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/undeploy" \
-X PUT \
-u u_jsmith@example.com_u0VmztKJrwqf: \
-d hostname="controller-host"
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "readyToDeploy"
}
Undeploys an adapted language pair from a given host
HTTP Request
PUT /api/v2/adapted-language-pairs/{languagePairId}/undeploy
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
hostname |
string | Name of host |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
Export an adapted language pair
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497/export" \
-X GET \
-u u_jsmith@example.com_u0VmztKJrwqf:
Example Response:
"application/zip"
Exports an adapted language pair from a given host
HTTP Request
GET /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}/export
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
trainingId |
string | Training ID of adapted language pair |
Returns
Returns an archive containing the adapted language pair and supporting metadata
Import an adapted language pair
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/import" \
-X PUT \
-u u_jsmith@example.com_u0VmztKJrwqf: \
-H "Content-Type: multipart/form-data" \
-F input="@<file>"
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "readyToDeploy"
}
Imports an exported adapted language pair
HTTP Request
PUT /api/v2/adapted-language-pairs/import
Request Parameters
| Name | Type | Description | Default Value |
|---|---|---|---|
input |
file | An archive of an adapted LP that was obtained by exporting an adapted LP |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object
Delete an adapted language pair
Example Request:
curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497" \
-X DELETE \
-u u_jsmith@example.com_u0VmztKJrwqf:
Example Response:
{
"trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
"languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
"profile": {
"sourceLanguageId": "eng",
"targetLanguageId": "fra",
"baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
"comments": "",
"username": "jsmith@example.com",
"trainingHost": "controller-host"
},
"input": {
"training": {
"characters": {
"source": 127303,
"target": 118796
},
"words": {
"source": 23685,
"target": 22621
},
"translationUnits": 1306
},
"test": {
"characters": {
"source": 98330,
"target": 91968
},
"words": {
"source": 18386,
"target": 17559
},
"translationUnits": 1000,
"autoExtracted": true
}
},
"bleu": {
"baseline": 25.3,
"adapted": 25.3
},
"timestamps": {
"queued": "2019-02-05 13:52:50.98704786Z",
"trainingStart": "",
"completed": ""
},
"errorMessage": "",
"state": "deleted"
}
Delete a language pair adaptation
HTTP Request
DELETE /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}
Path Parameters
| Name | Type | Description |
|---|---|---|
languagePairId |
string | ID of adapted language pair |
trainingId |
string | Training ID of adapted language pair |
Returns
Returns information on the adapted language pair as an AdaptedLanguagePair object