RWS Logo
Show / Hide Table of Contents

Language Pair Adaptation

Retrieve adapted language pairs

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs" \
	-X GET \
	-u u_jsmith@example.com_u0VmztKJrwqf:

Example Response:

{
    "languagePairs": [
        {
            "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
            "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
            "profile": {
                "sourceLanguageId": "eng",
                "targetLanguageId": "fra",
                "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
                "comments": "",
                "username": "jsmith@example.com",
                "trainingHost": "controller-host"
            },
            "input": {
                "training": {
                    "characters": {
                        "source": 127303,
                        "target": 118796
                    },
                    "words": {
                        "source": 23685,
                        "target": 22621
                    },
                    "translationUnits": 1306
                },
                "test": {
                    "characters": {
                        "source": 98330,
                        "target": 91968
                    },
                    "words": {
                        "source": 18386,
                        "target": 17559
                    },
                    "translationUnits": 1000,
                    "autoExtracted": true
                }
            },
            "bleu": {
                "baseline": 25.3,
                "adapted": 25.3
            },
            "timestamps": {
                "queued": "2019-02-05 13:52:50.98704786Z",
                "trainingStart": "",
                "completed": ""
            },
            "errorMessage": "",
            "state": "training",
            "progress": 54
        }
    ],
    "options": {
        "sourceLanguageId": "",
        "targetLanguageId": "",
        "state": "",
        "sortBy": ""
    },
    "page": 1,
    "perPage": 10,
    "totalPages": 1,
    "totalItems": 1
}

Retrieve the list of adapted language pairs

HTTP Request

GET /api/v2/adapted-language-pairs

Request Parameters

These parameters should be added to the URL as a query string.
Name Type Description Default Value
sourceLanguageId (optional) string Filters adapted language pairs to those that share the same source language ID
targetLanguageId (optional) string Filters adapted language pairs to those that share the same target language ID
baselineModel (optional) string Filters adapted language pairs to those that have the given baseline model
username (optional) string Filters adapted language pairs to those that were submitted by a given user
domain (optional) string Filters adapted language pairs to those that have the given domain
model (optional) string Filters adapted language pairs to those that have the given model
state (optional) string Filters adapted language pairs to those that share the same state
sortBy (optional) string Sorts the returning list by either startDate (descending by queued time) or languagePairId languagePairId

Returns

Returns a list of all adapted language pairs filtered by the given parameters

Name Type Description
languagePairs array of objects List of adapted language pairs as an array of AdaptedLanguagePair objects
page int Page number returned
perPage int Number of adapted language pairs returned per page
totalPages int Total number of pages available
totalItems int Total number of adapted language pairs

Training Job States

The status of a job is represented by a state. There are twelve possible states.

State Description
dispatching job is being dispatched to a training engine
initializing job is beginning training
training job is being trained
computingBleu job is having its bleu score computed
readyToDeploy job has finished training and can be deployed
deleting job is being deleted
deleted job has been deleted
canceled job has been canceled
failed job has failed

Create an adapted language pair

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs" \
	-X POST \
	-u u_jsmith@example.com_u0VmztKJrwqf: \
	-H "Content-Type: multipart/form-data" \
	-F baselineLanguagePairId="EngFra_Generic_SRV_TNMV_8_4_x_1"
	-F comments="This is my adapted LP."
	-F hostname="controller-host"
	-F name="myAdaptedLP"
	-F testData="@<file>"
	-F trainingData="@<file>"

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "dispatching"
}

Begin training a language-pair for adaptation

HTTP Request

POST /api/v2/adapted-language-pairs

Request Parameters

Name Type Description Default Value
name string Alphanumeric, unique name of adapted language pair
baselineLanguagePairId string Language pair being adapted
trainingData file A single TMX document or a zip file containing multiple TMX documents to use during training
testData (optional) file TMX document to use when testing the baseline and adapted LPs to measure the BLEU scores
hostname string Name of host to train the language pair on. Host must have the baseline language pair on it.
comments (optional) string Optional comments to add to the adapted LP
verifySegmentation (optional) boolean Optional flag to tell the trainer to verify that each line has one segment (and to normalize it otherwise)

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Name Type Description
trainingId string Identifier of training job
languagePairId string ID of adapted
profile object Description of training job in the queue (expanded below)
→ sourceLanguageId string 3-letter code of the source language of the adapted LP
→ targetLanguageId string 3-letter code of the target language of the adapted LP
→ baselineLanguagePairId string Language pair being adapted
→ comments string Optional comments added to the adapted LP
→ username string username of the creator of the adapted LP
→ trainingHost string host the language pair is trained on
input object Statistics of the data used to train the language pair (expanded below)
→ training object Statistics of the training data
→ → characters object Character count statistics
→ → → source int Character totals of source segments
→ → → target int Character totals of target segments
→ → words object Word count statistics
→ → → source int Word totals of source segments
→ → → target int Word totals of target segments
→ → translationUnits int Number of translation units used for training data
→ test object Statistics of the test data
→ → characters object Character count statistics
→ → → source int Character totals of source segments
→ → → target int Character totals of target segments
→ → words object Word count statistics
→ → → source int Word totals of source segments
→ → → target int Word totals of target segments
→ → translationUnits int Number of translation units used for test data
→ → autoExtracted boolean True if the test data was automatically extracted from the input training data. If false, test data was supplied.
→ verifySegmentation boolean True if verification of the one-segment-per-line rule for the input was requested.
bleu object The bleu score results of the adapted language pair versus the baseline language pair's result (expanded below)
→ baseline float The bleu score of the baseline language pair
→ adapted float The bleu score of the adapted language pair
timestamps object Timestamps of checkpoints in training job progress (expanded below) in UTC
→ queued string When the training job was created before being submitted to a training engine (when state became dispatching)
→ trainingStart string When the training job was submitted to a training engine (when state became initializing)
→ completed string When the training job result was returned from a training engine (when state first became readyToDeploy)
errorMessage string Error message if state is failed
state string Training job state (see table)
hostsDeployedTo array of strings A list of hosts this language pair is currently deployed to

Analyze training input data

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/analyze" \
	-X POST \
	-u u_jsmith@example.com_u0VmztKJrwqf: \
	-H "Content-Type: multipart/form-data" \
	-F baselineLanguagePairId="EngFra_Generic_SRV_TNMV_8_4_x_1"
	-F name="myAdaptedLP"
	-F testData="@<file>"
	-F trainingData="@<file>"

Example Response:

{
    "processorErrors": [
        "Error processing TMX file /opt/sdl/ets/data/training-queue/EngGer_Adapted-test_SRV_TNM_8_4_x_1/training-set-2.tmx: XML syntax error on line 141: unexpected EOF"
    ],
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    }
}

Analyze training input data for language-pair adaptation

HTTP Request

POST /api/v2/adapted-language-pairs/analyze

Request Parameters

Name Type Description Default Value
name string Alphanumeric, unique name of adapted language pair
baselineLanguagePairId string Language pair being adapted
trainingData file A single TMX document or a zip file containing multiple TMX documents to use during training
testData (optional) file TMX document to use when testing the baseline and adapted LPs to measure the BLEU scores

Returns

Returns anlaysis on the input data as an AdaptedAnalysisResults object

Name Type Description
processorErrors array of objects List of errors the training data processor encountered during analysis
input object Statistics of the data used to train the language pair (expanded below)
→ training object Statistics of the training data
→ → characters object Character count statistics
→ → → source int Character totals of source segments
→ → → target int Character totals of target segments
→ → words object Word count statistics
→ → → source int Word totals of source segments
→ → → target int Word totals of target segments
→ → translationUnits int Number of translation units used for training data
→ test object Statistics of the test data
→ → characters object Character count statistics
→ → → source int Character totals of source segments
→ → → target int Character totals of target segments
→ → words object Word count statistics
→ → → source int Word totals of source segments
→ → → target int Word totals of target segments
→ → translationUnits int Number of translation units used for test data
→ → autoExtracted boolean True if the test data was automatically extracted from the input training data. If false, test data was supplied.
→ verifySegmentation boolean True if verification of the one-segment-per-line rule for the input was requested.

Retrieve information on adapted language pairs

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497" \
	-X GET \
	-u u_jsmith@example.com_u0VmztKJrwqf:

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "training",
    "progress": 54
}

Retrieve information on a given adapted language pair

HTTP Request

GET /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair
trainingId string Training ID of the adaptation job

Request Parameters

These parameters should be added to the URL as a query string.
Name Type Description Default Value
includeTestData (optional) boolean Whether to include test data in the response false
includeTrainingData (optional) boolean Whether to include training data in the response false

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Update a language pair adaptation

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497" \
	-X PUT \
	-u u_jsmith@example.com_u0VmztKJrwqf: \
	--data-urlencode comment="This is my new adapted LP comment."

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "readyToDeploy"
}

Update a given adapted language pair

HTTP Request

PUT /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair
trainingId string Training ID of adapted language pair

Request Parameters

Name Type Description Default Value
comment string The new comment on the given adapted language pair

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Cancel a language pair adaptation

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497/cancel" \
	-X PUT \
	-u u_jsmith@example.com_u0VmztKJrwqf:

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "canceled"
}

Cancel the adaptation of a language pair

HTTP Request

PUT /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}/cancel

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair
trainingId string Training ID of adapted language pair

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Deploy an adapted language pair

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497/deploy" \
	-X PUT \
	-u u_jsmith@example.com_u0VmztKJrwqf: \
	-d hostname="controller-host"

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "readyToDeploy",
    "hostsDeployedTo": [
        "controller-host"
    ]
}

Deploys an adapted language pair to a given host

HTTP Request

PUT /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}/deploy

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair
trainingId string Training ID of adapted language pair

Request Parameters

Name Type Description Default Value
hostname string Name of host

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Undeploy an adapted language pair

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/undeploy" \
	-X PUT \
	-u u_jsmith@example.com_u0VmztKJrwqf: \
	-d hostname="controller-host"

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "readyToDeploy"
}

Undeploys an adapted language pair from a given host

HTTP Request

PUT /api/v2/adapted-language-pairs/{languagePairId}/undeploy

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair

Request Parameters

Name Type Description Default Value
hostname string Name of host

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Export an adapted language pair

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497/export" \
	-X GET \
	-u u_jsmith@example.com_u0VmztKJrwqf:

Example Response:

"application/zip"

Exports an adapted language pair from a given host

HTTP Request

GET /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}/export

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair
trainingId string Training ID of adapted language pair

Returns

Returns an archive containing the adapted language pair and supporting metadata

Import an adapted language pair

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/import" \
	-X PUT \
	-u u_jsmith@example.com_u0VmztKJrwqf: \
	-H "Content-Type: multipart/form-data" \
	-F input="@<file>"

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "readyToDeploy"
}

Imports an exported adapted language pair

HTTP Request

PUT /api/v2/adapted-language-pairs/import

Request Parameters

Name Type Description Default Value
input file An archive of an adapted LP that was obtained by exporting an adapted LP

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

Delete an adapted language pair

Example Request:

curl "https://controller-host:8001/api/v2/adapted-language-pairs/EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0/training-jobs/06f0be27-96f0-4961-a630-49372f21f497" \
	-X DELETE \
	-u u_jsmith@example.com_u0VmztKJrwqf:

Example Response:

{
    "trainingId": "06f0be27-96f0-4961-a630-49372f21f497",
    "languagePairId": "EngFra_Adapted-myAdaptedLP_SRV_PB_8_0_x_0",
    "profile": {
        "sourceLanguageId": "eng",
        "targetLanguageId": "fra",
        "baselineLanguagePairId": "EngFra_Generic_SRV_TNMV_8_4_x_1",
        "comments": "",
        "username": "jsmith@example.com",
        "trainingHost": "controller-host"
    },
    "input": {
        "training": {
            "characters": {
                "source": 127303,
                "target": 118796
            },
            "words": {
                "source": 23685,
                "target": 22621
            },
            "translationUnits": 1306
        },
        "test": {
            "characters": {
                "source": 98330,
                "target": 91968
            },
            "words": {
                "source": 18386,
                "target": 17559
            },
            "translationUnits": 1000,
            "autoExtracted": true
        }
    },
    "bleu": {
        "baseline": 25.3,
        "adapted": 25.3
    },
    "timestamps": {
        "queued": "2019-02-05 13:52:50.98704786Z",
        "trainingStart": "",
        "completed": ""
    },
    "errorMessage": "",
    "state": "deleted"
}

Delete a language pair adaptation

HTTP Request

DELETE /api/v2/adapted-language-pairs/{languagePairId}/training-jobs/{trainingId}

Path Parameters

Name Type Description
languagePairId string ID of adapted language pair
trainingId string Training ID of adapted language pair

Returns

Returns information on the adapted language pair as an AdaptedLanguagePair object

In this page
Back to top