作业操作接口
更新时间:2022-04-12
添加作业
接口描述
向指定的集群批量添加自定义作业
权限说明
请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求,请参考 鉴权认证。
注意事项
如果请求中没有用户验证信息(即匿名访问),返回403 Forbidden
,错误信息:AccessDenied
。
请求结构
POST /v{version}/cluster/{clusterId}/step?clientToken={clientToken} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}
{
"steps": [
{
"actionOnFailure": "Continue",
"name": "my-mapreduce-streaming",
"properties": {
"arguments": "-D mapreduce.job.reduces=20",
"input": "bos://mybucket/inputs/txt-1m",
"mapper": "cat",
"output": "bos://mybucket/outputs/streaming",
"reducer": "cat"
},
"type": "Streaming",
"additionalFiles":[]
},
{
"actionOnFailure": "Continue",
"name": "my-java-program",
"properties": {
"arguments": "bos://mybucket/inputs/java bos://mybucket/outputs/java",
"jar": "bos://mybucket/jars/my-java.jar",
"mainClass": "org.apache.hadoop.mapred.WordCount"
},
"type": "Java",
"additionalFiles":[]
}
]
}
请求头域
除公共头域外,无其它特殊头域。
请求参数
参数名 | 参数类型 | 是否必须 | 参数位置 | 参数描述 |
---|---|---|---|---|
version | String | 是 | URL参数 | API版本号,当前取值1 |
clusterId | String | 是 | URL参数 | 待指定的集群ID,只有状态为RUNNING或WAITING的集群才能添加作业 |
steps | List\<StepConfig> | 是 | RequestBody参数 | 需要添加的作业信息 |
clientToken | String | 否 | Query参数 | 幂等性Token,是一个长度不超过64位的ASCII字符串 |
响应头域
除公共头域外,无其它特殊头域。
响应参数
参数名 | 参数类型 | 参数描述 |
---|---|---|
stepIds | List\ |
作业ID的集合,作业ID是系统生成的作业唯一标识 |
请求示例
POST /v1/cluster/0ce4f730-4af2-4f37-8fa2-b14f2f44e50e/step?clientToken=be31b98c-5e41-4838-9830-9be700de5a20 HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2015-03-24T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903
{
"steps": [
{
"actionOnFailure": "Continue",
"name": "my-mapreduce-streaming",
"properties": {
"arguments": "-D mapreduce.job.reduces=20",
"input": "bos://mybucket/inputs/txt-1m",
"mapper": "cat",
"output": "bos://mybucket/outputs/streaming",
"reducer": "cat"
},
"type": "Streaming",
"additionalFiles":[
{
"remote":"bos://mybucket/a",
"local":""
},
{
"remote":"",
"local":"D://a"
}
]
},
{
"actionOnFailure": "Continue",
"name": "my-java-program",
"properties": {
"arguments": "bos://mybucket/inputs/java bos://mybucket/outputs/java",
"jar": "bos://mybucket/jars/my-java.jar",
"mainClass": "org.apache.hadoop.mapred.WordCount"
},
"type": "Java",
"additionalFiles":[]
}
]
}
响应示例
HTTP/1.1 201 CREATED
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8
{
"stepIds": [
"31fd0974-452b-491f-a3fc-c28b3b9d7046",
"f99f669b-be45-48c9-8549-fd8dd473be0f"
]
}
查询作业列表
接口描述
获取指定集群的作业列表,支持分页查询
权限说明
请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求,请参考 鉴权认证。
注意事项
如果请求中没有用户验证信息(即匿名访问),返回403 Forbidden
,错误信息:AccessDenied
。
请求结构
GET /v{version}/cluster/{clusterId}/step?pageNo={pageNo}&pageSize={pageSize} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}
请求头域
除公共头域外,无其它特殊头域。
请求参数
参数名 | 参数类型 | 是否必须 | 参数位置 | 参数描述 |
---|---|---|---|---|
version | String | 是 | URL参数 | API版本号,当前取值1 |
clusterId | String | 是 | URL参数 | 指定的集群ID |
pageNo | Int | 否 | Query参数 | 查询起始页 |
pageSize | Int | 否 | Query参数 | 每页返回的条目数 |
响应头域
除公共头域外,无其它特殊头域。
响应参数
参数名 | 参数类型 | 参数描述 |
---|---|---|
total | Int | 返回的总条目数 |
pageNo | Int | 请求的页数 |
pageSize | Int | 每页返回的条目数 |
steps | List\<Step> | 返回的作业列表 |
请求示例
GET /v1/cluster/0ce4f730-4af2-4f37-8fa2-b14f2f44e50e/step?pageNo=1&pageSize=10 HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2015-03-24T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903
响应示例
HTTP/1.1 200 OK
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8
{
"steps": [
{
"clusterId": "6802b200-6e3a-4fee-9db8-575d7ee76c28",
"launcherJobId": "",
"status": {
"state": "Failed",
"code": "",
"message": "",
"creationDateTime": "2022-01-24T11:20:02Z",
"endDateTime": "2022-01-24T11:22:34Z",
"startDateTime": "2022-01-24T11:22:34Z"
},
"logUri": "bos://data-wh-sdk-bigdata/clusterlog/6802b200-6e3a-4fee-9db8-575d7ee76c28/steps/1e33774e-0c92-4ce0-89c6-bae69f6959b3",
"stderr": "bos://data-wh-sdk-bigdata/clusterlog/6802b200-6e3a-4fee-9db8-575d7ee76c28/steps/1e33774e-0c92-4ce0-89c6-bae69f6959b3/stderr",
"stdout": "bos://data-wh-sdk-bigdata/clusterlog/6802b200-6e3a-4fee-9db8-575d7ee76c28/steps/1e33774e-0c92-4ce0-89c6-bae69f6959b3/stdout",
"syslog": "bos://data-wh-sdk-bigdata/clusterlog/6802b200-6e3a-4fee-9db8-575d7ee76c28/steps/1e33774e-0c92-4ce0-89c6-bae69f6959b3/syslog",
"id": "1e33774e-0c92-4ce0-89c6-bae69f6959b3",
"name": "my-mapreduce-streaming",
"type": "Streaming",
"actionOnFailure": "Continue",
"additionalFiles": [],
"properties": {
"input": "bos://mybucket/inputs/txt-1m",
"mapper": "cat",
"output": "bos://mybucket/outputs/streaming",
"reducer": "cat"
}
}
],
"pageNo": 1,
"pageSize": 10,
"total": 1
}
查询作业信息
接口描述
查询指定集群中的一个指定作业的信息
权限说明
请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求,请参考 鉴权认证。
注意事项
如果请求中没有用户验证信息(即匿名访问),返回403 Forbidden
,错误信息:AccessDenied
。
请求结构
GET /v{version}/cluster/{clusterId}/step/{stepId} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}
请求头域
除公共头域外,无其它特殊头域。
请求参数
参数名 | 参数类型 | 是否必须 | 参数位置 | 参数描述 |
---|---|---|---|---|
version | String | 是 | URL参数 | API版本号,当前取值1 |
clusterId | String | 是 | URL参数 | 指定的集群ID |
stepId | String | 是 | URL参数 | 待查询的作业ID |
响应头域
除公共头域外,无其它特殊头域。
响应参数
参数名 | 参数类型 | 参数描述 |
---|---|---|
actionOnFailure | String | 作业失败策略 |
id | String | 作业ID,是一个定长字符串,只包含大小写字母、数字、连字号(-)和下划线(_) |
name | String | 作业名称 |
properties | Object | 作业描述,具体内容由作业类型决定 |
status | StepStatus | 作业状态信息 |
type | String | 作业类型 |
请求示例
GET /v1/cluster/0ce4f730-4af2-4f37-8fa2-b14f2f44e50e/step/f5540043-b1b1-4a52-95a7-6005cb0ebc64 HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2015-03-24T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903
响应示例
HTTP/1.1 200 OK
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8
{
"actionOnFailure": "Continue",
"id": "f5540043-b1b1-4a52-95a7-6005cb0ebc64",
"name": "my-mapreduce-streaming",
"properties": {
"arguments": "-D mapreduce.job.reduces=20",
"input": "bos://mybucket/inputs/txt-1m",
"mapper": "cat",
"output": "bos://mybucket/outputs/streaming",
"reducer": "cat"
},
"status": {
"code": null,
"creationDateTime": "2014-08-06T11:00:05Z",
"endDateTime": "2014-08-06T11:02:37Z",
"message": null,
"startDateTime": "2014-08-06T11:00:33Z",
"state": "Completed"
},
"type": "Streaming"
}