集群操作接口

更新时间：2022-06-30

创建集群

接口描述

向BMR服务请求创建一个新的集群。

权限说明

请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求，请参考鉴权认证。

注意事项

如果请求中没有用户验证信息（即匿名访问），返回403 Forbidden，错误信息：AccessDenied。

请求结构

POST /v{version}/cluster?clientToken={clientToken} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}
{
    "applications": [
        {
            "name": appilicationName,
            "version": applicationVersion
        },
        {
            "name": appilicationName,
            "version": applicationVersion
        }
    ],
    "securityGroup": securityGroupId,
    "vpcId": vpcShortId,
    "subnetId": subnetShortId,
    "autoTerminate": false,
    "imageType": imageType,
    "imageVersion": imageVersion,
    "serviceHaEnabled":serviceHaEnabled,
    "safeModeEnabled":safeModeEnabled,
    "instanceGroups": [
        {
            "instanceCount": instanceCount,
            "instanceType": instanceType,
            "type": instanceGroupType,
            "rootDiskSizeInGB": rootDiskSizeInGB,
            "rootDiskMediumType": rootDiskMediumType,
            "cds": [
                {
                    "sizeInGB": sizeInGB,
                    "mediumType": mediumType
                },
                {
                    "sizeInGB": sizeInGB,
                    "mediumType": mediumType
                }
            ]
        }
    ],
    "logUri": logUri,
    "name": clusterName,
    "adminPassword": adminPassword,
    "steps": [
        {
            "actionOnFailure": actionOnFailture,
            "name": stepName,
            "properties": {
                "arguments": arguments,
                "mapper": mapper,
                "reducer": reducer,
                "input": input,
                "output": output
            },
            "type": stepType,
            "additionalFiles":[]
        }
    ]
}

请求头域

除公共头域外，无其它特殊头域。

售卖套餐请参考https://cloud.baidu.com/doc/BMR/s/6jwvxw85z

请求参数

名称	类型	是否必须	参数位置	参数描述
version	String	是	URL参数	API版本号，当前取值1
imageType	String	是	RequestBody参数	集群类型，比如 hadoop，参考BMR集群
imageVersion	String	是	RequestBody参数	集群版本，比如 2.2.2，参考BMR集群
instanceGroups	List<InstanceGroupConfig>	是	RequestBody参数	实例组配置
clientToken	String	否	Query参数	幂等性Token，是一个长度不超过64位的ASCII字符串
applications	List<ApplicationConfig>	否	RequestBody参数	需要安装的组件信息（如hive，pig，hbase）
autoTerminate	Boolean	否	RequestBody参数	为true则在运行完所有作业后自动释放集群，默认为true
logUri	String	否	RequestBody参数	用于上传运行日志的BOS路径，不提供则不上传日志
name	String	否	RequestBody参数	集群名称，默认为my-cluster
steps	List<StepConfig>	否	RequestBody参数	需要运行的作业信息，参考添加作业协议
vpcId	String	是	RequestBody参数	设置vpc短ID信息
subnetId	String	是	RequestBody参数	设置子网短ID信息
securityGroup	String	是	RequestBody参数	设置安全组短ID
adminPassword	String	是	RequestBody参数	设置集群密码，8-16位字符，英文，数字和符号必须同时存在，符号仅限!@#$%^*()，密码需要加密传输

响应头域

除公共头域外，无其它特殊头域。

响应参数

名称	类型	描述
clusterId	String	系统生成的集群唯一标识

请求示例

POST /v1/cluster?clientToken=be31b98c-5e41-4838-9830-9be700de5a20 HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2022-01-26T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903

{
    "applications": [
        {
            "name": "hive",
            "version": "3.1.0"
        },
        {
            "name": "tez",
            "version": "0.9.1"
        },
        {
            "name": "spark2",
            "version": "2.4.4"
        }
    ],
    "securityGroup": "g-ai3jzpiadv1x",
    "vpcId": "d5465344-e783-41b1-9765-ec3ee4bec332",
    "subnetId": "b35c3b60-18a0-42a6-87e7-e857dadd9ec3",
    "autoTerminate": false,
    "imageType": "hadoop",
    "imageVersion": "2.2.2",
    "serviceHaEnabled":true,
    "safeModeEnabled":false,
    "instanceGroups": [
        {
            "instanceCount": 1,
            "instanceType": "bmr.g1.xlarge",
            "type": "Master",
            "rootDiskSizeInGB": 40,
            "rootDiskMediumType": "ssd",
            "cds": [
                {
                    "sizeInGB": 50,
                    "mediumType": "ssd"
                }
            ]
        },
        {
            "instanceCount": 3,
            "instanceType": "bmr.g1.xlarge",
            "type": "Core",
            "rootDiskSizeInGB": 40,
            "rootDiskMediumType": "ssd",
            "cds": [
                {
                    "sizeInGB": 50,
                    "mediumType": "ssd"
                }
            ]
        }
    ],
    "logUri": "bos://data-wh-sdk-bigdata/clusterlog",
    "name": "my-cluster",
    "adminPassword": "7c658fefaaae7d4516d9305b27770a55",
    "steps": [
        {
            "actionOnFailure": "Continue",
            "name": "my-mapreduce-streaming",
            "properties": {
                "arguments": "-D mapreduce.job.reduces=20",
                "mapper": "cat",
                "reducer": "cat",
                "input": "bos://mybucket/inputs/txt-1m",
                "output": "bos://mybucket/outputs/streaming"
            },
            "type": "Streaming",
             "additionalFiles":[
             {
                "remote":"bos://mybucket/i",
                "local":""
             },
             {
                "remote":"",
                "local":""
             }
             ]
        }
     ]
 }

响应示例

HTTP/1.1 201 CREATED
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8
{
    "clusterId": "0181b994-576a-4a00-b04e-f6fa9d5eee48"
}

查询集群列表

接口描述

查询所有集群的信息，支持分页查询。

权限说明

请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求，请参考鉴权认证。

注意事项

如果请求中没有用户验证信息（即匿名访问），返回403 Forbidden，错误信息：AccessDenied。

请求结构

GET /v{version}/cluster?marker={marker}&maxKeys={maxKeys} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}

请求头域

除公共头域外，无其它特殊头域。

请求参数

参数名	参数类型	是否必须	参数位置	参数描述
version	String	是	URL参数	API版本号，当前取值1
marker	String	否	Query参数	批量获取列表的查询的起始位置，是一个由系统生成的字符串
maxKeys	Int	否	Query参数	每页包含的最大数量，最大数量不能超过1000。大于1000的会被修正为1000。缺省值为1000

响应头域

除公共头域外，无其它特殊头域。

响应参数

参数名	参数类型	参数描述
marker	String	标记查询的起始位置
isTruncated	Boolean	true表示后面还有数据，false表示已经是最后一页
clusters	List<Cluster>	返回的集群列表

请求示例

GET /v1/cluster?marker=9e0c8cf1-690c-444a-8727-04241f3beaa5&maxKeys=30 HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2022-01-26T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903

响应示例

HTTP/1.1 200 OK
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8
{
    "clusters": [
        {
            "id": "8fa594c8-665b-49cc-b1ec-743bf1bf5ca2",
            "name": "test_name",
            "payType": "postpay",
            "status": {
                "state": "Running",
                "code": "",
                "message": "",
                "orderStatus": "",
                "creationDateTime": "2022-01-25T06:03:42Z",
                "endDateTime": "2022-01-26T11:14:20.26516422+08:00",
                "readyDateTime": "2022-01-25T06:06:05Z",
                "expireDateTime": "2022-01-25T06:03:41Z",
                "expireDates": 0
            },
            "imageType": "hadoop",
            "imageVersion": "2.3.0",
            "enableAutoScale": true,
            "createTime": "2022-01-25T06:03:42Z",
            "tags": null,
            "applications": [
                {
                    "name": "hdfs",
                    "version": "3.1.1",
                    "properties": {}
                },
                {
                    "name": "yarn",
                    "version": "3.1.1",
                    "properties": {}
                },
                {
                    "name": "mapreduce2",
                    "version": "3.1.1",
                    "properties": {}
                },
                {
                    "name": "zookeeper",
                    "version": "3.4.6",
                    "properties": {}
                },
                {
                    "name": "hadoop-manager",
                    "version": "1.0.0",
                    "properties": {}
                },
                {
                    "name": "ldap",
                    "version": "2.4.48",
                    "properties": {}
                }
            ]
        },
           {
            "id": "06a30746-ec2e-4f11-9bd1-643fd06d28d2",
            "name": "hadoop_120_test",
            "payType": "postpay",
            "status": {
                "state": "Starting",
                "code": "",
                "message": "",
                "orderStatus": "",
                "creationDateTime": "2022-01-26T03:14:52Z",
                "endDateTime": "2022-01-26T11:17:21.453684987+08:00",
                "readyDateTime": null,
                "expireDateTime": "2022-01-26T03:14:51Z",
                "expireDates": 0
            },
            "imageType": "hadoop",
            "imageVersion": "1.2.0",
            "enableAutoScale": true,
            "createTime": "2022-01-26T03:14:52Z",
            "tags": null,
            "applications": [
                {
                    "name": "hdfs",
                    "version": "2.7.1"
                },
                {
                    "name": "zookeeper",
                    "version": "3.4.6"
                },
                {
                    "name": "yarn",
                    "version": "2.7.1"
                },
                {
                    "name": "mapreduce2",
                    "version": "2.7.1"
                },
                {
                    "name": "hadoop-manager",
                    "version": "1.0.0"
                },
                {
                    "name": "spark2",
                    "version": "2.1.0"
                },
                {
                    "name": "hive",
                    "version": "1.2.0",
                    "properties": {}
                },
                {
                    "name": "pig",
                    "version": "0.15.1"
                },
                {
                    "name": "hue",
                    "version": "3.10.0"
                },
                {
                    "name": "hbase",
                    "version": "1.1.2"
                },
                {
                    "name": "ranger",
                    "version": "0.5.0"
                },
                {
                    "name": "tez",
                    "version": "0.7.0"
                }
            ]
        }
    ],
    "isTruncated": false,
    "marker": "9e0c8cf1-690c-444a-8727-04241f3beaa5"
}

查询集群信息

接口描述

查询一个指定集群的信息。

权限说明

请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求，请参考鉴权认证。

注意事项

如果请求中没有用户验证信息（即匿名访问），返回403 Forbidden，错误信息：AccessDenied。

请求结构

GET /v{version}/cluster/{clusterId} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}

请求头域

除公共头域外，无其它特殊头域。

请求参数

参数名	参数类型	是否必须	参数位置	参数描述
version	String	是	URL参数	API版本号，当前取值1
clusterId	String	是	URL参数	待查询的集群ID

响应头域

除公共头域外，无其它特殊头域。

响应参数

参数名	参数类型	参数描述
applications	List<Application>	需要安装的组件信息（如hive，pig，hbase）
orderId	String	订单号
id	String	集群ID，是一个定长字符串，只包含大小写字母、数字、连字号（-）和下划线（_）
name	String	集群名称
status	ClusterStatus	集群状态信息
imageType	String	集群类型，参考BMR集群
imageVersion	String	集群版本，参考BMR集群
payType	String	付费方式
logUri	String	用于上传运行日志的BOS路径
availabilityZone	String	集群所选可用区
vpcId	String	集群所选VPC
subnetId	String	集群所选子网ID
securityGroupId	String	集群所选安全组ID
serviceHaEnabled	Boolean	是否开启高可用模式
safeModeEnabled	Boolean	是否开启安全模式
enableAutoScale	Boolean	是否支持弹性伸缩
instanceGroup	InstanceGroup	实例组

请求示例

GET /v1/cluster/d647cda9-b308-47e4-a2ab-0a4908e17643 HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2022-01-26T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903

响应示例

HTTP/1.1 200 OK
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8
{
    "id": "cf866824-f35b-4e2c-b5f5-6ee067806d7c",
    "name": "test_hadoop_230_zzf",
    "availabilityZone": "zoneA",
    "payType": "postpay",
    "status": {
        "state": "TerminatedWithError",
        "code": "",
        "message": "openApi request[BatchCreateServer] get unexpected status code, code: 400, body: {\"requestId\":\"a3211ab4-672c-492c-5c24-3eafc8bb0b0c\",\"code\":\"Instance.BccExceedCapacity\",\"message\":\"Bcc capacity is not enough\"}",
        "orderStatus": "",
        "creationDateTime": "2022-01-24T02:47:12Z",
        "endDateTime": "2022-01-26T11:19:10.434269608+08:00",
        "readyDateTime": null,
        "expireDateTime": "2022-01-24T02:47:12Z",
        "expireDates": 0
    },
    "orderId": "",
    "imageType": "hadoop",
    "imageVersion": "2.3.0",
    "applications": [
        {
            "name": "hdfs",
            "version": "3.1.1",
            "properties": {}
        },
        {
            "name": "yarn",
            "version": "3.1.1",
            "properties": {}
        },
        {
            "name": "mapreduce2",
            "version": "3.1.1",
            "properties": {}
        },
        {
            "name": "zookeeper",
            "version": "3.4.6",
            "properties": {}
        },
        {
            "name": "hadoop-manager",
            "version": "1.0.0",
            "properties": {}
        },
        {
            "name": "ldap",
            "version": "2.4.48",
            "properties": {}
        },
        {
            "name": "spark2",
            "version": "2.4.4",
            "properties": {}
        },
        {
            "name": "hive",
            "version": "3.1.0",
            "properties": {
                "metastore": "default"
            }
        },
        {
            "name": "tez",
            "version": "0.9.1",
            "properties": {}
        },
        {
            "name": "azkaban",
            "version": "3.58.0",
            "properties": {}
        },
        {
            "name": "flink",
            "version": "1.12.4",
            "properties": {}
        },
        {
            "name": "presto",
            "version": "0.219",
            "properties": {}
        },
        {
            "name": "pig",
            "version": "0.17.0",
            "properties": {}
        },
        {
            "name": "zeppelin",
            "version": "0.8.0",
            "properties": {}
        },
        {
            "name": "hbase",
            "version": "2.2.7",
            "properties": {
                "backupEnabled": false,
                "restoreEnabled": false
            }
        },
        {
            "name": "oozie",
            "version": "5.1.0",
            "properties": {}
        },
        {
            "name": "hue",
            "version": "4.8.0",
            "properties": {}
        }
    ],
    "vpcId": "d5465344-e783-41b1-9765-ec3ee4bec331",
    "subnetId": "b35c3b60-18a0-42a6-87e7-e857dadd9eb2",
    "securityGroupId": "g-ai3jzpiadv1q",
    "serviceHaEnabled": true,
    "safeModeEnabled": true,
    "enableAutoScale": true，
    "instanceGroups":[
       {
           id:"b35c3b60-18a0-42a6-87e7-e8355",
           instanceType:"Master",
           name:"",
           type:"",
           instanceCount:12,
           rootDiskSizeInGB:1024,
           rootDiskMediumType:"hdd",
           cds:[
           {
              "sizeInGB":1024,
              "mediumType":"hdd"
           }
           ]
       }
    ]
}

释放集群

接口描述

释放集群，仅限单个集群释放。

权限说明

请求发起人需要具有合法的AccessKeyID和SecretAccessKey才能发起请求，请参考鉴权认证。

注意事项

如果请求中没有用户验证信息（即匿名访问），返回403 Forbidden，错误信息：AccessDenied。

请求结构

DELETE /v{version}/cluster/{clusterId} HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: {utc-date-string}
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: {bce-authorization-string}

请求头域

除公共头域外，无其它特殊头域。

请求参数

参数名	参数类型	是否必须	参数位置	参数描述
version	String	是	URL参数	API版本号，当前取值1
clusterId	String	是	URL参数	待释放的集群ID

响应头域

除公共头域外，无其它特殊头域。

响应参数

无参数

请求示例

DELETE /v1/cluster/0ce4f730-4af2-4f37-8fa2-b14f2f44e50e HTTP/1.1
accept-encoding: gzip, deflate
x-bce-date: 2022-01-26T13:02:00Z
host: bmr.bj.baidubce.com
connection: keep-alive
accept: */*
content-type: application/json
authorization: bce-auth-v1/46bd9968a6194b4bbdf0341f2286ccce/2022-01-26T13:02:00Z/1800/host;x-bce-date/994014d96b0eb26578e039fa053a4f9003425da4bfedf33f4790882fb4c54903

响应示例

HTTP/1.1 204 NO_CONTENT
Transfer-Encoding: chunked
x-bce-request-id: 73c4e74c-3101-4a00-bf44-fe246959c05e
Cache-Control: no-cache
Server: BWS
Date: Wed, 26 Jan 2022 13:02:01 GMT
Content-Type: application/json;charset=UTF-8

错误码

实例组操作接口

百度智能云

MapReduce

MapReduce

集群操作接口

创建集群

查询集群列表

查询集群信息

释放集群