Cluster(集群)

新建cluster

如下代码可以新建一个集群,集群包含1个master节点和2个core节点,且安装了Hive、Pig、HBase应用。请注意:参考下面样例代码时,需要修改相关的BOS路径为您的账户可用的BOS路径,包括withLogUri函数参数、HBase应用配置的withBackupLocation函数参数。

新建集群可以通过配置CreateClusterRequest对象的clientToken属性来保证创建请求的幂等性。clientToken是一个长度不超过64位的ASCII字符串,配置CreateClusterRequest对象的clientToken方法是:createClusterRequest.withClientToken(clientToken)

请求返回的CreateClusterResponse对象包含了新创建出集群的集群ID,获取方法为response.getClusterId()

public void createCluster(BmrClient bmrClient) {
    // 发送创建集群请求
    String clusterId = null;
    try {
        CreateClusterResponse response = bmrClient.createCluster(
                new CreateClusterRequest()
                        .withName("java-sdk-cluster")
                        .withImageType("hadoop")
                        .withImageVersion("0.1.0")
                        .withAutoTerminate(false)
                        .withLogUri("bos://path/to/logUri/")
                        .withServiceHaEnabled(false)
                        .withSafeModeEnabled(false)
                        .withInstanceGroup(new InstanceGroupConfig()
                                .withName("ig-master")
                                .withType("Master")
                                .withInstanceType("bmr.g1.2xlarge")
                                .withInstanceCount(1))
                                .withRootDiskSizeInGB(50)
                                .withRootDiskMediumType("ssd")
                                .withCds(new CdsItem()
                                    .withSizeInGB(100)
                                    .withMediumType(“ssd”));
                        .withInstanceGroup(new InstanceGroupConfig()
                                .withName("ig-core")
                                .withType("Core")
                                .withInstanceType("bmr.gh1.large")
                                .withInstanceCount(2))
                                .withRootDiskSizeInGB(50)
                                .withRootDiskMediumType("ssd")
                                .withCds(new CdsItem()
                                    .withSizeInGB(100)
                                    .withMediumType(ssd));
                        .withApplication(new PigApplicationConfig().withVersion("0.11.0"))
                        .withApplication(new HiveApplicationConfig().withVersion("0.13.0").withMetastore("default"))
                        .withApplication(new HBaseApplicationConfig()
                                .withVersion("0.98.0")
                                .withBackupEnabled(true)
                                .withBackupLocation("bos://tester01/hbase_backup")
                                .withBackupIntervalInMinutes(300)
                                .withBackupStartDatetime("2015-08-18T23:00:00Z"))
                        .withStep(new JavaStepConfig()
                                .withName("init-step")
                                .withActionOnFailure("Continue")
                                .withJar("bos://bmr/samples/mapreduce/libs/hadoop-mapreduce-examples.jar")
                                .withMainClass("org.apache.hadoop.examples.WordCount")
                                .withArguments("bos://bmr/samples/mapreduce/wordcount/hamlet.txt bos://tester01/out"))
        );
        // 获取新创建集群的集群ID。
        clusterId = response.getClusterId();
    } catch (BceServiceException e) {
        System.out.println("Create cluster failed: " + e.getErrorMessage());
    } catch (BceClientException e) {
        System.out.println(e.getMessage());
    }
}

列出全部cluster

如下代码可以罗列出属于请求调用者的所有集群,用户可以通过配置查询参数maxKeys来限制每次请求返回的集群数目,并且通过配置有效的查询参数marker来指定查询记录的起点。marker参数值是由BMR系统生成并返回的,因而初次查询请求中不需要配置该参数,它是在多次循环查询请求的后继请求中进行使用的。

public void listClusters(BmrClient bmrClient) {
    int maxKeys = 10;
    try {
        // 方法 1. 默认的查询请求
        ListClustersResponse response1 = bmrClient.listClusters();

        // 方法 2. 配置maxKeys的单次查询请求
        ListClustersResponse response2 = bmrClient.listClusters(maxKeys);

        // 方法 3. 配置maxKeys和marker参数的循环查询请求
        boolean isTruncated = true;
        int page = 0;
        String marker = null;
        while (isTruncated) {
            ListClustersResponse response3 = bmrClient.listClusters(marker, maxKeys);
            page++;
            System.out.format("Page %d: cluster count: %d\n", page, response3.getClusters().size());
            isTruncated = response3.isTruncated();
            marker = response3.getNextMarker();
        }

        // 方法 4. 自定义ListClustersResquest对象的查询请求
        ListClustersResponse response4 = bmrClient.listClusters(
                new ListClustersRequest().withMaxKeys(maxKeys)
        );

        // 输出各个集群的ID
        for (Cluster cluster : response4.getClusters()) {
            System.out.format("cluster id: %s\n", cluster.getId());
        }
    } catch (BceServiceException e) {
        System.out.println("List clusters failed: " + e.getErrorMessage());
    }
}

请求返回的ListClustersResponse对象包含了相关的集群对象数组List<Cluster>, 获取集群对象数组的方法为response.getClusters()。集群对象Cluster的属性包括了集群相关的配置信息,每个属性均有对应的getter访问器方法。

public class Cluster {
    private String id;                                       // 集群ID
    private String name;                                     // 集群名称
    private String imageType;                                // 集群虚拟机实例的镜像类型
    private String imageVersion;                             // 集群虚拟机实例的镜像版本
    private String logUri;                                   // 存储集群运行日志的 BOS 路径
    private boolean autoTerminate;                           // 集群是否自动终止的布尔标志位
    private List<Application> applications;                  // 集群已安装的应用信息
    private ClusterStatus status;                            // 集群当前的状态信息
    private boolean serviceHaEnabled;                        // 集群是否开启高可用
    private boolean safeModeEnabled;                         // 集群是否开始安全模式
}

public class ClusterStatus {
    private String state;                                    // 集群状态字段
    private Date creationDateTime;                           // 集群创建时间
    private Date endDateTime;                                // 集群终止时间
    private Date readyDateTime;                              // 集群完成部署时间
}

查询指定的cluster

获取了集群ID后,可用如下代码查询指定集群的信息。

请求返回的GetClusterResponse对象包含了获取集群属性的getter访问器方法,可以直接调用response的访问器方法来获得目标集群的属性信息。

public void getCluster(BmrClient bmrClient, String clusterId) {
    try {
        // 方法 1. 查询对应ID的集群信息
        GetClusterResponse response1 = bmrClient.getCluster(clusterId);

        // 方法 2. 自定义GetClusterRequest对象的查询请求
        GetClusterResponse response2 = bmrClient.getCluster(
                new GetClusterRequest().withClusterId(clusterId)
        );
        // 输出集群的镜像类型
        System.out.println(response1.getImageType());
        // 输出集群当前的状态
        System.out.println(response2.getStatus().getState());
    } catch (BceServiceException e) {
        System.out.println("Describe cluster failed: " +  e.getErrorMessage());
    }
}

终止指定的cluster

如下代码可以终止指定的集群:

public void terminateCluster(BmrClient bmrClient, String clusterId) {
    try {
        // 方法 1. 终止对应ID的集群
        bmrClient.terminateCluster(clusterId);

        /*
        方法 2. 自定义TerminateClusterRequest对象的终止请求
        bmrClient.terminateCluster(
                new TerminateClusterRequest().withClusterId(clusterId)
        );
        */
    } catch (BceServiceException e) {
        System.out.println("Terminate cluster failed: " + e.getErrorMessage());
    }
}