所有文档

          百度MapReduce BMR

          Cluster(集群)

          新建cluster

          如下代码可以新建一个集群,集群包含1个master节点和2个core节点,且安装了Hive、Pig、HBase应用。请注意:参考下面样例代码时,需要修改log_uri参数指定BOS路径为您的账户可用的BOS路径。

          instance_groups = [
              BmrClient.instance_group(
                  'Master',
                  'bmr.g1.xlarge	',
                  1,
                  'ig-master'
              ),
              BmrClient.instance_group(
                  'Core',
                  'bmr.g1.xlarge	',
                  2,
                  'ig-core'
              )
          ]  # 构造请求体
          request_body = {
              'applications': [
                  {
                      "name": "hive",
                      "version": "0.13.0"
                  },
                  {
                      "name": "pig",
                      "version": "0.11.0"
                  },
                  {
                      "name": "hbase",
                      "version": "0.98.0"
                  }
              ],
              'auto_terminate': False,
              'log_uri': 'bos://tester01/sdk/',
              'name': 'sdk-cluster01',
              'service_ha_enabled': False,
              'safe_mode_enabled': False
          }
          
          try:
              response = bmr_client.create_cluster(
                  'hadoop',
                  '0.1.0',
                  instance_groups,
                  **request_body)
              cluster_id = response.cluster_id
          except BceHttpClientError as e:
              if isinstance(e.last_error, BceServerError):
                  LOG.error('create_cluster failed. Response %s, code: %s, msg: %s'
                            % (e.last_error.status_code, e.last_error.code, e.last_error.message))
              else:
                  LOG.error('create_cluster failed. Unknown exception: %s' % e)

          售卖的套餐类型请参考https://cloud.baidu.com/doc/BMR/s/6jwvxw85z

          列出全部cluster

          如下代码可以罗列出属于请求调用者的所有集群,用户可以通过配置查询参数max_keys来限制每次请求返回的集群数目:

          try:
              response = bmr_client.list_clusters(max_keys=5)
              LOG.debug('list total %s clusters' % len(response.clusters))
              # 输出各个集群的信息
              for cluster in response.clusters:
                  LOG.debug('cluster: %s' % cluster)
          except BceHttpClientError as e:
              if isinstance(e.last_error, BceServerError):
                  LOG.error('list_clusters failed. Response %s, code: %s, msg: %s'
                            % (e.last_error.status_code, e.last_error.code, e.last_error.message))
              else:
                  LOG.error('list_clusters failed. Unknown exception: %s' % e)

          另外,也可以使用分页查询的方式来获取所有集群,通过配置max_keys来限制每页查询返回的集群数目:

          try:
              page = 1
              next_marker = None
              max_keys = 5
              is_truncated = True
              while is_truncated:
                  response = bmr_client.list_clusters(marker=next_marker, max_keys=max_keys)
                  LOG.debug('list %s clusters on Page %s' % (len(response.clusters), page))
                  page += 1
                  is_truncated = response.is_truncated
                  next_marker = response.next_marker
          except BceHttpClientError as e:
              if isinstance(e.last_error, BceServerError):
                  LOG.error('list_clusters failed. Response %s, code: %s, msg: %s'
                            % (e.last_error.status_code, e.last_error.code, e.last_error.message))
              else:
                  LOG.error('list_clusters failed. Unknown exception: %s' % e)

          查询指定的cluster

          获取了集群ID后,可用如下代码查询指定集群的信息:

          try:
              response = bmr_client.get_cluster(cluster_id)
              # 输出集群的状态信息
              status = response.status.state
              LOG.debug('check cluster %s: status %s' % (cluster_id, status))
          except BceHttpClientError as e:
              if isinstance(e.last_error, BceServerError):
                  LOG.error('get_cluster failed. Response %s, code: %s, msg: %s'
                            % (e.last_error.status_code, e.last_error.code, e.last_error.message))
              else:
                  LOG.error('describe_cluster failed. Unknown exception: %s' % e)

          终止指定的cluster

          如下代码可以终止指定的集群:

          try:
              response = bmr_client.terminate_cluster(cluster_id)
              LOG.debug('terminate cluster %s: status %s' % (cluster_id, response.status))
          except BceHttpClientError as e:
              if isinstance(e.last_error, BceServerError):
                  LOG.error('terminate_cluster failed. Response %s, code: %s, msg: %s'
                            % (e.last_error.status_code, e.last_error.code, e.last_error.message))
              else:
                  LOG.error('terminate_cluster failed. Unknown exception: %s' % e)
          上一篇
          InstanceGroup(实例组)
          下一篇
          Step(作业)