对象存储BOS

    手机美图APP实践

    场景概述

    移动互联时代手机上传数据的场景随处可见,为了方便开发者聚焦于产品的业务逻辑,用户可以直接将文件存储到BOS上。BOS产品基于STS授权方式为用户提供了安全的上传和下载方式,BOS还支持图片处理服务。BOS具有成本低、支持海量存储和弹性扩展的特性,能帮助开发者更方便实现移动APP业务的开发。

    本教程能帮助用户快速搭建一个基于BOS的文件直传+图片处理的手机APP,主要基于STS临时授权、Android SDK和图片处理API三个模块实现。

    • 由于用户的移动端应用存在泄密的风险所以不可能直接存储AK/SK信息,必须使用STS临时授权模式访问BOS。STS临时授权模式中会生成一个临时Token,该Token具有一定的时效性,即APP应用只有在Token的时效性内访问才可以完成上传或下载图片服务,过了时效需要重新获取。
    • Andriod SDK帮助用户实现新建BOS客户端并将文件存储到BOS或从BOS中下载文件。
    • 图片处理API主要实现图片的处理如缩略、裁剪、格式转换、旋转、加文字/图片水印等功能。

    手机美图APP数据交互如下:

    image.png

    美图APP示例

    手机美图示例APP下载地址:

    下载完APP并安装完成后可以直接通过应用服务器地址访问BOS,并进行图片处理。应用服务器地址是指搭建移动应用的后台服务器,默认开启的端口为8080。关于BOS的区域和Bucket设置都需要在应用服务器进行配置。

    APP操作方法

    该APP支持用户上传、下载和下载缩放图三个功能。

    • 上传:用户填写应用服务器地址后,选择本地需要上传的图片,图片会显示在操作界面下方,点击上传即可。上传成功会显示“File Uploaded”。
    • 下载:用户填写需要下载的文件名称,点击下载按钮即可。下载成功会显示“File Downloaded”。
    • 下载缩放图:下载缩放图时必须指定明确的图片后缀如jpg,然后设置下载图形的宽和高以及旋转角度,点击下载缩放图,则会获取经过处理的图形。下载成功会显示“File Downloaded”。Demo版本下载后的图片都不会存储到本地。

    如何搭建美图APP

    搭建美图APP包含以下几个步骤:

    1. 开启BOS服务并创建Bucket用于存储图片,开通和创建Bucket的详细操作请参见创建Bucket。如果要下载缩放图,需要保证指定的Bucket开启了图片处理服务。
    2. 开通STS服务,用于保证上传和下载图片的安全性。
    3. 部署应用服务器,实现和BOS及客户端的交互,美图APP代码请参考:BOS美图APP代码
    4. 下载安装美图APP。

    如何部署应用服务器

    1. 从github上下载sample code的代码包,代码包主要包含“bos_meitu_app”和“bos_meitu_app_server”两部分,其中“bos_meitu_app”主要用于定义APP界面及相关动作,“bos_meitu_app_server”为应用服务器相关配置。
    2. 修改“bos_meitu_app_server”中的“MeituAppServerHandler.java”文件,其中定义了ak/sk、BOS服务器对应的Endpoint和Bucket名称等信息。

       public String getBosInfo(String bosRequestType) {
            //配置ak、sk
            String bosAk = "开发者的ak";
            String bosSk = "开发者的SK";
            //百度智能云提供的stsendpoint
            String stsEndpoint = "http://sts.bj.baidubce.com";
      
            BceCredentials credentials = new DefaultBceCredentials(bosAk, bosSk);
            BceClientConfiguration clientConfig = new BceClientConfiguration();
            clientConfig.setCredentials(credentials);
            clientConfig.setEndpoint(stsEndpoint);
            StsClient stsClient = new StsClient(clientConfig);
            GetSessionTokenRequest stsReq = new GetSessionTokenRequest();
            // request expiration time
            stsReq.setDurationSeconds(1800);
            GetSessionTokenResponse stsToken = stsClient.getSessionToken(stsReq);
            String stsTokenAk = stsToken.getCredentials().getAccessKeyId();
            String stsTokenSk = stsToken.getCredentials().getSecretAccessKey();
            String stsTokenSessionToken = stsToken.getCredentials().getSessionToken();
      
            // BOS服务的Endpoint地址,及Bucket名称。
            String bosEndpoint = "http://bj.bcebos.com";
            String bucketName = "Bucket名称";
            if (bosRequestType.equalsIgnoreCase("download-processed")) {
                // the binded image processing domain set by App developer on bce console
                bosEndpoint = "http://" + bucketName + ".bj.bcebos.com";
            }
      
            // prefix is the bucket name, and does not specify the object name
            BosInfo bosInfo = new BosInfo(stsTokenAk, stsTokenSk, stsTokenSessionToken, bosEndpoint,
                    bucketName, "", bucketName);
      
            String res = "";
            ObjectMapper mapper = new ObjectMapper();
            try {
                res = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(bosInfo);
            } catch (JsonProcessingException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
                res = "";
            }
            System.out.println(res);
            try {
                res = new String(res.getBytes(), "utf8");
            } catch (UnsupportedEncodingException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
                res = "";
            }
            return res;
        }
    3. 将修改完成的服务器代码重新编译打包为bos_meitu_app_server.jar,将jar包上传到应用服务器上并执行命令java -jar bos_meitu_app_server.jar

    交互过程

    上传图片到BOS

    上传图片到BOS过程中APP、APP Server和BOS的交互过程如下图所示:

    1. APP上传图片时向APP Server发送获取上传方式请求。
    2. APP Server向STS服务器请求BOS使用STS访问的AK/SK,STS服务器向APP Server返回STS凭证,包括临时AK、SK和Session Token。
    3. APP Server将STS凭证及上传方式参数返回,上传方式参数包含Bucket名称、Endpoint等。
    4. APP根据返回的信息将文件上传到BOS上,BOS会将上传结果返回给APP。
    5. APP可以根据需要将上传结果提供给APP Server。

    从BOS下载图片

    从BOS下载图片过程中APP、APP Server和BOS的交互过程如下图所示:

    1. APP下载图片时向APP Server发送获取下载方式请求。
    2. APP Server向STS服务器请求BOS使用STS访问的AK/SK,STS服务器向APP Server返回STS凭证,包括临时AK、SK和Session Token。
    3. APP Server将STS凭证及下载方式参数返回APP,下载方式参数包含Bucket名称、Endpoint等。
    4. APP根据返回的信息
    5. 从BOS上下载文件,BOS会将下载结果返回给APP。
    6. APP可以根据需要将下载结果提供给APP Server。

    从BOS下载缩放图

    从BOS下载缩放图过程中APP、APP Server和BOS的交互过程如下图所示:

    从BOS下载缩放图和从BOS上下载图片交互过程基本类似,只是在从BOS下载缩放图时需要携带APP上设定的图片处理参数,如图片宽、高和旋转角度等。

    示例代码

    示例代码以Java语言为例讲解美图APP的实现,代码分为APP客户端和应用服务器端两部分。

    APP客户端代码样例

    APP端代码主要包括BOSClient初始化、从APP Server端获取BOS信息、及上传文件到BOS三个功能模块。

    BOSClient初始化

    public class bos {
    	private string ak = null;
        private string sk = null;
        private string endpoint = null;
        private string ststoken = null;
        private bosclient client = null;
    
        public bos(string ak, string sk, string endpoint, string ststoken) {
            this.ak = ak;
            this.sk = sk;
            this.endpoint = endpoint;
            this.ststoken = ststoken;
            client = createclient();
        }
    	
        public bosclient createclient() {
            bosclientconfiguration config = new bosclientconfiguration();
            bcecredentials credentials = null;
            if (ststoken != null && !ststoken.equalsignorecase("")) {
                credentials = new defaultbcesessioncredentials(ak, sk, ststoken);
            } else {
                credentials = new defaultbcecredentials(ak, sk);
            }
            config.setendpoint(endpoint);
            config.setcredentials(credentials);
            return new bosclient(config);
        }
    	
        public void uploadfile(string bucket, string object, file file) {
            client.putobject(bucket, object, file);
        }
    	
        public void uploadfile(string bucket, string object, inputstream inputstream) {
            client.putobject(bucket, object, inputstream);
        }
    	
        public void uploadfile(string bucket, string object, byte[] data) {
            client.putobject(bucket, object, data);
        }
    	
        public byte[] downloadfilecontent(string bucket, string object) {
            return client.getobjectcontent(bucket, object);
        }
    }

    上传文件到BOS代码实现

    public void uploadPicToBos() {
        // 1. get pic params from ui: file name, file location uri etc
        // 2. send params to app server and get sts, bucket name and region
        // 3. upload selected pic to bos with sts etc, which bos client needs
    
        EditText et = (EditText) findViewById(R.id.app_server_addr_edittext);
        final String appServerAddr = et.getText().toString();
    
    	    new Thread(new Runnable() {
            @Override
            public void run() {
                Map<String, Object> bosInfo = AppServer.getBosInfoFromAppServer(appServerAddr, "user-demo",
                        AppServer.BosOperationType.UPLOAD);
    
                if (bosInfo == null) {
                    return;
                }
                showToast(bosInfo.toString(), Toast.LENGTH_LONG);
    
                String ak = (String) bosInfo.get("ak");
                String sk = (String) bosInfo.get("sk");
                String stsToken = (String) bosInfo.get("stsToken");
                String endpoint = (String) bosInfo.get("endpoint");
                String bucketName = (String) bosInfo.get("bucketName");
                String objectName = (String) bosInfo.get("objectName");
                String prefix = (String) bosInfo.get("prefix");
                Log.i("UploadFileToBos", bosInfo.toString());
    
                // specify a object name if the app server does not specify one
                if (objectName == null || objectName.equalsIgnoreCase("")) {
                    objectName = ((EditText) findViewById(R.id.bos_object_name_edittext)).getText().toString();
                    if (prefix != null && !prefix.equalsIgnoreCase("")) {
                        objectName = prefix + "/" + objectName;
                    }
                }
    
                Bos bos = new Bos(ak, sk, endpoint, stsToken);
                try {
                    byte[] data = Utils.readAllFromStream(MainActivity.this.getContentResolver().openInputStream(selectedPicUri));
                    bos.uploadFile(bucketName, objectName, data);
                } catch (Throwable e) {
                    Log.e("MainActivity/Upload", "Failed to upload file to bos: " + e.getMessage());
                    showToast("Failed to upload file: " + e.getMessage());
                    return;
                }
                // finished uploading file, send a message to inform ui
                handler.sendEmptyMessage(UPLOAD_FILE_FINISHED);
            }
        }).start();
    }

    从APP Server上获取BOS信息代码实现

    public class AppServer {
        /**
         * get info from app server for the file to upload to or download from BOS
         *
         * @param appServerEndpoint app server
         * @param userName          the app user's name, registered in app server
         * @param bosOperationType  download? upload? or?
         * @return STS, and BOS endpoint, bucketName, prefix, path, object name etc
         */
        public static Map<String, Object> getBosInfoFromAppServer(String appServerEndpoint, String userName, BosOperationType bosOperationType) {
            String type = "";
            switch (bosOperationType) {
                // to simplify
                case UPLOAD: {
                    type = "upload";
                    break;
                }
                case DOWNLOAD: {
                    type = "download";
                    break;
                }
                case DOWNLOAD_PROCESSED: {
                    type = "download-processed";
                    break;
                }
                default:{
                    break;
                }
            }
            // TODO: this url should be url encoded
            String appServerUrl = appServerEndpoint + "/?" + "userName=" + userName + "&command=stsToken&type=" + type;
    
            // create a http client to contact app server to get sts
            HttpParams httpParameters = new BasicHttpParams();
            HttpClient httpClient = new DefaultHttpClient(httpParameters);
    	
            HttpGet httpGet = new HttpGet(appServerUrl);
            httpGet.addHeader("User-Agent", "bos-meitu-app/demo");
            httpGet.setHeader("Accept", "*/*");
            try {
                httpGet.setHeader("Host", new URL(appServerUrl).getHost());
            } catch (MalformedURLException e) {
                e.printStackTrace();
            }
            httpGet.setHeader("Accept-Encoding", "identity");
    	
            Map<String, Object> bosInfo = new HashMap<String, Object>();
            try {
                HttpResponse response = httpClient.execute(httpGet);
                if (response.getStatusLine().getStatusCode() != 200) {
                    return null;
                }
                HttpEntity entity = response.getEntity();
                long len = entity.getContentLength();
                InputStream is = entity.getContent();
                int off = 0;
                byte[] b = new byte[(int) len];
                while (true) {
                    int readCount = is.read(b, off, (int) len);
                    if (readCount < 0) {
                        break;
                    }
                    off += readCount;
                }
                Log.d("AppServer", new String(b, "utf8"));
                JSONObject jsonObject = new JSONObject(new String(b, "utf8"));
                Iterator<String> keys = jsonObject.keys();
                while (keys.hasNext()) {
                    String key = keys.next();
                    bosInfo.put(key, jsonObject.get(key));
                }
            } catch (IOException e) {
                e.printStackTrace();
                return null;
            } catch (JSONException e) {
                e.printStackTrace();
                return null;
            }
            return bosInfo;
        }
    	
        public enum BosOperationType {
            UPLOAD,
            DOWNLOAD,
            DOWNLOAD_PROCESSED,
        }
    }

    APP Server端代码样例

    APP Server端基于Jetty框架,接口主要处理Android APP获取BOS信息的请求。APP Server端会返回临时AK、SK、Session Token、bucket名称、资源路径和资源请求的Endpoint等参数。以下为Jetty处理的代码示例。

    @Override
    public void handle(String target, Request baseRequest, HttpServletRequest request, HttpServletResponse response)
            throws IOException, ServletException {
    
        // Inform jetty that this request has now been handled
        baseRequest.setHandled(true);
    
        if (!request.getMethod().equalsIgnoreCase("GET")) {
            response.setStatus(HttpServletResponse.SC_NOT_IMPLEMENTED);
            return;
        }
    
        // expected url example: localhost:8080/?command=stsToken&type=download
        Map<String, String[]> paramMap = request.getParameterMap();
        if (paramMap.get("command") == null || paramMap.get("type") == null) {
            // invalid request
            response.setStatus(HttpServletResponse.SC_BAD_REQUEST);
            return;
        }
    
        if (!paramMap.get("command")[0].equalsIgnoreCase("stsToken")) {
            response.setStatus(HttpServletResponse.SC_NOT_IMPLEMENTED);
            return;
        }
    
        String responseBody = "";
        responseBody = getBosInfo(paramMap.get("type")[0]);
    
        // Declare response encoding and types
        response.setContentType("application/json; charset=utf-8");
    
        // Declare response status code
        response.setStatus(HttpServletResponse.SC_OK);
    
        // Write back response, utf8 encoded
        response.getWriter().println(responseBody);
    }
    
    /**
     * Generates bos info needed by app according to requset type(upload, download etc)
     * this is the key part for uploading file to bos with sts token
     * @param bosRequestType 
     * @return utf8 encoded json string
     */
    public String getBosInfo(String bosRequestType) {
        // configuration for getting stsToken
        // bce bos credentials ak sk
        String bosAk = "your_bos_ak";
        String bosSk = "your_bos_sk";
        // bce sts service endpoint
        String stsEndpoint = "http://sts.bj.baidubce.com";
    
        BceCredentials credentials = new DefaultBceCredentials(bosAk, bosSk);
        BceClientConfiguration clientConfig = new BceClientConfiguration();
        clientConfig.setCredentials(credentials);
        clientConfig.setEndpoint(stsEndpoint);
        StsClient stsClient = new StsClient(clientConfig);
        GetSessionTokenRequest stsReq = new GetSessionTokenRequest();
        // request expiration time
        stsReq.setDurationSeconds(1800);
        GetSessionTokenResponse stsToken = stsClient.getSessionToken(stsReq);
        String stsTokenAk = stsToken.getCredentials().getAccessKeyId();
        String stsTokenSk = stsToken.getCredentials().getSecretAccessKey();
        String stsTokenSessionToken = stsToken.getCredentials().getSessionToken();
    
        // **to simplify this demo there is no difference between "download" and "upload"**
        // parts of bos info
        String bosEndpoint = "http://bj.bcebos.com";
        String bucketName = "bos-android-sdk-app";
        if (bosRequestType.equalsIgnoreCase("download-processed")) {
            // the binded image processing domain set by App developer on bce console
            bosEndpoint = "http://" + bucketName + ".bj.bcebos.com";
        }
    
        // prefix is the bucket name, and does not specify the object name
        BosInfo bosInfo = new BosInfo(stsTokenAk, stsTokenSk, stsTokenSessionToken, bosEndpoint,
                bucketName, "", bucketName);
    
        String res = "";
        ObjectMapper mapper = new ObjectMapper();
        try {
            res = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(bosInfo);
        } catch (JsonProcessingException e) {
            e.printStackTrace();
            res = "";
        }
        try {
            res = new String(res.getBytes(), "utf8");
        } catch (UnsupportedEncodingException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            res = "";
        }
        return res;
    }