Android投屏(屏幕共享)設計需要考慮的幾個關鍵因素

在做智慧教室同屏、會議同屏之類的方案時，基於Andriod平台的采集，往往遇到各種各樣的問題，以下就幾個點，拋磚引玉：

1. 內網環境下，組播還是RTMP？

回答：這個問題，被無數的開發者問到，為此，單獨寫了篇博客論證：https://blog.csdn.net/renhui1112/article/details/86741428，感興趣的可以參考下，簡單來說，能RTMP的，就RTMP，如果真是內網環境下，沒有並發瓶頸的同屏，可以啟動內置RTSP服務（走單播），然后，其他終端拉流也不失為一個好的方案。

2. 推送分辨率如何設定或縮放？

回答：一般來說，好多Android設備，特別是高分屏，拿到的視頻原始寬高非常大，如果推原始分辨率，編碼和上行壓力大，所以，一般建議，適當縮放，比如寬高縮放至2/3，縮放一般建議等比例縮放，此外，縮放寬高建議16字節對齊。

廢話不多說，上實例代碼：

    private void createScreenEnvironment() {

        sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();
        screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();

        Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "
                + screenWindowHeight);

        if (sreenWindowWidth > 800)
        {
            if (screenResolution == SCREEN_RESOLUTION_STANDARD)
            {
                scale_rate = SCALE_RATE_HALF;
                sreenWindowWidth = align(sreenWindowWidth / 2, 16);
                screenWindowHeight = align(screenWindowHeight / 2, 16);
            }
            else if(screenResolution == SCREEN_RESOLUTION_LOW)
            {
                scale_rate = SCALE_RATE_TWO_FIFTHS;
                sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);

            }
        }

        Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);

        int pf = mWindowManager.getDefaultDisplay().getPixelFormat();
        Log.i(TAG, "display format:" + pf);

        DisplayMetrics displayMetrics = new DisplayMetrics();
        mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);
        mScreenDensity = displayMetrics.densityDpi;

        mImageReader = ImageReader.newInstance(sreenWindowWidth,
                screenWindowHeight, 0x1, 6);

        mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);
    }

3. 橫豎屏自動適配

回答：因為橫豎屏狀態下，采集的屏幕寬高不一樣，如果橫豎屏切換，這個時候，需要考慮到橫豎屏適配問題，確保比如豎屏狀態下，切換到橫屏時，推拉流兩端可以自動適配，橫豎屏自動適配，編碼器需要重啟，拉流端，需要能自動適配寬高變化，自動播放。

4. 一定的補幀策略

回答：好多人不理解為什么要補幀，實際上，屏幕采集的時候，屏幕不動的話，不會一直有數據下去，這個時候，比較好的做法是，保存最后一幀數據，設定一定的補幀間隔，確保不會因為幀間距太大，導致播放端幾秒都收不到數據，當然，如果服務器可以緩存GOP，這個問題迎刃而解。

5. 異常網絡處理、事件回調機制

回答：如果是走RTMP，網絡抖動或者其他網絡異常，需要有好重連機制和狀態回饋機制。

    class EventHandeV2 implements NTSmartEventCallbackV2 { @Override public void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) { Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id); String publisher_event = ""; switch (id) { case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED: publisher_event = "開始.."; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING: publisher_event = "連接中.."; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED: publisher_event = "連接失敗.."; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED: publisher_event = "連接成功.."; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED: publisher_event = "連接斷開.."; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP: publisher_event = "關閉.."; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE: publisher_event = "開始一個新的錄像文件 : " + param3; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED: publisher_event = "已生成一個錄像文件 : " + param3; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY: publisher_event = "發送時延: " + param1 + " 幀數:" + param2; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE: publisher_event = "快照: " + param1 + " 路徑：" + param3; if (param1 == 0) { publisher_event = publisher_event + "截取快照成功.."; } else { publisher_event = publisher_event + "截取快照失敗.."; } break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL: publisher_event = "RTSP服務URL: " + param3; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE: publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3; break; case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT: publisher_event ="服務器不支持RTSP推送, 推送的RTSP URL: " + param3; break; } String str = "當前回調狀態：" + publisher_event; Log.i(TAG, str); Message message = new Message(); message.what = PUBLISHER_EVENT_MSG; message.obj = publisher_event; handler.sendMessage(message); } }

6. 部分屏幕數據采集

回答：我們遇到的好多場景下，教室端，會拿出來3/4的區域用來投遞給學生看，1/4的區域，用來做一些指令等操作，這個時候，就需要考慮屏幕區域裁剪，接口可做如下設計：

	/** * 投遞裁剪過的RGBA數據 * * @param data: RGBA data * * @param rowStride: stride information * * @param width: width * * @param height: height * * @param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的寬; clipedHeight: 裁剪后的高; 確保傳下去裁剪后的寬、高均為偶數 * * @return {0} if successful */ public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight); 

                        //實際裁剪比例，可酌情自行調整 int left = 100; int cliped_left = 0; int top = 0; int cliped_top = 0; int cliped_width = width_; int cliped_height = height_; if(scale_rate == SCALE_RATE_HALF) { cliped_left = left / 2; cliped_top = top / 2; //寬度裁剪后，展示3/4比例 cliped_width = (width_ *3)/4; //高度不做裁剪 cliped_height = height_; } else if(scale_rate == SCALE_RATE_TWO_FIFTHS) { cliped_left = left * 2 / 5; cliped_top = top * 2 / 5; //寬度裁剪后，展示3/4比例 cliped_width = (width_ *3)/4; //高度不做裁剪 cliped_height = height_; } if(cliped_width % 2 != 0) { cliped_width = cliped_width + 1; } if(cliped_height % 2 != 0) { cliped_height = cliped_height + 1; } if ( (cliped_left + cliped_width) > width_) { Log.e(TAG, " invalid cliped region settings, cliped_left: " + cliped_left + " cliped_width:" + cliped_width + " width:" + width_); return; } if ( (cliped_top + cliped_height) > height_) { Log.e(TAG, "invalid cliped region settings, cliped_top: " + cliped_top + " cliped_height:" + cliped_height + " height:" + height_); return; } //Log.i(TAG, " clipLeft: " + cliped_left + " clipTop: " + cliped_top + " clipWidth: " + cliped_width + " clipHeight: " + cliped_height); libPublisher.SmartPublisherOnCaptureVideoClipedRGBAData(publisherHandle, last_buffer, row_stride_, width_, height_, cliped_left, cliped_top, cliped_width, cliped_height );

7. 文字、圖片水印

回答：好多場景下，同屏者會把公司logo，和一定的文字信息展示在推送端，這個時候，需要考慮到文字和圖片水印問題，具體可參考如下接口設置：

   /** * Set Text water-mark(設置文字水印) * * @param fontSize: it should be "MEDIUM", "SMALL", "BIG" * * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT". * * @param xPading, yPading: the distance of the original picture. * * <pre> The interface is only used for setting font water-mark when publishing stream. </pre> * * @return {0} if successful */ public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading); /** * Set Text water-mark font file name(設置文字水印字體路徑) * * @param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf * * @return {0} if successful */ public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName); /** * Set picture water-mark(設置png圖片水印) * * @param picPath: the picture working path, e.g: /sdcard/logo.png * * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT". * * @param picWidth, picHeight: picture width & height * * @param xPading, yPading: the distance of the original picture. * * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre> * * @return {0} if successful */ public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);

總結：其實一個好的同屏系統，需要考慮的地方遠不止以上幾點，比如編碼參數策略等，都需要考量，后續有機會再和大家做進一步分享。