美團外賣APP爬蟲算法

抓包

經過對數據包的分析比對，可以看到每次請求都會自動生成相應的請求參數，並計算出一個加密后的參數__skcy，服務端根據這些參數信息和請求內容做校驗，校驗通過服務端才會返回正確的結果。

找到計算__skcy關鍵函數

經過搜索關鍵字我們進入到
CandyPreprocessor類的getParametersSignature方法

private String getParametersSignature(Builder builder, Context context) throws Exception {
        Object[] objArr = new Object[]{builder, context};
        ChangeQuickRedirect changeQuickRedirect = changeQuickRedirect;
        String str = "d7fd4e92b3bd07b96007e804b4226165";
        if (PatchProxy.isSupport(objArr, this, changeQuickRedirect, false, str, 6917529027641081856L)) {
            return (String) PatchProxy.accessDispatch(objArr, this, changeQuickRedirect, false, str);
        }
        if (builder != null) {
            Object baseString = baseString();
            if (TextUtils.isEmpty(baseString)) {
                throw new Exception("CandyPreprocessor getParametersSignature normalizedURI is null");
            }
            List arrayList = new ArrayList();
            appendList(arrayList, builder, false);
            if (this.version == CandyVersion.Ver1_0) {
                arrayList.add(new MyEntry("__sksc", this.candyOriginalMaterial.getScheme()));
            }
            if (formURLEncoded() != null) {
                builder = new StringBuilder("/?");
                builder.append(new String(this.candyOriginalMaterial.getPostContent()));
                appendList(arrayList, Uri.parse(builder.toString()).buildUpon(), true);
            }
            builder = getPercentList(arrayList);
            dictionarySort(builder);
            builder = getNormalizedParameters(builder);
            StringBuilder stringBuilder = new StringBuilder();
            stringBuilder.append(this.candyOriginalMaterial.getHttpMethod());
            stringBuilder.append(StringUtil.SPACE);
            stringBuilder.append(baseString);
            stringBuilder.append(StringUtil.SPACE);
            stringBuilder.append(builder);
            builder = stringBuilder.toString();
            if (formURLEncoded()) {
                builder = builder.getBytes();
            } else if (this.candyOriginalMaterial.getPostContent() == null) {
                builder = builder.getBytes();
            } else {
                builder = builder.getBytes();
                Builder builder2 = new byte[(builder.length + this.candyOriginalMaterial.getPostContent().length)];
                System.arraycopy(builder, 0, builder2, 0, builder.length);
                System.arraycopy(this.candyOriginalMaterial.getPostContent(), 0, builder2, builder.length, this.candyOriginalMaterial.getPostContent().length);
                builder = builder2;
            }
            return CandyJni.getCandyDataWithKeyForJava(context, builder, "CandyKey");
        }
        throw new Exception("CandyPreprocessor getParametersSignature builder is null");
    }

在接下來的跳轉鏈之后，我們又找到了CandyJni的getCandyDataWithKeyForJava方法：

public static String getCandyDataWithKeyForJava(Context context, byte[] bArr, String str) {
Object[] objArr = new Object[]{context, bArr, str};
        ChangeQuickRedirect changeQuickRedirect = changeQuickRedirect;
        String str2 = "8806cdcfdd305bd7b7224b07a9fb85e3";
        if (PatchProxy.isSupport(objArr, null, changeQuickRedirect, true, str2, 6917529027641081856L)) {
            return (String) PatchProxy.accessDispatch(objArr, null, changeQuickRedirect, true, str2);
        }
        if (MTGuard.selfExceptionCheck() && bArr != null) {
            if (bArr.length != 0) {
                return getCandyDataWithKey(context, bArr, str);
            }
        }
        return null;
}

然后我們進入

public static native String getCandyDataWithKey(Object obj, byte[] bArr, String str);

這是一個本地方法,因此，我們需要在原生代碼中找到getCandyDataWithKey方法。

打開so文件

經過靜態分析 我們得知是在getCandyDataWithKey是在 libmtguard.so 中，我們用IDA打開它

我們在導出函數列表中可以看到，只有JNI_Onload，

動態調試還原算法

經過一系列的動態調試，最終還原出具體算法。請求推薦接口

成功取到數據。