雜記

Typora 上傳圖片到博客園

import requests
import json
from sys import argv

uploadUrl = 'https://upload.cnblogs.com/imageuploader/processupload?host=www.cnblogs.com'

headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36",
    "cookie": "__gads=ID=0df8844e3171cf5d:T=1626703371:S=ALNI_MbCQzZlPNaCt56Y3ZVBMgBgxo-LiA; UM_distinctid=17b02249c8337a-021de176cab159-3f3a5e08-1fa400-17b02249c84437; _ga_3Q0DVSGN10=GS1.1.1629294955.2.1.1629294963.0; _ga=GA1.2.1874840476.1626703371; .Cnblogs.AspNetCore.Cookies=CfDJ8NACB8VE9qlHm6Ujjqxvg5A9JzQnWwTQwllXBS3jsuzbCTLy2gQscie0pLfzrbMgn704PWOBvn29tPVe9SrWwTZDRV8hTTISIxYAGXMLFead8T1oy-hTGkXxDwlskdW8yHcTMpvI481DuI0BGX4LeP5jib45Xj6vGjzibGkPiBou2eNyGQt3tVlMbHaSKcAspkeQmALgMVJEB-9ZoiijWgHo8M3_6F7RIUNgXml1yjrhu9bvWHeD6-tRHOigQRAQjS4QBG0hP_LXMT7RcXKFdy8AE2ZuCqTNZzrrNjwqXM7TpNw7uXpo2t0lo7tJ68MwJsoQPDkV3XIBIqKXyhc7bbzkm-OxcdlBNseZaxVm0gge1oRIIOo8NZoWePmQjjfF2TmwHgpdoZfH6kkc1FLCsB8THIUuAQGee9p7iK3Gx1CEmAfhILRECIwPxE0wxPdjl6FWvIANUJ8rELfJfUWKfxhVhhU3So1Ywlgv9tPTKegzgUSooWyzn7Fnlb4cLkGrXM4ncoAHUOFzwpslnrwS4vp3-12wXeOb2ggb7NfVL3TlEPojFPPj1hHDRTkeOfGo9A; _gid=GA1.2.205401666.1631025338; affinity=1631273060.452.42.609693" 
}

# 類型映射
mimeMapping = {".png": 'image/png', '.gif': 'image/gif', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg'}

for i in argv[1:]:
    # 圖片地址參數
    imgPath = i

    # 對應的mime
    mime = imgPath[imgPath.rindex("."):]

    file = [
        ("",("fileName", open(imgPath, "rb"), mimeMapping[mime]))
    ]

    response = requests.post(uploadUrl,headers = headers,files = file)

    data = json.loads(response.text)
    print(data['message'])

偏好設置中需要配置為上傳服務

Flink on Yarn 集群報錯

報錯內容

 Exception in thread "Thread-5" java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:164)
        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:183)
        at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2737)
        at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:2993)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2952)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2925)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2805)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1199)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1787)
        at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
        at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
        at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)

解決方案

在conf/flink-conf.yaml,大約在192行添加

classloader.check-leaked-classloader: false

重新執行 Flink per-job cluster 調用WordCount樣例

bin/flink run -m yarn-cluster -yjm 1024 -ytm 1024 examples/batch/WordCount.jar

正常打印輸出結果.

Flink On Session 模式

• 先創建一個on Yarn的Flink 容器

bin/yarn-session.sh -jm 1024m -tm 1024m -d

執行測試PI腳本

bin/flink run ./examples/batch/WordCount.jar 

查看默認配置信息

more /tmp/.yarn-properties-root

殺掉yarn上的Flink 容器

yarn application -kill application_1631228277006_0008