參考：登上《Cell》封面的AI醫療影像診斷系統：機器之心專訪UCSD張康教授
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning 2018-2-22 Cell
讀《Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning》
沒有問題就無法學習：
1. 文中的數據規模是多少？這些數據有什么特點？
2. 為什么要使用遷移學習？
3. 文章是如何驗證自己模型的准確性的？
4. 摘要中的每一個結論，在文章都是如何具體論證的？
亮點：
1，首次使用如此龐大的標注好的高質量數據進行遷移學習，來建立人工智能系統；
2，基於光學相干斷層成像（OCT）數據有效進行圖像黃斑變性和糖尿病視網膜黃斑水腫的識別和嚴重性定量評估;
3，准確區分患兒胸部X光片上的細菌和病毒性肺炎（差異性分析和准確判斷）
目標：
利用遷移學習識別視網膜（OCT）圖像中的類別，同時也用遷移學習測試了胸部X光片的肺炎識別。
該研究也通過顯示神經網絡激活區域的方法向人們提供了機器診斷的可解釋性。
真正啟動是在去年（2017 年）初。啟動到發表總共1年。
機器之心：神經網絡的推理是一個「黑箱」，你們提出的新方法是如何解釋計算機作出「診斷」的依據的？
張康：我們在視網膜 OCT 圖像的研究中加入了「遮擋測試」——通過卷積一個遮擋核心到輸入圖像上，機器會通過計算預測做出正確診斷最可能的部位，並輸出含有高亮色塊的「遮擋」圖，這些色塊就是 AI「認為」的病變部位，得出直觀的為臨床醫生信任的診斷依據。
首先，通過輸入大量的數據，神經網絡可以獲得遠超過人類醫生的「經驗」，計算出超越人類的准確結果，在我們的系統中，我們使用超過 20 萬張醫學圖像，通過不同的疾病分類，最終使用近 11 萬張視網膜 OCT 圖像訓練機器。在眼病方面，能在 30 秒內正確鑒別脈絡膜新生血管、糖尿病黃斑水腫、玻璃膜疣以及正常視網膜的 OCT 圖像，結果的准確率、敏感度、特異度均在 95% 以上，並能得出與人類相似甚至更高的准確率。其次，計算機對比圖像像素與像素之間的差異，觀察到人類關注不到的細節，從而得出更精准的判斷，且不像人類一樣受主觀性干擾。另外，我們通過「遷移學習」這種算法，還能診斷不同系統的疾病，比如我們的系統目前還能准確鑒別肺炎和正常胸部 X 線平片，區分肺炎的病原體為細菌還是病毒，准確率可達 90% 以上。
「遷移學習」被認為是一種高效的學習技術，尤其是面臨相對有限的訓練數據時。相較於其他大多數學習模型的「從零開始」，「遷移學習」利用卷積神經網絡（Convolutional Neural Network，CNN）學習已有的已經標記好的預訓練網絡系統，以醫學圖像學習為例，該系統會識別預系統中圖像的特點，我們再繼續導入含有第一層圖像相似參數和結構的網絡系統，最終構建出終極層級。在我們的系統中，第一層網絡就是視網膜 OCT 圖像，第二級網絡系統使用第一級的圖像尋找相應的特點，通過前向傳播固定低層圖像中的權重，找到已經學習的可辨別的結構，再提取更高層的權重，在其中進行反復的自我調整和反饋、傳遞，達到學習區分特定類型的圖像的目的。我們首次使用如此龐大的標注好的高質量視網膜 OCT 數據進行遷移學習，進行常見視網膜致盲性疾病的檢測及推薦治療手段，得到與人類醫生相似甚至更高的准確性。此人工智能系統還可以「舉一反三」，將遷移學習用於小兒肺炎診斷。
遷移學習是深度學習的一個自然發展方向，遷移學習能讓深度學習變得更加可靠，還能幫我們理解深度學習的模型。比如，我們能夠知曉哪部分特征容易遷移，這些特征所對應的是某個領域比較高層、抽象的一些結構型概念。把它們的細節區分開，就能讓我們對這個領域的知識表達形成一個更深的理解。這樣一來，機器就可以像生物的神經系統一樣終身學習，不斷地對過去的知識進行總結、歸納，讓一個系統越學越快，而且在學習過程中還能發現如何學習。
遷移學習在深度學習上面有極為廣闊的應用前景，在圖像數據資源有限的醫療領域，更高效、所需圖像數量更少的遷移學習，可以說是未來 5 年內 AI 發展的熱點以及深度學習成功應用的驅動力。
比如，有一些圖像特征較為模糊的圖像，如老年黃斑變性，某些較大的玻璃膜疣和脈絡膜新生血管非常相似，我們就會偏向於采取更為嚴重的疾病診斷，因為我們研究的最終目的是幫助病人更可能的推薦給相應的專科醫生，從而更快的獲得治療。另外，我們還可以通過我們的想法設定更為貼合實際的過濾器，並按照我們臨床醫生的需求不斷調整；通過「遮擋實驗」能夠反映機器得出判斷的依據。並且，我們的研究還能指導治療方案的確定。因此我們的研究可能更能達到臨床醫生想要的效果，並且為臨床醫生所信任，也許能更快更直接的應用於臨床。
學習以下神經網路，適合中國人的視頻，簡直不要太通俗：遷移學習 Transfer Learning
學習方法及資料：https://github.com/jindongwang/transferlearning
Cell paper數據和代碼地址：https://data.mendeley.com/datasets/rscbjbr9sj/2
跑一下代碼玩一下！
文章原文解讀：
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
鑒定醫療診斷和可治療疾病
Image-based deep learning classifies macular degeneration（[眼科] 黃斑變性） and diabetic retinopathy（[眼科] 糖尿病視網膜病變） using retinal optical coherence tomography（視網膜光學相干斷層掃描） images and has potential for generalized applications in biomedical image interpretation and medical decision making.
- An artificial intelligence system using transfer learning（遷移學習） techniques was developed
- It effectively classified images for macular degeneration and diabetic retinopathy
- It also accurately distinguished bacterial and viral pneumonia（細菌和病毒性肺炎） on chest X-rays
- This has potential for generalized high-impact application in biomedical imaging
科普下遷移學習：什么是遷移學習 (Transfer Learning)？這個領域歷史發展前景如何？
The implementation of clinical-decision support algorithms for medical imaging faces challenges with reliability and interpretability.
Here, we establish a diagnostic tool based on a deep-learning framework for the screening of patients with common treatable blinding retinal diseases.
Our framework utilizes transfer learning, which trains a neural network with a fraction of the data of conventional approaches（常規方法）. Applying this approach to a dataset of optical coherence tomography images, we demonstrate performance comparable to that of human experts in classifying agerelated macular degeneration and diabetic macular edema. We also provide a more transparent and interpretable diagnosis by highlighting the regions recognized by the neural network. We further demonstrate the general applicability of our AI system for diagnosis of pediatric pneumonia using chest X-ray images.
This tool may ultimately aid in expediting the diagnosis and referral of these treatable conditions, thereby facilitating earlier treatment, resulting in improved clinical outcomes.
Figure 2. Representative Optical Coherence Tomography Images and the Workflow Diagram
結果
The primary application of our transfer learning algorithm was in the diagnosis of retinal OCT images.
Spectral-domain OCT uses light to capture high-resolution in vivo optical cross sections of the retina that can be assembled into three-dimensional-volume images of living retinal tissue. OCT is critical to guiding the administration of anti-VEGF therapy by providing a clear cross-sectional representation of the retinal pathology in these conditions (Figure 2A), allowing visualization of individual retinal layers, which is impossible with clinical examination by the human eye or by color fundus photography.
主要就干了以下四件事：
Patient and Image Characteristics
數據：We initially obtained 207,130 OCT images. 108,312 images (37,206 with choroidal neovascularization, 11,349 with diabetic macular edema, 8,617 with drusen, and 51,140 normal) from 4,686 patients passed initial image quality review and were used to train the AI system.
After 100 epochs (iterations through the entire dataset), the training was stopped due to the absence of further improvement in both accuracy (Figure 3A) and cross-entropy loss (Figure 3B).
Performance of the Model
We evaluated our AI system in diagnosing the most common blinding retinal diseases.
Comparison of the Model with Human Experts
An independent test set of 1,000 images from 633 patients was used to compare the AI network’s referral decisions with the decisions made by human experts.
Occlusion Testing遮擋測試
We performed an occlusion test on 491 images to identify the areas contributing most to the neural network’s assignment of the predicted diagnosis.
Application of the AI System for Pneumonia Detection Using Chest X-Ray Images
To investigate the generalizability of our AI system in the diagnosis of common diseases, we applied the same transfer learning framework to the diagnosis of pediatric pneumonia.
案例二：In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images
- transmitted-light
- fluorescence images
計算機中的標注：預測熒光標簽在沒被標注的圖像中
Seeing More with In Silico Labeling of Microscopy Images
Multi-scale architectures ：可以用於處理多重輸入，平時我們都是輸入一張圖片，現在我們可以輸入多張圖片。
In silico labeling, a machine-learning approach, reliably infers fluorescent measurements from transmitted-light（透射光） images of unlabeled fixed or live biological samples.
Microscopy is a central method in life sciences.
Many popular methods, such as antibody labeling, are used to add physical fluorescent labels to specific cellular constituents.
However, these approaches have significant drawbacks, including inconsistency; limitations in the number of simultaneous labels because of spectral overlap; and necessary perturbations of the experiment, such as fixing the cells, to generate the measurement.
Here, we show that a computational machine-learning approach, which we call ‘‘in silico labeling’’ (ISL), reliably predicts some fluorescent labels from transmitted-light images of unlabeled fixed or live biological samples. ISL predicts a range of labels, such as those for nuclei, cell type (e.g., neural), and cell state (e.g., cell death). Because prediction happens in silico, the method is consistent, is not limited by spectral overlap, and does not disturb the experiment. ISL generates biological measurements that would otherwise be problematic or impossible to acquire.
As such, it is unclear whether deep learning approaches would provide a significant and broad-based advance in image analysis and are capable of extracting useful, not readily apparent, information from unlabeled images.
we show it can accurately predict the location and texture of cell nuclei, the health of a cell, the type of cell in a mixture, and the type of subcellular structure.
We also show that the trained network exhibits transfer learning: once trained to predict a set of labels, it could learn new labels with a small number of additional data, resulting in a highly generalizable algorithm, adaptable across experiments.
Training and Testing Datasets for Supervised Machine Learning
Developing Predictive Algorithms with Machine Learning
Network Predictions of Cell Nuclei
Network Predictions of Cell Viability
Network Predictions of Cell Type and Subcellular Process Type
Adapting the Generic Learned Network to New Datasets: Transfer Learning
待續~