darktrace 亮點是使用的無監督學習(貝葉斯網絡、聚類、遞歸貝葉斯估計)發現未知威脅——使用無人監督 機器學習反而允許系統發現罕見的和以前看不見的威脅,這些威脅本身並不依賴 不完善的訓練數據集。 學習正常數據,發現異常!


先說說他們的產品:企業免疫系統(基於異常發現來識別威脅)

可以看到是面向企業內部安全的!

優點
整個網絡拓撲的三維可視化
企業威脅級別的實時全局概述
智能地聚類異常
泛頻譜觀測 - 高階網絡拓撲;特定群集,子網和主機事件
可搜索的日志和事件
重播歷史數據
設備和外部IP的整體行為的簡明摘要
專為業務主管和安全分析師設計
100%的能見度


企業免疫系統是世界上最先進的網絡防御機器學習技術。受到人體免疫系統自我學習智能的啟發,這種新技術在復雜和普遍的網絡威脅的新時代中,使組織自我保護方式發生了根本轉變。

人體免疫系統非常復雜,並且不斷適應新形式的威脅,例如不斷變異的病毒DNA。它的工作原理是了解身體的正常情況,識別和消除那些不符合正常發展模式的異常值。

Darktrace將相同的邏輯應用於企業和工業環境。在機器學習和人工智能算法的支持下,企業免疫系統技術迭代地為網絡中的每個設備和用戶學習獨特的“生活模式”(“自我”),並將這些見解聯系起來,以發現新出現的威脅,否則這些威脅將被忽視。

與人體免疫系統一樣,企業免疫系統不需要先前的威脅或活動模式經驗,以了解它可能具有威脅性。它可以在沒有先驗知識或簽名的情況下自動工作,實時檢測並抵御網絡內部的微妙,隱秘攻擊。

 

https://www.engerati.com/system/files/7.18.18_machine_learning_in_the_era_of_cyber_ai.pdf

要點摘錄:

從一開始,Darktrace就拒絕了與歷史攻擊相關的數據可以預測未來數據的假設。相反,Darktrace的網絡AI平台使用無監督的機器學習來大規模地分析網絡數據,並根據它所看到的證據進行數十億次基於概率的計算。它不依賴於過去威脅的知識,而是獨立地對數據進行分類並檢測引人注目的模式。

 

 使用無人監督
機器學習反而允許系統發現罕見的
和以前看不見的威脅,這些威脅本身並不依賴
不完善的訓練數據集。 與歷史攻擊有關的數據
不一定能防范未來的。

它看到了而不是依賴過去威脅的知識,
由此,它形成了對“正常”的理解
整個網絡的行為,與設備,用戶有關,
或任一實體的組,並檢測與此的偏差。
不斷發展的“生活模式”可能指向一種正在發展的威脅。
Darktrace機器學習的核心原則
它了解網絡中“正常工作”中的正常情況
- 它不依賴於先前攻擊的知識。
它在現代的規模,復雜性和多樣性上蓬勃發展
企業,每個設備和人都是獨一無二的。
它將攻擊者的創新轉變為對抗他們 - 任何
異常活動是可見的。

 

具體技術除了無監督的異常檢測,聚類技術還有用於分類的深度學習技術,要點如下:

(1)使用的是貝葉斯網絡。Darktrace使用貝葉斯概率作為其中的一部分
獨特的無監督機器學習方法。

詳細如下:

Technical Overview
Darktrace’s transformative approach to cyber defense
relies on probabilistic methods developed by Cambridge
mathematicians. Employing multiple unsupervised, supervised,
and deep learning techniques in a Bayesian framework, the
Enterprise Immune System can integrate a vast number
of weak indicators of anomalous behavior to produce a single
clear measure of threat probabilities.
For each unique environment, Darktrace generates millions
of interrelated mathematical models which are correlated to
ensure that only truly anomalous behavior is detected without
a profusion of false positives. Unlike rules-based computation,
the results that probabilistic mathematics generate cannot
simply be categorized as ‘yes’ or ‘no’ but instead indicate
degrees of certainty, reflecting the ambiguities that
inevitably exist in dynamic data environments.
Ranking threat
The Enterprise Immune System accounts for ambiguities by
distinguishing between the subtly differing levels of evidence
that characterize network data. Instead of generating the
simple binary outputs ‘malicious’ or ‘benign’, Darktrace’s
mathematical algorithms produce outputs marked with
differing degrees of potential threat. This enables users of
the system to rank alerts in a rigorous manner, and prioritize
those which most urgently require action, while removing
the problem of numerous false positives associated with a
rule-based approach.
At its core, Darktrace mathematically characterizes what
constitutes ‘normal’ behavior, based on the analysis of a
large number of different measures of a device’s network
behavior, including: ——基於行為異常發現威脅。
Server access
Data volumes
Timings of events
Credential use
Connection type, volume, and directionality
Directionality of uploads/downloads
File type
Admin activity
Resource and information requests

也就是數據維度包括:

服務器訪問
數據量
活動時間
憑證使用
連接類型,大小和方向性
上傳/下載的方向性
文件類型
管理活動
資源和信息請求

咋感覺是用在企業數據保護場景里。。。

(2)使用聚類技術來識別正常的設備行為。

Darktrace采用了許多不同的聚類
方法,包括基於矩陣的聚類,基於密度的方法
聚類和層次聚類技術。該
然后使用所得到的聚類來建模
個別設備的規范行為。

Clustering devices
In order to model what should be considered as normal for a
device, its behavior is analyzed in the context of other similar
devices on the network. Darktrace leverages the power of
unsupervised machine learning to algorithmically identify
significant groupings of devices, a task which is impossible
to do manually on even modestly-sized networks.
To create a holistic image of the relationships within the
network, Darktrace employs a number of different clustering
methods, including matrix-based clustering, density-based
clustering, and hierarchical clustering techniques. The
resulting clusters are then used to inform the modeling of
the normative behaviors of individual devices.

(3)識別網絡拓撲結構中的變化
Network topology
A network is far more than the sum of its individual parts,
with much of its meaning contained in the relationships
among its different entities. Darktrace employs many
mathematical methods to model the multiple facets of a
network’s topology, allowing it to track subtle changes in
structure that are indicative of threats.(識別網絡拓撲結構中的些微變化)
One approach is based on iterative matrix methods that
reveal important connectivity structures within the network,
in a similar way to advanced page-ranking algorithms.
In tandem with these, Darktrace has developed innovative
applications of models from the field of statistical physics,
which allows the modeling of a network’s ‘energy landscape’
to reveal anomalous substructures that could represent
the first symptoms of compromise.(發現異常子結構)

(4)識別網絡中異常行為,應該是根據網絡協議、IP等識別異常的流量。

Network structure
A further important challenge in modeling the behaviors of a
dynamically evolving network is the huge number of potential
predictor variables. For the observation of packet traffic and
host activity within an enterprise LAN or WAN, where both
input and output can contain many inter-related features
(protocols, source and destination machines, log changes,
and rule triggers etc.協議,源和目標機器,日志更改,

和規則觸發器等), learning a sparse and consistent

structured predictive function is crucial.——預測網絡流量嗎?
In this context, Darktrace employs a cutting-edge large-scale
computational approach to understand sparse structure
in models of network connectivity based on applying L1-
regularization techniques (the lasso method). This allows
the Enterprise Immune System to discover true associations
between different elements of a network(發現網絡元素之間的關系) which can be cast
as efficiently solvable convex optimization problems and
yield parsimonious models.

(5)使用遞歸貝葉斯估計來發現網絡設備(狀態、行為)的時間演進關系

(https://blog.csdn.net/Young_Gy/article/details/78642271 感覺RBE就是求解xt和xt-1之間的關系,無非就是用到了貝葉斯概率而已

Recursive Bayesian Estimation
To combine these multiple analyses of network behavior, (
生成網絡設備的全面狀態圖)

generating a single comprehensive picture of the state of the
devices that comprise a network, Darktrace leverages the
power of Recursive Bayesian Estimation (RBE). Using RBE,
Darktrace’s mathematical models are able to constantly
adapt to new information as it becomes available to the
system. Continually recalculating threat levels in the light
of new data, the Enterprise Immune System can discern
significant patterns in data flows indicative of attacks, where
conventional signature-based methods see only chaos.傳統的簽名方法只能看到混亂。

(6)他們也使用了深度學習的分類技術

Darktrace & Deep Learning
Darktrace also uses deep learning to enhance modeling
processes. Deep learning is a subset of machine learning
that uses the cascading interactions of layered mathematical
processes – known as neural nets – to give intelligent
systems a higher degree of insight. Multi-layered neural
nets can improve the detection and remediation of certain
threats, for example, in the identification of DNS anomalies,
which are less effectively tracked by other machine learning
methods. Darktrace’s deep learning system assigns a score
to all DNS data from a device, with the purpose of identifying
suspicious activity even faster.(識別DNS異常,其他機器學習不太有效地跟蹤它們。 分析來自設備的所有DNS數據,用於識別

DNS可疑活動。

Darktrace also clusters devices into peer groups, based on
its own understanding of how those devices behave, and
uses supervised learning to uncover sequences of breaches,
unusual patterns, or to detect aberrant activity at a higher,(
對這些設備的行為方式的理解,以及使用有監督的學習來發現違規行為,

不尋常的模式,或檢測更高的異常活動

more holistic level. For example, the WannaCry ransomware
was easily detected by Darktrace as it breaches a number of
different ‘pattern of life’ models. Using supervised learning,
Darktrace can replicate the process of a human interpreting
various sets of breaches for a device or network over time
and so present correlated alerts instead of a multitude.
Supervised learning is also used by Darktrace to understand
more about the environment, without a human having to label
it. By observing millions of different smartphones, for example,
Darktrace gets faster and faster at identifying a new device as a
‘smartphone’, and even what type of smartphone it is.
Using deep and supervised techniques to complement its core
unsupervised machine learning algorithms, Darktrace builds
up unique, contextual knowledge about network activity and
integrates the insights of our global deployments to improve
threat detection.
Finally, Darktrace also uses deep learning techniques to
automate repetitive and time-consuming tasks carried out
during investigation workflows. By analyzing how seasoned
cyber analysts interact with the Threat Visualizer, triage
alerts, and leverage third-party sources, Darktrace is able
to replicate those expert behaviors and automate certain
analyst functions.(
Darktrace還使用深度學習技術

自動執行重復且耗時的任務,這個是要干嘛?沒太明白

 

Darktrace’s technology has become a vital tool for security
teams attempting to understand the scale of their network,
observe levels of activity, and detect areas of potential
weakness.


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM