Vision sensors, which can detect renderable entities(Renderable objects are objects that can be seen or detected by vision sensors), should be used over proximity sensors mainly when color, light or structure plays a role in the detection process. However, depending on the graphic card the application is running on, or on the complexity of the scene objects, vision sensors might be a little bit slower than proximity sensors. Following illustrates applications using vision sensors:
[(1) industrial robot observed by 2 vision sensors, (2) Line tracer vehicle equipped with 3 vision sensors]
視覺傳感器與攝像機都能顯示場景中的圖像但是也存在着區別（一個側重視覺檢測和處理，一個側重場景顯示）：
- A vision sensor has a fixed resolution. A camera has no specific resolution (i.e. it adjusts automatically to the view size).
- A vision sensor's image content can be accessed via the API, and image processing filters are available. A camera's image content is not directly available via the API (but via a callback mechanism), and image processing not directly supported.
- A vision sensor generally requires more CPU time and operates slower than cameras.
- A vision sensor can only display renderable objects. A camera can display all object types.（只有設置了Renderable屬性的物體才能被視覺傳感器檢測處理）
- Vision sensors can only operate while a simulation is running; this means that a vision sensor's image content is only visible during simulation.
視覺傳感器可分為正交投影型和透視投影型，它們的視場形狀不一樣：
[Orthogonal projection-type and perspective projection-type vision sensors]
視覺傳感器有近端剪切平面（near clipping plane）和遠端剪切平面，使用剪切平面可以排除場景的一些幾何體，只查看或渲染場景的某些部分。比近端剪切平面近或比遠端剪切平面遠的對象是不可視的。可以通過傳感器屬性對話框中的"Near / far clipping plane"設置剪切平面的位置。
透視模式下傳感器的視場角（FOV）可以通過"Perspective angle [deg] / Orthographic size"來設置。Perspective angle: the maximum opening angle of the detection volume when the sensor is in perspective mode. 如下圖所示設置視場角為60°，當X/Y分辨率一樣時水平視場角和垂直視場角的大小相同。
正交模式下傳感器的視場大小可以通過"Perspective angle [deg] / Orthographic size"來設置。Orthographic size: the maximum size (along x or y) of the detection volume when the sensor is not in perspective mode. 設置為Orthographic size為1m，X/Y方向分辨率為64/32，則X方向視場為1m，Y方向為0.5m，如下圖所示：
- Vision sensor filter composition
使用視覺傳感器的目的就是進行圖像檢測與處理，VREP中的視覺傳感器在仿真過程中可以產生兩種數據流：彩色圖像（color image ）和深度圖（depth map）。我們可以通過API函數獲取數據，然后遍歷圖像的每個像素進行處理，這樣做靈活性很大，但是使用起來比較麻煩而且處理速度不夠快。VREP提供了一種內部的filter來對圖像進行處理（It is much more convenient (and fast!) to use the built-in filtering and triggering capabilities）。最簡單的圖像處理流程由3部分組成：輸入→濾波→輸出：
[Vision sensor filter with 3 components]
在Image processing and triggering對話框中可以添加30多種filter對圖像進行快速處理，比如：
- Selective color on work image：根據RGB/HSL值和公差選取圖中指定顏色，進行保留或移除等操作
- Rotate work image：對圖像進行旋轉
- Resize work image：對圖像進行縮放
- Flip work image horizontally/vertically：對圖像進行水平/豎直翻轉
- Edge detection on work image：對圖像進行邊緣檢測
- Sharpen work image：圖像銳化
- Binary work image and trigger：對圖像進行二值化處理
- 3×3 / 5×5 filter on work image：使用3×3或5×5的模板對圖像進行濾波
下面以均值濾波為例進行說明，3×3矩陣中各個分量設為1/9，則濾波器將會對原始圖像每個像素周圍的9個像素點取平均，對圖像進行平滑，減小噪聲：
復雜的圖像處理流程可由多個部分組成，處理環節能完成4種基本的操作：
- Transfer data from one buffer to another (e.g. transfer input image to work image)——傳輸數據
- Perform operations on one or more buffers (e.g. invert work image) ——對數據進行操作
- Activate a trigger (e.g. if average image intensity > 0.3 then activate trigger)——激活觸發
- Return specific values that can be accessed through an API call (e.g. return the position of the center of mass of a binary image)——返回特定值
下圖顯示了圖像處理流程中的各種緩存和相互之間的操作：
[Vision sensor buffers and operations between buffers]
The input image and input depth image are volatile buffers (易變緩存 i.e. normally automatically overwritten with new data at each simulation pass)；The work image, buffer1 image and buffer2 image are persistent buffers (i.e. their content is not modified unless a component operates on them)
下面看一個比之前復雜點的例子，將原始圖像邊緣提取后旋轉90°再疊加到原始圖像上進行輸出：先將要進行操作的work image保存到buffer 1中，然后對work image進行圖像處理操作，接着將buffer 1疊加到work image上，最后將合成的圖像進行輸出。
參考：