IRIS+™ Architecture and Engineering

general specification for video analytics solution the proposed video analysis solution should meet all the requirements detailed in this a\&e specification document supported cameras onvif/rtsp the solution should support analysis of any onvif / rtsp video streams from fixed, angled, or overhead cameras these could be ip cameras or analog cameras via encoders ccd cameras the solution should support the analysis of video streams of optical cameras the solution should support a minimum resolution of 480p and should be able to support higher resolution streams (i e 720p / 1080p) to improve detection distance and accuracy the maximum supported resolution should be 4k the solution should support the analysis of video streams with a minimum frame rate of 8 fps thermal cameras the solution should support the analysis of video streams of thermal cameras the solution should support a minimum resolution of qvga (320 x 240 pixels) and should be able to support higher resolution streams to improve detection distance and accuracy the solution should support the analysis of video streams with a minimum frame rate of 8 fps video analysis technology the solution should be based on deep learning technology for target detection and classification target types the solution should support the automatic detection and classification of the following target types person standing, fallen, or lying on the ground two wheeled vehicle motorcycle, bicycle vehicle car, van, bus, truck object bags, suitcases, backpacks, boxes, purses smoke and fire smoke, fire the solution should be able to detect and ignore the following objects automatically clouds birds dogs/cats vegetation the solution should support detecting the existence or absence of custom object types the customer should be required to provide a relevant set of several hundred images of the desired object type, and a custom object detection model should be prepared for the customer rule based event detection & analysis capabilities analytics rules the solution should offer a suite of analytic rules to provide real time detection of the following behaviors target/s moving in an area/loitering target is moving in the region of interest for a user defined duration target/s crossing a line – the target has crossed a user defined line in a specific direction or any direction stopped vehicle – the target has stopped in the region of interest for a user defined duration vehicle speeding – the target vehicle crosses a line at a higher speed than defined grouping – detection of a dense group of people (a configurable number) in the region of interest, detected for a user defined duration crowd density – the solution can handle counting very dense crowds, up to 5000 people in the field of view of one camera object left behind – detection of suitcase/bag/backpack/purse left behind in the region of interest for a user defined duration asset protection – mark an object in the field of view and receive an alert when that object is removed traffic counter flow – a vehicle is traveling in the opposite direction slip & fall detection – a person is falling on the ground and detected lying on the floor face recognition a person in a scene is detected with the same attributes as an individual from a watchlist each detection rule should apply to the relevant target types the user should be able to select several relevant target types for each detection rule the solution should be able to detect the existence or disappearance of custom objects in/from a user defined region of interest rule configuration and setup the solution should provide the ability to execute bulk operations for activating, deactivating, and scheduling multiple analytics rules the solution should enable any combination of analytics rules to run on the same camera simultaneously, without limitations the solution should enable the operator to define multiple detection regions per camera scene calibration each camera should support automatic and manual calibration calibration refers to the translation from pixels in the image to actual size (meters/feet) in different parts of the image the system should automatically calibrate object sizes in the image based on standard sizes of dnn classified targets in the scene, over time the system should have the possibility to override the automatic calibration or calibrate scenes where no movement occurs manual calibration should support different pixel to meter translations for different parts of the image, creating a flexible calibration mesh across the image frame all calibration methods should support complex translations creating accurate translations also in challenging environments resulting from e g fish eye/distorted camera images and scenes with multiple levels event generation the solution should provide real time generation of events to alert operators when a behavior is detected that matches the user defined rule the solution should support simultaneous tracking of multiple targets within the detection regions and/or the cross lines the solution should generate a short event video clip for each detected event, showing several seconds before and after the event, and include a bounding box around the target that triggered the event event integration the solution should be able to send the events to the following external systems milestone xprotect vms genetec security center vms immix cs and immix gf sentinel patriot mobotix mxhub and mxmanagementcenter other systems based on webhooks protocol (http push) other systems based on smtp protocol anomaly detection the system should be able to continuously ‘learn’ the typical behavior of the scene the system should then be able to automatically detect abnormal behaviors of detected targets and generate anomaly events in real time anomaly detection configuration the user should be able to activate/de activate anomaly detection for a single or multiple camera/s the user should be able to change the default anomaly detection configuration parameters for a single or multiple camera analyzed target types multiple selections of relevant target types anomaly detection sensitivity measured by the average number of anomaly events to be generated daily/weekly anomaly detection and event generation the system should analyze the behavior of a target based on the following parameters the target’s existence in the camera’s field of view the target’s path the target’s size the target’s speed the system should analyze the relations of several targets in the scene based on the following parameters the number of targets in the scene and their location in the camera’s field of view the distance between the targets in the scene the system should analyze the object’s normal behaviors per different weekdays and hours of the day the system should generate an anomaly event including an event clip with several seconds before and after the event date and time, bounding boxes around the relevant targets, and a description of the detected anomaly anomaly detection training and user feedback the system should require an initial learning (‘training’) period of a few weeks the system should enable the user to fine tune the anomaly events by letting the user tag the anomaly events as relevant or irrelevant irrelevant events should feed the ongoing training process and should subsequently be ignored the system should re learn the scene’s normal behavior regularly anonymization the solution should support static and dynamic video anonymization functionality the functionality should be individually configurable per video stream different anonymization methods should be supported standard pixelation application to grayscale images and streams standard pixelation application to color images and streams pixelation implementations each frame in the video data is permanently, destructively, and masked the original video data should not be possible to recover video investigation method of operation the system should analyze all cameras in real time and create metadata that will be stored in a database it should be possible to search any camera with a delay of no more than 10 seconds in real time the system hardware specification will facilitate the real time processing of all cameras it should be possible to search any or all cameras in the installation simultaneously and without needing to process the cameras in small batches, regardless of the number of cameras installed in the system defining search criteria the solution should not require the operator to apply any rule or behavior configuration in advance as a pre requisite for performing video investigation the video investigation should be conducted simultaneously over single or multiple selected cameras, either from a list or map the solution should offer to search for the following suite of behaviors of the supported target types person / two wheeled vehicle (motorcycle/bicycle) / vehicle (car, van, bus, truck) moving for a specified time, in the entire field of view (fov) or in a specified area of interest (aoi) person / two wheeled vehicle (motorcycle/bicycle) / vehicle (car, van, bus, truck) crossing a line in a specific direction or in either direction people grouping for a specified time, in the entire fov, or in a specified aoi (based on a user defined number of people threshold) persons occupying for a specified time the entire fov or a specified aoi (based on a user defined occupancy threshold) two wheeled vehicle (motorcycle/bicycle) / vehicle (car, van, bus, truck that stopped for a specified time, in the entire fov or a specified aoi bags/backpacks/suitcases that were added for a specified time, to the entire fov or a specified aoi persons that exhibit similar attributes to a search for similar image the solution should allow filtering of search results based on target color characteristics for people, the solution should allow specifying upper body color and lower body color the solution should be capable of searching over various time range options over the past n minutes, hours, or days (e g , over the past 3 hours; past 7 days) from a start date and time to an end date and time over a recurring time interval across a date interval (e g , between 8 9 a m , every day between jan 1 10) the solution should provide the capability to search for similar targets if a target is found, another search can be performed in the recorded video (generated from the same camera or any group of cameras) to find targets that are the same as or similar to the found target viewing search results the solution should be able to display video recording for any search result around the time that the search target/behavior was found, without requiring integration into a 3rd party recording solution the solution should continuously display a bounding box over the target (target tracking) the solution should enable the user to pause and re play the video playback the solution should enable the user to use the progress bar to navigate to any time position along the playback segment the solution should provide multiple options for viewing search results event thumbnails after searching for the relevant results, the solution should be capable of displaying a thumbnail for each result, each of which shows the detected target it should be possible to play back the video of each thumbnail target path / location after searching, the solution should be capable of displaying all motion paths in a scene over the field of view reference image motion paths should be colorized based on the primary object class (human, vehicle, two wheeled vehicle) and filterable based on object class heatmap after searching, the solution should be capable of overlaying a grid where each cell is colorized depending on the amount of activity in that part of the image it should be possible to filter the heatmap overlay depending on the object class original video after searching, the solution should be capable of showing the original video in an integrated video player process and investigation capabilities the solution should provide the following process and investigation capabilities save search query users should be able to save a search query with a given name for later reuse save search results users should be able to save search results with snapshots of the detections and the results’ identifying information (camera id, time) export clip the solution should enable users to export a search result to a video file, for a single result as well as for a complete video summary the exported clip will include the target tracking display personalized ai the solution should offer advanced object classification found within images of interest selected by the user the object classifications selected by the user can be categorized into the following parent level categories people vehicle two wheeled the statistics the solution should offer analytics rules to provide statistical analysis capabilities for multiple target types including but not limited to count the number of targets moving directionally, i e , crossing a virtual line/s that is operator defined in the camera’s field of view the solution should optimize the precise counting of targets by distinguishing individual targets in a cluster if a cluster of 4 people crosses a line (for example), a count of 4 will occur rather than 1 calculate the average speed of vehicles crossing a line the solution should offer statistics of events generated in the system, over time aggregation of events over cameras and hours/days should be supported the solution should offer statistics of health alerts generated in the system, over time aggregation of events over cameras/devices/sources and hours/days should be supported the statistical data should be available using apis note specifically crowd statistics and crowd density share the same algo but can be applied in both manners per sensor geospatial awareness geospatial mapping the solution should enable the user to configure the following geospatial data per video source connected to the solution the video source’s location on a map the video source’s fov (field of view) registration – correlation of points in the fov with a map alternatively, the solution should retrieve the geospatial data from 3rd party systems geospatial analysis the solution should be able to present real time events or video investigation search results over a map the solution should allow a map based selection of relevant cameras within a user defined zone, for video investigation the solution should be able to present a tracked target path over a map system health monitoring the solution should self monitor its main components to ensure high availability and reliable video analysis this monitoring should include the following aspects ability to properly pull the onvif/rtsp video stream minimal video stream frame rate and resolution scene lighting (too dark/saturated / blocked) event delivery status monitoring of analytics servers (the computer running the video analytics) the solution should support configurable thresholds for detecting and generating health alerts the solution should be able to send health alerts to a configurable email recipients list system architecture the solution should be based on the following main components analytics server(s) the analytics server(s) pulls video streams from cameras and performs initial video analysis the analytics server(s) communicates with the core server(s) the analytics server(s) should support low bandwidth connection to the core services, down to 5 kbps per camera the analytics server(s) should be able to scale to support any number of cameras and should support both scale up and scale out resource scaling to increase performance a gpu should not be required the analytics server should support both cloud based, on premise, and hybrid deployments on premise installations without internet access (“fully offline”) should be supported real time event integrations to third party systems from the analytics server should be supported core server(s) the core server(s) should provide central management for all video analysis, rule configuration, and setup the core server(s) should support both cloud based and on premise deployments on premise installations without internet access (“fully offline”) should be supported the core server(s) should be able to scale to support any number of cameras and should support both scale up and scale out resource scaling to increase performance the core server(s) should support high availability deployments multi tenant – the solution should support an unlimited number of separate accounts that should be completely isolated from each other an unlimited number of users – each account in the system should allow setting up an unlimited number of users role based authentication – each user in the system should be assigned a role with specific access permissions to different modules of the solution real time event integrations to third party systems from the core server(s) real time health alert integration to third party systems should be supported public api the system should expose a public, documented, api the api should require authentication and encrypted communication the api should support all functionality in the system, including management of cameras and analytics, management of events and health alerts, monitoring of system health and real time events, and the ability to create and get results from a video investigation