See and do more with analytics integration
Download the Tech Note
The term ‘analytics’ typically refers to Video Content Analysis (VCA) – the act of automatically analysing video content to identify and assign meaning to specific events that may indicate security threats, process hazards, or details that warrant closer investigation.
"There are two main ways this functionality can be facilitated within a surveillance solution – either through software integration at the system level or through embedded camera capabilities."
The first option – sometimes known as ‘server-based content analysis’ – entails deploying analytics software (based on analytic algorithms) on a given IT server and/or using proprietary hardware from a third-party provider.
The alternative is with a camera or ‘edge-based’ deployment, whereby software is embedded within a camera’s processing unit. However, a combination of system-level analytics and onboard camera capability is also common.
Yes, open-architecture video management systems (VMS) or command and control solutions can be used to integrate, control, and manage VCA systems. The way this is achieved will vary depending on the VCA solution provider.
Some providers will only allow events to be received by the VMS or command and control solutions, i.e. notification that a specific anomaly type, such as movement in a static-classified area, has occurred. Other solution providers will enable more open, two-way communication, enabling the integration of a wider range of analytical data, metadata, and control of the exact configuration.
VCA can be used to facilitate a wide range of capabilities, including:
Virtual Tripwires/Motion Detection
Identifies when a boundary is crossed by a moving object. This is either done by using the camera’s entire field of view as the boundary, i.e. any movement detected against the fixed background scene (by frame/pixel referencing) is considered a breach, or by drawing a line virtually across a specific area to create a region of interest. Filtering capabilities can then be applied to discount any anticipated motion, e.g. birds flying or foliage movement.
Employs set background analysis to identify when an object, not normally in that background, is present and remains in place for a set time span. A common application of this is in transport settings for left or unaccompanied luggage detection. Again, filters can be applied using shape recognition.
Enables users to detect static or slow movement against a fixed background scene over a set period, and apply filters and shape classification to specifically recognise human activity, i.e. loitering in a designated area.
Often used in connection with footfall analytics (see below), heat mapping tracks activity density – the activity in question depending on the application – and overlays footage with a visual representation of these results. When used with footfall analysis, this capability enables, for example, retail establishments to see the areas of their outlet favoured by customers.
Flame and Smoke Detection
Video content is rapidly analysed for characteristics relating to fire or smoke. This may include colour variance, movement patterns, light behaviour e.g. flickering, and scene obscurity. Chemical composition analytics is another solution increasingly utilised in industrial applications, such as flare stack monitoring.
Algorithms blend motion and counting capabilities, alongside object classification, to accurately monitor the number of individuals passing through a specific scene or designated area of interest.
Facial Recognition/Face Detection
Facial recognition (sometimes known as biometric face recognition) utilises database verification to compare and contrast key facial markers to identify a specific individual. Face detection is a more common VCA application that detects the presence and number of human faces in view. This can be used in conjunction with access control where at least two people must be present to enter an area or for health and safety purposes, e.g. to identify when faces are present but no hard hats are detected.
Another common use for VCA is to support efficient traffic and road incident management. In combination with virtual tripwires and ANPR/LPR (Automatic Number Plate Recognition/Licence Plate Recognition), vehicles parking or stopping in restricted or prohibited zones can be quickly identified and logged. Similarly, rules can be programmed to identify behaviours or indicators that signify a potential accident, including swerving or the presence of smoke etc. When linked to broader city-management systems, VCA can also monitor traffic flow and trigger automated responses to support efficient operations, for example, by opening up additional lanes or updating digital signage.
It depends on the analysis application. If analytics is server-based, camera selection can be dictated purely by resolution, range, operating conditions etc., rather than specifically requiring models with onboard VCA capabilities. In fact, many server-based VCA solutions will work very well with lower video resolutions. Where precise image detail is a primary objective, HD IP or megapixel cameras may be needed.
Server-based analytics integration may not deliver the full variety of analysis required – particularly on large sites with a complex range of security and operational areas with different needs. In this case, a combination of server-based VCA, ‘normal’ cameras, and cameras with onboard analytics may be preferable (making it important to use an open-architecture surveillance solution that supports third-party integrations).
Yes, the network needs to support the additional video and data stream demands generated by analytics, but this can be factored into the setup. It makes multicasting, for example, a vital part of network configuration – particularly for large or complex sites that need to efficiently manage traffic and bandwidth capacity.
VCA-enabled cameras generally fall into one of three categories:
Basic - Most commercially available VCA-enabled cameras offer simple analytics such as motion detection and basic virtual tripwire capabilities. There are a huge number of vendors in this category.
Specialist - For very specific applications such as oil spill detection, fire/smoke/chemical composition detection, or behavioural analytics, it is better to select specialist camera vendors.
Open software - These cameras usually have some existing onboard analytics but also support a platform to host and run third-party analytics programs or engines.
Server-based analytics solutions can choose dynamically which and how many video streams to feed into analytics engines and the appropriate rules to run.
"While VCA is by far the most common form of analytics in terms of surveillance solutions, pure data analytics integration is also possible."
This can be implemented at either system level – with some command and control solutions now featuring onboard capabilities of this nature – or through integration with edge devices designed to collect specific data sets, for instance, passenger counters, process monitoring technology, or environmental sensors (e.g. air quality or chemical detection). Command and control solutions conformant with ONVIF Profile C can directly integrate with edge devices. VCA applications are typically more about identifying specific events, whereas data analytics are more commonly used where patterns need to be identified.
Where a security or immediate threat-detection requirement is present, i.e. where live situational awareness is vital, it is advisable to integrate analytics through a surveillance command and control solution that can alarm specific data to generate alerts and automated workflows for operators to follow.
"It is useful to remember that, if your surveillance command and control solution has an open architecture and features integral functionality to generate rules and manage alarms, this will negate the need for a separate alarm management platform."
However, analytics integration is not always required for security or threat-detection purposes. Information obtained can also play a significant role in optimising operational performance.
Retail outlets, for example, increasingly integrate analytics with their surveillance solution to track footfall or shopper behaviour (such as linger times at certain displays), informing decisions around store layout.
Casinos can use a command and control solution to apply tailored rules to data obtained from, e.g. facial recognition, POS systems, slots, and gaming tables, to identify a wide range of scenarios that go way beyond security – extending to player behaviours, spend patterns, and operational pinch points that could be improved.
Organisations reliant on equipment and functional infrastructure can apply such rules to integrated analytics to identify pre-fail conditions and automate predictive maintenance protocols. The range of potential benefits is extensive and will vary significantly between sectors and applications.
Having a clear understanding of what analytics is needed for, i.e. what you want to be able to do, will dictate other factors that need to be considered and implemented to maximise solution performance. Some of these factors may include:
Storage - In extreme cases (where analysis of significant volumes of live metadata is required), storage requirements may be three or four times larger than normal. For most applications, however, requirements should no more than double. Storage and integration of this information is important, particularly where post-event investigative requirements are high. For example, an airport security team may use live, alarmed VCA to identify specific threats in real time. Metadata is captured and stored, allowing retrospective searches to be conducted by data filtering to show any red vehicles loitering within a particular parking zone over a specific time period.
Lighting considerations - It is no use having state-of-the-art video analytics if the image being analyzed is compromised due to poor lighting. Examples of poor lighting may include scenes with very high contrast (i.e. with both very bright light and shadow areas), or simply night-time scenes where there is not enough additional light available to the camera. If analytics is required purely to detect movement, using thermal cameras in low-/no-light areas is a clear option.
Camera placement - The optimal placement of cameras for traditional security purposes is likely to differ significantly from cameras used for analytics. For example, a train operator may rely on headcount analytics for accurate passenger numbers, which can only really be achieved from elevated ‘top-down’ positioning, e.g. to avoid overlooking people standing behind taller passengers. Thermal or radiometric cameras may also be useful to further support this type of counting accuracy.
For organisations integrating analytics at the system level, camera placement and field of view are especially important to remember because while the analytics may not need specialist VCA-enabled cameras, they also may not be able to simply utilise cameras currently set up for security or process surveillance. It is always advisable to conduct expert site surveys to identify optimal solution design.
Business intelligence - If business intelligence is your objective (i.e. recognising trends and patterns from data sets to influence aspects such as security deployment, customer service, staffing, and even building/facility layout), then it is not enough to simply use VCA.
"You also need a command and control solution capable of unifying data strands to make appropriate connections and correlations, and ideally able to display information for clear and actionable insight in real-time and review scenarios."