Best GPU / 3D graphics card
As GPUs (or graphics cards) evolve quickly, it can be challenging for a Fortem partner or customer to pick the right model within their budget. This is our CTO’s personal recommendation as of April 2017.
Any NVIDIA Geforce GTX 9×0 (e.g. 970) or 10×0 (e.g. 1070) series will work great as long as it has at least 3GB video memory (VRAM). While they are primarily designed for video games, they work great with our software. The GTX 1070 and 1080 offer the best bang-for-the-buck and should offer good performance for years to come. I have a GTX 1070 at home, and I’m very pleased with it. I’m experimenting with Deep Learning during my spare time and the extra VRAM (8GB+) makes a huge difference there.
As for manufacturers, EVGA, MSI, ASUS are all good brands.
We cannot recommend the NVIDIA Quadro cards. These cards are designed for Workstation applications that use old-style OpenGL. Our software is optimized to act like a video game, so Quadro cards are overkill and on rare occasions, cause trouble. They are very pricy, they tend to be one generation behind their consumer cards, and they have subtle differences in their OpenGL drivers that sometimes cause problems for us. 3 years back, one of our customers was pressured into buying a Quadro card for $1500 instead of the $250K GeForce card we had recommended. They ran into a OpenGL driver issue we couldn’t reproduce in our office. In the end, they had to buy an extra Quadro card, ship it to us, and we had to spend 3 days diagnosing and working around the issue. Please don’t make the same mistake.
We also cannot recommend AMD GPUs and Intel Integrated graphics at this stage. While these cards and drivers generally work with our software, we don’t test on them as much, and these companies haven’t made as big an investment as NVIDIA in new technologies like Deep Learning, that we foresee having a big impact on our future products.
I hope this helps, don’t hesitate if you’d like us to double-check the card you plan to pick.
Accelerating adoption of Augmented Reality in the Enterprise
This post expands on Augmented Reality for Physical Security and explores how to accelerate the adoption of Augmented Reality (AR), in the security industry and beyond.
Seamless integration of real-life devices in AR – a killer app
It will take a couple of years before AR headsets become affordable by the mass market. Some early adopters will buy AR headsets simply to play games or for 3D design. But large companies and mission-critical government agencies are the most able to afford the first wave of headsets, and they will likely have one common objective: to enhance their productivity by integrating their existing stuff in AR: security systems, video cameras, GPS, industrial automation, etc. Think of it as the intersection of two buzzwords: IoT (Internet of Things) and Augmented Reality (AR).
Whipping out a quick prototype integrating a few devices in AR will not be too complicated, so we can expect a lot of companies to dabble in AR, starting from scratch. But Enterprise customers demand proven, advanced features that require years of development and millions of dollars to develop, for instance:
- scalability to tens of thousands of devices, lots of concurrent users, multiple sites
- figure out the best abstractions for heterogeneous third-party systems – each with their own limitations, performance tradeoffs, etc.
- minimize latency, especially for video, Pan-Tilt Zoom (PTZ) control, positioning information, etc.
- automatic fail-over with no loss of critical data
- intuitive user interface that can readily adapt to each user’s position and orientation
- information security, access control, auditability, etc.
Leveraging a proven code base and expertise like Fortem’s could bring considerable competitive advantage, save time and money.
Adapting to a dynamic 3D environment is a big challenge (and opportunity)
AR will be disruptive with high risks, high rewards. Some companies will disappear, others will reach new heights as they become an essential part of the daily life of their customers who are no longer tied to a desk or limited by the small screens of their phones.
To unlock AR’s potential will require sizable investment and vision from today’s large enterprise software vendors. Most typical enterprise software consider maps a minor feature, e.g. they simply slap a component like Google Maps within their old user interface. Their maps are static, i.e. change rarely, maybe once a month. They expect static objects (i.e. that stay in place), so they can get away with manually hard-coding the relationship between objects, e.g. list of cameras for specific alarms.
AR brings with it huge complexity: extremely detailed, dynamic 3D maps that may change within seconds, e.g. whenever a chair is moved. Users and their headsets change position and orientation 60 times a second. They capture 3D data in real-time. Relatively simple features that could easily be implemented by an average software developer in a few days suddenly require understanding advanced 3D math and GPUs, months of careful design and implementation, etc.
To ride this wave and increase their market share, enterprise software vendors will need to get out of their comfort zone. They must train, hire or acqui-hire people with a rare combination of 3D and domain expertise, people who can efficiently re-design and re-implement critical parts of their software, and create new complementary products. It could take several iterations and a couple of years to make the transition and finally unlock real value for their customers.
Fortem’s existing software was designed from the very beginning for a dynamic 3D world and devices, by 3D experts with a passion and track-record for mission-critical software. Fortem already uses AR, albeit in a limited fashion, e.g. overlaying 3D information over video from surveillance cameras. Its patented algorithms can already accommodate the real-time movement of users and cameras. While Fortem’s technology will need additional investment to fully leverage AR headsets, that investment and technical risk will be modest compared to starting from scratch.
AR will require new user interfaces adapted to its strengths and limitations
Many large enterprise vendors will initially discount AR’s importance and wait out until most of the early AR limitations are addressed, before trying to catch-up at great cost… like Blackberry did for touch screens.
Instead of ignoring the upcoming AR revolution until it’s late, better to make careful investments in understanding and working-around the early AR limitations. This will require innovative thinking. For instance, if you need to wave your hands in front of your face every time you want to bring up relevant information or control an object, you’ll get physically tired and frustrated within minutes. AR vendors will of course provide programming toolkits for common operations like overlaying simple information for an object directly in front of you. But what about nearby objects of interest, that aren’t right in front of you?
Fortem’s algorithms can provide the next step: immediately overlay videos from the optimal cameras surrounding the user, e.g. from CCTV or users’ headsets. Automatically redirect and zoom PTZ cameras toward a point of interest, etc. We came up with new ways to address long-standing issues of the security industry, and we’re eager to take on new challenges.
Augmented Reality for Physical Security
Why Augmented Reality (AR) will transform Physical Security
Anticipated for decades by science-fiction movies, this technology is finally becoming useful and available for early adopters in 2016. In the next 3-5 years, AR will become widely available and affordable for consumers and enterprises alike. AR has the potential to change many aspects of our lives, including Physical Security. AR will grant personnel instant access to critical information, allow instant and intuitive control over equipment, and ultimately allow organizations to save money by eliminating costly infrastructure such as video walls.
Along the way there will be a lot of hype. I’ll conclude with some of the most important limitations expected of the first batch of products coming to market in 2016.
What is Augmented Reality?
Augmented Reality devices are headsets or tablets that superimpose a computer-generated image on someone’s view of the real world. The best way to understand it is to look at the Microsoft Hololens video below.
Example 1: Microsoft Hololens
Example 2: Google Project Tango
Anticipated Benefits for Physical Security
#1: Project and interact with any information, anywhere
Instead of looking at information through monitor, you’ll be able to walk around and look and interact with information attached to any object or surface. By waiving your hands in front of your face, you’ll be able to bring up more information and control equipment.
A Security Director getting an emergency call in the middle of the night will no longer need to drive to his work place to manage a crisis. Instead, he will simply put on his glasses, go to his kitchen, and see the information projected on his table, on his walls and ceiling.
It will feel as if you had tablets or giant touch screens floating anywhere you want. For instance, you’ll be able to overlay access control and video over any door in your facility, to instantly see who most recently accessed, bring up most relevant videos, mark a card stolen, etc.
Ultimately, this will make video walls obsolete and greatly reduce the need for workstations. Unlike a video wall, that shows the same information to everyone in the room, each person will be able to see its own augmented reality, based on what’s most relevant and actionable for that person at that moment. Classified information will literally become “for your eyes only”.
#2: X-Ray Vision and gaze tracking
AR can also give the impression to see through walls. For instance, facility managers and security integrators will be able to see network cables, pipes and other equipment hidden behind a wall or under the floor. You will be able to bring up live and archived video from nearby rooms. In an airport or port, you’ll be able to stare a moment at a distant target to automatically bring up zoomed video(s) focused on the target, through an integration with PTZ cameras on the ground and on UAVs.
Tracking the gaze of a person will enable smarter algorithms to pinpoint exactly what information you’re interested in, right now.
#3: Precise indoor positioning
One underlying technology that makes AR possible is extremely accurate positioning, that tracks the position and orientation of an AR device and it user, within the environment. Think of it as ultra-precise GPS that works indoor. This information, by itself, will be extremely helpful in increasing productivity, responding to incidents faster, etc.
First, this will eliminate a lot of radio chatter, normally spent informing dispatchers of one’s position. It will enable more precise and faster dispatching based on who is available, where.
Precise indoor positioning will lead to increased security, e.g. by automatically assessing some alarms. It will become possible for security systems to match motion alarms with nearby resources, to only alert security operators when there is motion while no guard is around.
Eventually, an entire team’s movements could be optimized in real-time, by sending visual instructions to each person within their AR view of the world.
#4: Capture of 3D video, 3D maps and measurements
Another underlying technology of AR is the capability to capture, in real-time, the 3D (depth) information in front of the AR device.
This can improve collaboration, by allowing team members to instantly see through each other’s “eyes”.
This depth information will allow multiple viewpoints to be precisely combined in 3D, and looked at from different angles, e.g. from a bird’s eye view.
The depth information can also make some types of video analytics (e.g. head counting) much more accurate, simpler to implement, and less prone to false alarms. 3D capture will become a feature of security cameras.
By simply walking around a facility, anyone will be able to create or update a precise 3D map of the environment. 3D maps can increase situational awareness, and combined with advanced algorithms, can help automate or accelerate decision making.
Vendors
Microsoft and Google are not the only industry giants betting big on augmented reality:
Magic Leap
Magic Leap, a Florida start-up, is the strongest contender to Microsoft Hololens for augmented reality headset. They raised over $500M from Google Ventures, Qualcomm and others.
Apple
Apple hasn’t announced anything yet, but their acquisitions of RealSense ($345M in 2013) and MetaIO (2015) make it obvious that they are working on an Augmented Reality headset and/or phones.
Intel and partner Daqri
Intel and its partner Daqri showed off a high-tech hardhat that packs in augmented reality features to increase workplace safety.
Others
Vendors currently focused on VR (Virtual Reality), such as Oculus (owned by Facebook) and Samsung, are also likely to announce Augmented Reality devices in 2016-2017.
This industry interest should come as no surprise as Augmented Reality has a real potential to completely displace smart phones as the main computing interface over the next 5-10 years.
Short-term Limitations
The first versions of devices coming out in 2016 and early 2017 will likely be very exciting for early adopters and developers, but of limited practical use due to limitations such as:
- Battery life – most likely limited to 2-3 hours of active use, less than a day of standby.
- Field of view – the image will appear as a small window in the middle of your field of view, limited to 30 degrees. Until it’s addressed in years to come, this will severely limit the feeling of immersion.
- Limited range and requires 3D mapping – The Google Tango devices have been demonstrated successfully in a large museum but the environment must be mapped and kept up-to-date. This will likely take significant effort to get started.
- Indoor use only – Some of the features rely on infrared lasers that don’t work reliably if exposed to direct sunlight. Most of the first devices will be expensive and most likely not waterproof or shock-proof.
- Interference with each other – some AR devices will not support having multiple users using them in the same spot at once.
- Heavy and bulky – they will most likely be uncomfortable for more than an hour at a time
- Limited software – software will need to be optimized to unlock most of the potential. This will take years and significant financial investment, especially for large security manufacturers that don’t have the 3D expertise and prefer to wait until a critical mass of devices and users are on the market.
- Device cost – the first Microsoft Hololens (development kit) cost $2,500.
All of these limitations will be overcome in due time. Take the price, for example. $2,500 may sound like a lot, but keep in mind that you’re basically talking about the first generation of a very light, high-resolution, super-fast computer equipped with advanced sensors, that you’re putting on your face. Moore’s law isn’t what it was, but there’s no physical or business reason why the price wouldn’t come down to $150 within ten years, just like it did for smart phones.
TSA failed to detect 67 out of 70 weapons, explosives
ABC News reported yesterday that TSA airport screeners at dozens of U.S. airports failed to detect banned weapons, mock explosives and bomb vests, in 67 out of 70 tests. The top TSA official in charge has been replaced. Details will soon be available from the DHS OIG. Homeland Security News Wire covers the story.
Experienced security managers remind me constantly that “what you don’t measure, you can’t improve”, and its corollary, “if you never test your performance, you’re failing and you don’t even know it.” Shockingly, even the most security-conscious US agencies often fail to apply these simple lessons until their own security incidents make the headlines.
This incident should remind security managers that:
- Your weakest link is usually human complacency, especially amongst your lowest paid employees, therefore you must put in place a process that self-corrects and self-improves, through regular testing, checks-and-balances, redundancy, incentives (financial and otherwise).
- Reserve enough budget to conduct security penetration tests, held by external security consultants experienced in your industry. Resist pressure to go for the cheapest provider – your organization’s reputation is at stake and your job may be on the line. You don’t want to make the headlines… at least, not until you can demonstrate a quantified improvement in performance and justify your next pay raise!
- You need advance warning when problems are about to arise, e.g. response time is degrading for a specific facility, person. You must collect enough of the right metrics to pinpoint outstanding performance (e.g. to reward employees that work well or discovered new ways to improve productivity) and to correct bad performance (e.g. train or replace some employees or managers). Automation is key here. You must assume that any manual step will eventually be forgotten or done incorrectly, unless you constantly verify compliance.A good security management software can help streamline performance measurement and automatically verify that no member of your team drops the ball.
The media and general public will complain loudly about these new TSA failures, and rightly so. But the TSA has taken the first step to make things right. They are confronting reality. They finally performed these tests and that are going to share some of the results publicly. Many other agencies don’t provide sufficient security transparency or have ineffective security regulations that don’t test performance, or worse, test vanity metrics.
I’m an optimist, so I bet that 2 years from now we’ll start to hear positive stories. The best airports will proudly announce their near-perfect screening score. The DHS, TSA and other aviation authorities/regulators should take this opportunity to quantify the gains achieved through their security funding, to reward airports that spent our taxpayers’ money wisely, and to pressure airports that perform poorly to replace incompetent employees and force them to improve procedures.