The Internet of Things and the human population have two things in common: we’re talking billions, and we’re looking at growth. According to the latest research, there are between 8.74-11.7 billion IoT devices globally, surpassing the human population. By 2025, it’s estimated that there will be 41 billion IoT devices. With a faster rate of growth, there will be far more IoT devices than humans soon.
At Palo Alto Networks, unsurprisingly, we have a big and ever-growing number of IoTs – about 35% of all devices on our network. This includes video conferencing systems, printers, cameras, HVAC systems, PDUs, networking devices, etc., across our global sites. Let me walk you through our journey of deploying IoT Security so that you see what a world of difference clarity and visibility can make.
Got a Situation
First, a heads up, at Palo Alto Networks we call our IoT security product – you guessed it – IoT Security (formerly known as Zingbox, which we acquired). Previously we didn’t have end-to-end capabilities to manage our IoT Security. Instead, IoT devices were hidden in our vast networks. We needed to overcome a set of security challenges:
- Lack of visibility: We didn’t have a view of all IoT devices. We also lacked a system tracking IoT device risk in a comprehensive manner based on
- IoT device context – static context such as OS, firmware, patching levels, and dynamic context such as IoT device behavior anomalies, abnormal communication patterns.
- Vulnerabilities – CVEs, default passwords.
- Threats – related to malware & exploits.
- Inability to secure IoT devices: Most IoT devices (video cameras, printers, PDUs, HVACs) cannot be secured with endpoint software. Additionally, if IoT devices are on the same network segment (VLANs) as other devices, e.g., employee network or guest network, then that’s a huge headache as well.
- Lack of runtime security rules governing IoT behavior based on normal function of IoT devices: It is powerful to be able to observe IoTs for a period of time and recommend firewall security rules for IoTs based on their make/model and normal runtime behavior.
- Lack of a single security platform for managing the lifecycle of all IoTs: Such as IT IoTs (video conference systems, zoom rooms, servers, networking equipment) and OTs (PDUs, HVACs, Workplaces related IoTs, building control systems).
- Lack of well-defined organizational roles in managing IoT lifecycle: While IoT business owners can onboard IoT devices in any organization, we weren’t clear on the role of multiple stakeholders throughout the IoT lifecycle model.
- Who onboards IoTs on the network?
- Who fixes security incidents?
- Who does the initial risk assessment of IoTs?
- Who manages IoT configurations, software updates, software vulnerability fixes and patching to IoTs?
IoT Device Visibility
We started seeing approximately 4000 IoT devices in a matter of days without having to put any new hardware in our network, thanks to IoT Security consuming our firewall logs. Our IoT Security considers everything as IoT devices except for laptops and mobile phones (user driven devices).
As figure 1 below shows, we can see the following:
- 3754 total IoT devices, including multiple categories of IoT devices – consumer IoT devices, facilities IoT devices, generic IoT devices, industrial IoT devices, network devices, office IoT devices, traditional IT IoT devices
- Overall risk score – for 38/100 for all IoT devices combined. Additionally we can track the risk score of individual IoT devices.
- 519 Protocols used by IoT devices.
- 2600+ IoT alerts and 40+ vulnerabilities over a period of 1 year.
We were able to tag IoT devices based on profile, category, vendor and model by leveraging “User_tags” [Profile, Category, Vendor, Model]. This allows us to map multiple IoT device families to IoT business owners. As we’ll see, mapping IoTs to owners is a concrete step in defining and automating a robust RACI model.
Figure 2 below shows IoTs related to User Tag “IT_audio_video”. Other examples of IoT user tags for our environment are :ip_telephony, It_facilities, Printer, etc. The third leftmost column shows the risk score for each IoT, which is tracked by our IoT Security. This means we can raise vulnerability incidents that can be mapped to IoT owners.
We had visibility of all IT IoT devices and OT IoT devices using a single platform. We can see several types of IT and OT IoTs:
Full Security Lifecycle
Being able to track IoT risk score for all IoTs is great in securing IoT devices. If an IoT’s risk score goes above a certain threshold, the IoT Security service issues security or vulnerability alerts, which can be tracked and resolved by SOC, InfoSec and/or IoT business owners.
Known IoT threats are avoided by leveraging existing firewall subscriptions such as wildfire, URL filtering, DNS Security and threat prevention. Unknown threats are detected by the backend machine learning of the IoT Security service, which can determine when an IoT starts behaving abnormally.
A Simplified RACI Model
When we talked to various stakeholders across the company, we quickly realized that to manage IoT Security effectively, we needed to build a simplified RACI model, as shown below in figure 5. The new RACI model made a huge difference and allowed teams to have role-based ownership instead of tripping on each other.
IoT Security Architecture
All IoTs, wired or wireless, connect to our firewalls. All our firewalls send logs to Cortex Data Lake. IoT Security is able to see these logs (DHCP logs, enhanced application logs, traffic logs) and track IoT risk, generating security and vulnerability alerts. The ability of IoT Security to look at Data Lake logs is powerful. If we add a new IT site, say because of expansion or M&A, we can start seeing IoTs right away.
XSOAR pulls IoT alerts from IoT Security. Some alerts are converted to XSOAR incidents and ServiceNow (SNOW) incidents which are resolved by different stakeholders (SOC and IoT business owners, as highlighted in step 3 of figure 6).
Automation Workflows
We built an IoT security automation pack using XSOAR and leveraged XSOAR’s out-of-box SNOW integration to generate security and vulnerability incidents as shown in figure 7.
About 30% of IoT security alerts are converted into XSOAR incidents for SOC to investigate. Another 40% alerts are consumed by SOC for threat hunting. This allows our SOC team to pick alerts of interest and ignore unnecessary ones – a useful strategy for managing alert fatigue. On average we get 15 IoT security incidents a month.
Confirmed vulnerability alerts from IoT security are promoted in XSOAR to IoT vulnerability incidents and are sent to ServiceNow where IoT business owners can resolve them by patching or configuring IoTs (e.g., disabling SNMP v1) based on recommendations from IoT Security. So far, we see 15+ IoT vulnerability incidents a month. Note that an IoT vulnerability incident can map to multiple IoT devices (e.g., all IP phones).
XSOAR Automation Pack for IoT Security
The IoT XSOAR Automation pack built by our First Customer team in IT is used to implement IoT Security incident management and is available on XSOAR marketplace for anyone interested in deploying & automating IoT Security. This should give customers a good 2-3 months worth of boost in 2 weeks. Customers can also build additional XSOAR automation workflows based on their own needs.
More on the Horizon
Our IoT Security is where it is today because we were able to get quick visibility and ability to track our IoT risk. We built XSOAR automation workflows to implement our simplified RACI model, and we leveraged a scalable architecture based on Cortex Data Lake so that any new IoTs in a new site or existing site can be discovered immediately. Next up we’ll be working on:
- Device-ID and policy recommendation: This is where IoT Security and PANOS come together. We want to use IoT security firewall policies based on machine learning driven baselining of normal behavior of IoT devices. IoT Security can recommend and push these policies to Panorama, where a firewall admin can review and commit these policies. IoT Security provides policy recommendations for runtime security rules governing IoT behavior based on normal function of IoT devices. This ensures a security posture based on normal IoT device behavior.
- IoT Security integration: With SNMP for managing IoT segmentation, Wireless Controllers and Rapid7.
- New XSOAR workflows for:
- Enforcing IoT VLAN segmentation
- Auto-tagging new IoTs based on user tagging
Stay tuned!
