By Bill Palifka
The deadline is looming for medium-size utilities to begin risk and resiliency assessments, and those in the small category an encouraged to budget and begin work by the end of the year. Water and wastewater utilities provide critical lifeline services to their communities and their regions. Safe water and clean water are essential for public health, ecosystem protection and economic strength.
IT and OT compromises can have great impact on a utility. Operational disruptions could jeopardize public health and environmental protection. Supporting these important functions requires secure information technology (IT) and operational technology (OT)
To support water and wastewater utilities and the wider critical infrastructure community in its cybersecurity goals, below are 3 quick actions to reduce cyber risk for our water and wastewater utilities.
1. Perform Asset Inventories
Since you cannot protect or secure what you do not know you have, identifying assets is the foundation of a cybersecurity risk management strategy and essential for prioritizing cyber defense. While the value of asset inventory usually goes unchallenged, too few organizations do it effectively, if at all. ICS network defenders need to understand which assets are on their networks and what information those assets provide.
There are multiple methods for discovering assets. The best approach will likely include multiple methods. The SANS ICS Security Blog post, “Know Thyself Better than the Adversary – ICS Asset Identification and Tracking,” discusses four approaches to asset identification: physical inspection, passive scanning, active scanning, and configuration analysis.
Asset Inventory Database
An accurate and comprehensive asset inventory is much more than a list of devices. Data, processes, personnel and supporting infrastructure and dependencies to other systems should also be identified. An asset repository should include all components on the IT and OT networks and in the field, including third party and legacy equipment. The inventory record should be granular enough for appropriate tracking and reporting. Details should include but not be limited to asset owner, location, vendor, device type, model number, device name, hardware/firmware/software versions, patch levels, device configurations, active services, protocols, network addresses, asset value and criticality. Furthermore, an asset inventory is not a singular task, but an ongoing process. One approach to keeping the asset inventory current is to incorporate it into change management processes.
Performing an inventory will help reveal blind spots by identifying things that do not belong, such as a rogue wireless access point or other unapproved devices or connections. Inventories also illuminate processes and procedures that could enable the detection of unauthorized configuration changes or other anomalies within the environment.
An asset inventory would be incomplete without physical inspection. Network scanning methods reveal what is connected to the network at the time of the scan but may not readily account for disconnected devices that could be connected later, such as rogue or wireless devices. Additionally, a network diagram showing the relative physical locations and roles of the assets is essential for thoroughly documenting the system.
Not only is the asset inventory a foundation for cyber defense, it is also vital information for incident response. In the same way asset inventory and network diagram documentation are of paramount importance to the asset owner, they are also very attractive to an adversary. Hence, this information needs to be as rigorously protected as the ICS system itself.
2. Assess Risk
Risk assessments are instrumental in identifying security gaps and vulnerabilities. They are vital to prioritizing the application of controls and countermeasures to protect the organization. Once asset inventory has been completed or updated, thorough and regular risk assessments must be conducted to identify and prioritize (or re-prioritize) risk to key assets. The importance of risk assessments cannot be overstated. Indeed, risk and resilience assessments are now required of drinking water systems every five years per the America’s Water Infrastructure Act (AWIA) (S. 3021; Public Law 115-270, enacted October 23, 2018,) which amended Sec. 1433 of the Safe Drinking Water Act.
Risk is a function of vulnerability, threat and consequence but is often daunting to measure. The goal of a risk assessment is to identify and prioritize risk based on the likelihood that a threat or vulnerability could adversely impact an organization. There is no one-size-fits-all process for performing risk assessments. However, several free and voluntary programs and frameworks are available to assist organizations in determining their security posture, which includes assessing risk of its people, processes and technologies.
While not a risk assessment standard per se, the National Institute of Standards and Technology (NIST) Cybersecurity Framework is one of the foremost resources for informing risk assessments. It was originally released in 2014 in response to Executive Order 13636. Updated in 2018, the framework provides a prioritized, flexible and free approach to managing cybersecurity risks. It has been designed to help organizations better understand, manage and reduce cybersecurity risk and to foster relevant conversations across organizational stakeholders.
The American Water Works Association (AWWA) risk assessment standard, “J100-10: Risk and Resilience Management of Water and Wastewater Systems,” provides guidance on conducting risk assessments. It documents a process for identifying vulnerabilities to man-made threats, natural hazards and dependencies, and provides methods to evaluate the options for improving weaknesses.
Specifically designed for water and wastewater utilities is the “AWWA Cybersecurity Guidance & Tool,” which provide a water sector-specific approach to applying the NIST framework. The AWWA cybersecurity resources have been recognized by the Water Sector Coordinating Council, the U.S. Environmental Protection Agency (EPA), the Department of Homeland Security, NIST and multiple states as the baseline for assessing cybersecurity risk management. Through posing a series of use cases designed to best represent a utility’s application of various technology, the AWWA cybersecurity tool generates a report with prioritized controls that, if implemented, can help the utility mitigate cyber risks. Updated versions of the guidance and tool are due out in the summer of 2019. The updates will broaden their scope to address cybersecurity provisions in AWIA and enhance the functionality of the output to support utility self-assessment of the implementation status of recommended controls.
Another helpful tool is the EPA Vulnerability Self-Assessment Tool (VSAT,) which is compliant with the J100-10 standard. VSAT is a web-based tool that steps a utility through producing an assessment. According to EPA, a utility can use the tool to identify the highest risks to mission critical operations and find the most cost-effective measures to reduce those risks. EPA has also produced
An additional resource may be NIST’s SP 800-30 “Guide for Conducting Risk Assessments.” SP800-30 provides guidance for carrying out each step in the risk assessment process.
The Department of Homeland Security’s National Cybersecurity and Communications Integration Center (DHS NCCIC) Critical Infrastructure Assessment Program offers many free products and services to help raise awareness, identify security gaps and provide recommendations to assist organizations in managing cyber risk. Several consulting firms also provide these services. The outcome of any risk assessment will provide an organization with a current risk profile and inform prioritization of the initiatives that will improve the cybersecurity posture.
3. Minimize Control System Exposure
It is particularly important to understand any communication channels that exist between the industrial control system (ICS) network and other internal networks. According to critical infrastructure site assessments performed in the water and wastewater sector by NCCIC for FY2017, the most commonly identified weakness is a lack of appropriate boundary protection controls.
While isolating a control system from the rest of the world would be ideal, it may not be possible. Connections are difficult to avoid given the practical demands for remote system access by vendors and staff and due to the need to export control system data for regulatory and business purposes.
Even if these connections could be avoided, there are always control system upgrades and patches that make some kind of interface with the outside world unavoidable. Minimizing control system exposure requires a combination of physical and logical network segmentation, devices and software that restrict traffic, protection of control system design and configuration documents, encrypted communications, restrictive procedures and physical security.
External (Untrusted) Pathways
The control systems of some organizations may not directly face the internet. However, a connection likely exists if those systems are connected to another part of the network, such as the enterprise IT network, that has a communication pathway to or from the internet.
As most compromises to ICS networks emanate from the IT/business network, it is vital to eliminate any unnecessary communication channels discovered between devices on the control system network and equipment on other networks. Any connections that remain need to be carefully evaluated, managed and strengthened to reduce network vulnerabilities.
Similarly, a utility may have equipment or components that use Bluetooth or other short-range communications protocol for configuration. Despite the limited communication range of such devices, these connections represent another entry point for an adversary. Organizations may be unaware of these short-range connections, but cyber threat actors can find such pathways to access and exploit industrial control systems.
Access to network segments can be restricted by physically isolating them entirely from one another, which is optimal for industrial control systems, or by implementing technologies such as firewalls, demilitarized zones (DMZs), virtual local area networks (VLANs), unidirectional gateways and data diodes.
Network segmentation also entails classifying and categorizing IT and ICS/OT assets, data and personnel into specific groups or zones, and restricting access based on these groupings. By placing resources into different segments of a network, and restricting access to specific zones, a compromise of one device or system is less likely to translate into the exploitation of the entire system. When interconnected, cyber threat actors may be able to exploit any vulnerability within an organization’s system – the weakest link in the chain – to gain entry and move laterally throughout a network to access sensitive equipment and data. Given the rise of the “industrial internet of things” (IIoT,) whereby many previously non-internet connected protocols are being replaced with protocols like EtherCAT and Modbus TCP/IP to access greater automation, the importance of segmenting and partitioning networks is greater than ever
When installed and configured properly, firewalls, ICS-DMZs, VLANs, unidirectional gateways and data diodes provide crucial functions in filtering or blocking unwanted traffic that could adversely impact availability, reliability and safety of the control system network. By reducing the number of pathways into and between networks and by properly implementing security protocols on the pathways that do exist, it is much more difficult for a threat actor to compromise the network and gain access to other systems.
Creating network boundaries and segments and classifying assets and data empowers an organization to enforce both detection and protection controls within its infrastructure. The capability to monitor, restrict and govern communication flows provides a practical ability to baseline network traffic, especially traffic traversing a network boundary, and identifies anomalous or suspicious communication flows. These boundaries provide a means to detect potential lateral movement, network foot-printing and enumeration, and device communications attempting to traverse from one zone to another. To ensure unwanted traffic is not traversing the network, firewall and segmentation rules should be reviewed regularly to assess the status of unnecessary ports or services.
Although many water and wastewater utilities have invested necessary time and resources in cybersecurity, more progress is required within the sector to secure IT and OT systems. It is our goal to try to help establish best practices for utilities preparing risk and resilience assessments and emergency response plans required by the America’s Water Infrastructure Act (AWIA).