Incident Preparedness
Incidents happen all of the time. They vary in type and impact. As we saw with the Crowdstrike outage (July 19, 2024), it's not always a cyber-attack. While I have been involved with nearly 100 ransomware incidents, I have also seen a wide variety of problems. Here are some memorable ones from my own experience:
-
A Microsoft bug caused virtual NICs to reset - this led to servers going down at many Hospitals
-
One Hospital's systems went down 5 times in 2 years due to Squirrel activity (check out cyberSquirrel for a map of power outages due to squirrels and other critters)
-
Windows servers would exhaust all available ports when not rebooted in over a year (due to a microsoft bug) - periodically a Hospital's EHR would go down due to this. ​
-
A major vulnerability in a product was found which allowed PHI to be cached on end user devices
Incident preparedness goes beyond the creation of incident response policies, procedures, and playbooks. It's a process! Key stakeholders from across the organization must be involved. The process itself has great value. Critical systems and processes will be identified and contingency plans created. The stakeholders will learn to collaborate together so that they will function as a team when an incident does occur.
​​
​
Learn more in the article below, but first a word from Ike.
Building Your Incident Preparedness Program
​1. Determine the Scope of your Incident Response program — Your program will depend a lot on the type of products and services you provide. Work with the executives to determine which products and services are top priorities, and the types of incidents which concern them the most.
​
2. Perform a Business Impact Analysis — For each identified critical system or service (e.g., email, zoom, customer support platforms, sales platform, O365, etc.), determine the following:
-
What is the maximum acceptable outage (MAO)? Anything longer than this time would result in unacceptable impacts on the business.
-
Recovery Time Objective (RTO) - The target time for recovery (it should be less than the MAO)
-
Recovery Point Objective (RPO) - The maximum acceptable amount of data loss, measured in time (for example, the last 30 minutes of data may be considered an acceptable loss)
-
Impact - Possible impacts of downtime
-
Responsible Parties - Person primarily responsible for maintaining the system
-
Call List - People who must be informed (including customers) when the system goes down, and how this is done (Posting on website, phone call, email)
-
Contingency Plan - Alternative systems or processes that go into effect when this system is down. (e.g., if the phone system is down, staff may use their own cellular phones, and a phone directory is provided)
​
3. Incident Response Policies — Create policies which express the intent and priorities of the executives. Generally the policy will define such key objectives as:
-
Human safety is the top priority
-
Ensure critical systems are available for our customers
-
Applicable Regulations (such as HIPAA, GPDR, Data Privacy Laws, FCC, and FTC regulations)
-
External communications must all go through General Counsel (or another executive)
-
When Law Enforcement should be contacted
​
4. Assemble a Team — Assemble a team from across the organization that will be involved in Incident Response. IR is not simply an IT exercise. It should include executives, general counsel, senior management, and operations staff.
​
5. Incident Response Planning — Starting with most critical systems, work through the incident response plans for each system with their responsible team. Using a template, create a series of playbooks for likely incidents.
Common incidents include:​
-
Phishing email
-
Credential theft
-
"CEO Scams" which lead to wire transfer fraud or the exposure of confidential information, such as W2s
-
Vishing - voice phishing where a staff member has been contacted by a criminal by phone
- Data Breach
- Insider Threat
- Fraud (by an insider)
-
Malware on an end user device or server
-
Hardware Failure
-
Compromise of a Third Party (a vendor or supplier)
-
Physical Security breach, including active shooter
​​
6. Education
-
Educate the Incident Response team on the IR priorities, plans, and processes.
-
Educate the company at large on how to report incidents, the importance of confidentiality, and how to do their job during a down time.
​
7. Tabletop Exercises — Create and perform tabletop exercises which run through the most important playbooks. A tabletop exercise should include:
-
A "hot wash" afterwards to garner lessons learned
-
Feedback from participants on the process
-
After Action report - Written documentation of what occurred, the lessons learned, and improvements to be made.
​
8. Rinse and Repeat — You are never done. Planning is something that goes on for the life of the organization. There will always be new threats, new vulnerabilities, new systems, new priorities, and new staff.
​
​Crisis Management, Crisis Communications
You may feel you have done a good job managing a crisis, but how does it appear to your customers and the public at large? Your lawyers no doubt advised you to release little information, and that led to your slow acknowledgement of the incident. Worse yet, your organization has failed to comment publicly and the media has pieced together a story based on rumor and social media posts.​
​
When an organization fails at crisis communications in this manner, they have often "won the battle but lost the war." True, they have made their lawyers happy but what about their customers? Has your hard won reputation been lost or tarnished?
​
In Conclusion
There is a lot to consider when planning for an Incident. Wondering where to start? Please contact me for a free consultation.
​
Would you like a free consultation on how to get started with Incident Response Preparedness?​
Book a meeting using my calendly.
Helpful Resources:
-
CISA Incident Response Plan (IRP) Basics — https://www.cisa.gov/sites/default/files/2024-02/Incident-Response-Plan-Basics_508c_1Feb2024.pdf
-
NIST Computer Security Incident Handling Guide — https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-61r2.pdf
-
Carnegie Mellon "Guide to Effective Incident Management Communications" — https://insights.sei.cmu.edu/library/guide-to-effective-incident-management-communications/
-
"Crisis Communications — The Definitive Guide to Managing the Message" by Steven Fink (book)
-
Homeland Security Exercise and Evaluation Program — https://www.fema.gov/emergency-managers/national-preparedness/exercises/hseep
-
FEMA Safe Exercise Best Practices - https://www.fema.gov/sites/default/files/documents/fema_safe-exercise-best-practice_06072021.pdf