MAJOR SYSTEM FAILURE
(Electric, Heat, Air Conditioning, Gas Leak, Water/Sewer, ITS/Telecommunication)
To provide a safe and secure environment while assessing a situation. All reasonable efforts will be made to provide temporary service until permanent repairs are made. If the safety of the building occupants cannot be assured, senior management will be consulted for the relocation of occupants.
In the event of any major system failure University Police or the Facilities Services personnel will advise management of the situation in the following order:
- Assistant Director of Facilities Services
- Director of Facilities Services
- Director of Facilities Management
An immediate assessment will be made to determine if all life safety systems are operational and that no hazardous conditions are present. The senior facilities services and security personnel on site will make the final determination if a building is safe to occupy or if there should be relocation to another building.
Restoration of services will begin as soon as the safety and security of occupants and workers can be assured.
First responders to the scene shall conform to the following procedure as much as possible.
- Assess the situation
- Ensure the safety of occupants
- Ensure the safety of workers
- Abate hazardous conditions
- Secure University property to prevent further damage
- Notify management
PURPOSE AND RESPONSIBILITIES
To Information Technology Services (ITS), and many computer users on campus, a network, email, administrative system, or other outage constitutes an emergency. Naturally, these inevitable problems conflict with the goal of ITS, which is to secure wherever possible the integrity of the computing and network infrastructure, and provide uninterrupted reliable service to our clients.
In the event of a mass telecommunications failure affecting one or more buildings, the priority for the Crisis Management Team is to establish a method of communications so that the building occupants will be able to continue to operate and still have emergency communications as necessary.
The senior staff member on site at the time of the incident or the first on site following an incident will contact the Chief Information officer (CIO) and/or Directors, Applications, Communications, and infrastructure for a determination of the need to declare an incident.
The senior technology staff member on site at the time of the incident will assume immediate responsibility. The responsibility will be to ensure people are evacuated as needed. If injuries have resulted or may occur as a result of the incident, immediate attention will be given to those persons injured. Suffolk Police and Facilities Management will be notified if necessary. If the situation allows, attention will be focused on shutting down systems, turning off power, etc., but evacuation is the highest priority.
Campus servers and systems located in the Sawyer Data Center are connected to central Uninterruptable Power Supply (UPS) providing backup power for a period of 12 minutes in an event of a power loss. This is ample time for the emergency generator to automatically start providing continuous power to the systems in Data Center.
All telephone switches on campus are connected to a battery backup power source, providing a runtime of 4hrs after loss of power. However, the phone switch in Sawyer is connected to the data center UPS, and emergency power generator providing continuous power in an event of an extended power loss.
- Evaluate the extent of the damage, and if only degraded service can be obtained, determine how long it will be before full service can be restored.
- Replace hardware/software as needed to restore service to at least a degraded service.
- Perform system installation as needed to restore services. If backup files are needed and are not available from the on-site backup tapes, they will be requested and transferred from the off-site storage facility.
- Work with the various vendors, as needed, to ensure support in restoring full service.
- Keep disaster coordinators informed of status, progress, and problems.
The law school server room will be used as a secondary staging area in an event the main data center should the Sawyer building become inaccessible. All ITS directors will meet at the Law school for incident coordination.
TEAM MEMBERS/NECESSARY STAFF
The Team Leader(s) of Applications, Infrastructure and Communications Recovery Team are directly responsible for the monitoring and execution of the Disaster Plan and reporting progress and problems to the CIO who is a member of the Disaster Recovery Management team. Due to their Interdependencies, the above groups will be in constant communication during the recovery process.
Communications: Together they provide network infrastructure; network security management; switch integration and management; hubs and router management; off-campus network access (Internet), and Telephone systems. The recovery strategy is the implementation of recovering network functionality within the time constraints necessary to meet the requirements of operating critical systems. Communications services staff members are: Anupam Singh, Chris Ulrich, Derek Lam, Paul Gaurino, Norman Mortada, Jane Scherban, Andrea Bishop, Devan Deluca, and Meghan Sutherland.
Infrastruture: Provides comprehensive management of the university web servers, Email system, administrative system, Storage Area Network (SAN), and the Microsoft Active Directory infrastructure on campus. The recovery strategy is to restore the University’s computer processing capability and to recover computer support services. This group determines Hardware/Software requirements for recovery processing. The planned recovery hardware is kept current and reviewed periodically by this group. Computer Services staff members are Brendan Collins, Dave Macisso, Tim O’Neal, Keith Hersh, Greg Curtis, Dan Gomes, and James O’Connor.
Applications: Responsible for Blackboard, and Datatel/Colleague systems; detailed systems analysis; verification of applications methodologies and tools; and end-user applications and support. Administrative Information Systems staff members are Robert DiGuardia, Levon Sukiasyan, Paula DeGloria, Darlene Poplawski, Sue Scott, Heidi Galpern, and Bibiana Gonzalez.
Web Services: Responsible for the university web servers, and Red Dot Content Management System (CMS). Web Services staff members are Geert Kinthaert, Anne MacDonald, Michael Schneider, Steve Becker, Connie Gagnon, Joe MCEttrick, Carolyn Potts, and Adam Sparrow.
Telephones, Cell phones, 2-way radios, Laptop computers, printer, fax machine, copier and desks/chairs.
It is essential that both the Sawyer data center and Law school server room maintain HVAC cooling and emergency power. It is to be noted that the Law school HVAC is not currently on Emergency power.