NAS - Confirmed Service Issue
Incident Report for Duke IT
Resolved
OIT has monitored the service and can confirm that the issue with NAS has been resolved.

If you are still experiencing service issues please contact the OIT Service Desk: https://oit.duke.edu/help
Posted Feb 17, 2020 - 10:59 EST
Update
OIT will continue to monitor the service but initial indications are that the issue with NAS was resolved late Friday evening.

If you are still experiencing service issues please report those to the OIT Service Desk: https://oit.duke.edu/help
Posted Feb 17, 2020 - 08:45 EST
Monitoring
We have identified a number of problematic jobs that were causing problems for the storage system and stopped them. The system appears to be stable but we are closely monitoring it for signs of additional problems.

Research NAS users may now establish new jobs.
Posted Feb 14, 2020 - 23:50 EST
Identified
OIT is still working to resolve continuing NAS performance issues (slowness in client connections over NFS) on research storage impacting the following NAS servers:

ssri-nas-fe13
oit-nas-fe13
oit-nas-fe13dc
oit-nas-fe13f

Research Toolkits (Rapid VMs) and Research Computing (DCC) home and group directories are also impacted by this issue.

We have disabled logins to the Duke Computer Cluster while we continue to work on this issue.

At 10 pm we will disconnect client connections to the research NAS. At that time all client connections from Research Computing services will be interrupted for up to 10 minutes. Most clients will automatically reconnect after the reset.

We ask that research NAS users do not establish new jobs on any services until we conclude these activities.
Posted Feb 14, 2020 - 09:08 EST
Update
This NAS issue continues to be intermittent. OIT is working with the vendor on a permanent solution.
Posted Feb 13, 2020 - 09:34 EST
Monitoring
OIT has monitored the service and indications are that the issue with NAS has stabilized. OIT continues to work with the vendor on a permanent solution.

If you are still experiencing service issues please report those to the OIT Service Desk: https://oit.duke.edu/help
Posted Feb 12, 2020 - 14:10 EST
Update
This issue continues to impact the NAS service intermittently. OIT staff are still working on a permanent solution but there is no estimated time to resolution.
Posted Feb 11, 2020 - 14:58 EST
Identified
OIT has confirmed a service issue with certain NAS servers. There are performance issues (slowness in client connections over NFS) on research storage which impacts the following NAS servers:

ssri-nas-fe13
oit-nas-fe13dc
oit-nas-fe13f

Research Computing (DCC) home and group directories are also impacted by this issue.

OIT staff are working to restore service but there is no estimated time to resolution.

If you have questions about this outage or would like to report additional service impacts please contact the OIT Service Desk: https://oit.duke.edu/help

For more information about NAS visit OIT's web page: https://oit.duke.edu/what-we-do/applications/nas-server-and-archive
Posted Feb 10, 2020 - 13:44 EST
This incident affected: Virtualization and Storage (NAS).