Extracted Ticket


Ticket Number  : SUNETTICKET-606
Ticket Type    : Unscheduled
Ticket Status  : Closed
Ticket Summary : Loss of connectivity
Ticket Scope   : 

Ticket Opened  : 20100308 18:14 UTC
Ticket Closed  : 20100311 12:11 UTC

Problem Start  : 20100308 18:26 UTC
Problem End    : 20100310 13:03 UTC

Affected organisations:

      * LTU, IRF, SUNET
        ----

Description:

      * After activation of new software (SUNETTICKET-596), we lost
        connectivity to NR_421_LL_Cy.
        ----

Impact:

      * Loss of management.
        LTU and IRF red network.
        ----

Update:

      * 20100308 18:26 UTC
        Escalated to IPnett.
        ----
        
      * 20100309 8:15 UTC
        Technician is on site and the main card in the 4200 will be
        reseated in attempt to restore management, this will cause a
        few minutes downtime on in the red network.
        ----
        
      * 20100309 8:24 UTC
        Management is still down, there will be a new intervention
        tonight at 16:00 UTC when the flash will be swapped.
        ----
        
      * 20100309 15:38 UTC
        The intervention is rescheduled for today at 16:30 UTC.
        ----
        
      * 20100309 16:26 UTC
        The maintenance starting now.
        ----
        
      * 20100309 19:22 UTC
        Red connection to LTU is still down after intervention,
        investigation is ongoing.
        ----
        
      * 20100309 21:27 UTC
        The management is restored but the red connection to LTU and
        IRF is still down. 
        
        ----
        
      * 20100310 0:33 UTC
        Connectivity to LTU has been restored at 21.50 UTC. 
        Connectivity to IRF is still down due to some configuration
        issue in the node after software upgrade and further
        troubleshooting will continue tomorrow 10/3.
        ----
        
      * 20100310 8:52 UTC
        Our supplier is working together with vendor in order to
        restore full functionality.
        ----
        
      * 20100310 13:18 UTC
        All the services are now fully restored.
        4200 in Luleå will be rebooted after 17:00 local time in
        order to activate new software once again, which may lead to
        short outage.
        
        ----
        
      * 20100310 16:52 UTC
        The maintenance planned for today is cancelled. 
        ----
        
        
Final ticket report:

      * The connectivity to the node NR_421_LL_Cy has been restored
        after swapping the flash card.
        ----