Active Incident

Updated a few seconds ago

SiteBacker/Traffic Controller/Simple Load Balancing Failover/Failback DelaysOperational

Incident Status

Operational

Components

UltraDNS

Service

Zone Propagation


September 29, 2024 6:57PM UTC
[Monitoring] Current Status: We have resolved all issues related to SiteBacker/Traffic Controller/Simple Failover failover/failback at this time. We will continue to monitor. Previous Impact: 15:30 - 16:50 UTC: Manual failover/failback for SiteBacker/Traffic Controller/Simple Failover pools were delayed. 15:30 - 18:50 UTC: Probe-driven failover/failback for SiteBacker/Traffic/Simple Failover pools were delayed. While the issue was ongoing, there would have been propagation delays to any SiteBacker/Traffic Controller/Simple Failover whose serving records needed to change as a result of either manual or probe-driven changes. There was no impact to SiteBacker/Traffic Controller/Simple Failover pools that didn't need to make record updates and no impact to DNS records that do not use SiteBacker/Traffic Controller/Simple Failover.

September 29, 2024 6:19PM UTC
[Monitoring] Current Status: Probe-driven (automated) failovers/failbacks for SiteBacker/Traffic Controller/Simple Failover are restored in some regions but still facing delays in a few regions. We are working to restore to normal propagation times in remaining regions and will update once this issue has been resolved in all locations. Previous Impact: 15:30 - 16:50 UTC: Manual failovers for SiteBacker/Traffic Controller/Simple Failover were delayed.

September 29, 2024 5:48PM UTC
[Monitoring] Start Time: 15:30 GMT Current Status: We see SiteBacker/Traffic Controller/Simple Load Balancing force fail/force active updates processing again. However, we are still seeing issues with new probes and updates to probe definitions for existing pools. Previous Impact: All failover/failback events for Sitebacker/Traffic Controller and Simple Load Balancing pools were delayed.

September 29, 2024 5:03PM UTC
[Monitoring] Start Time: 15:30 GMT End Time: 16:50 GMT Current Status: We have mitigated the root cause and see SiteBacker/Traffic Controller/Simple Load Balancing failover/failback updates processing as normal again. We will continue to monitor. Previous Impact: All failover/failback events for Sitebacker/Traffic Controller and Simple Load Balancing pools were delayed.

September 29, 2024 5:00PM UTC
[Monitoring] We are failing over service to USWEST, and things are coming up. We will share updates in next 15 mins.

September 29, 2024 4:41PM UTC
[Identified] Starting from 15:30 AM GMT, all failover/failback events for Sitebacker/Traffic Controller and Simple Load Balancing pools are delayed. We have identified the root cause and are working on mitigation.

UltraDNS

DNS Resolution

Operational

Zone Propagation

Operational

Primary Zone Transfer

Operational

Secondary Zone Transfer

Operational

API

Operational

Portal

Operational

Reporting

Operational

UltraDNS Firewall

DNS Resolution

Operational

API

Operational

Portal

Operational

Reporting

Operational

Categorization

Operational

Customer Support

Support Portal

Operational

Phone

Operational

2

Upcoming Maintenances

2

Incidents Last 30 Days

0

Maintenances Last 30 Days

External Services

Status.io

Scheduled Maintenance

UltraDNS Traffic Services MaintenancePlanned Maintenance

Schedule

October 6, 2024 4:30PM - 6:30PM UTC

Components

UltraDNS

Service

Zone Propagation

Description

Failover/Failback of customer's SiteBacker, Traffic Controller, and Simple Failover pools may be delayed for ~15 minutes during maintenance window. There will be no impact on domain resolution.
UltraDNS Database MaintenancePlanned Maintenance

Schedule

October 13, 2024 4:30AM - 2:30PM UTC

Components

UltraDNS

Service

Zone Propagation, API, Portal

Description

For 10 minutes, during maintenance window, customers may see non-fulfillment for few requests while creating, updating and deleting zones(primary/secondary) and zone’s records. Customers may also observe the propagation delay ~10mins. However, there will not be any impact on domain resolution.
System status for UltraDNS services. For inquiries, visit https://www.ultraproducts.support/hc/en-us

History (Last 7 days)

UltraDNS - DNS Resolution AdvisoryOperational

Incident Status

Operational


Components

UltraDNS


Service

DNS Resolution



September 29, 2024 7:09PM UTC
[Resolved] Resolving incident.

September 26, 2024 6:19PM UTC
[Monitoring] Start: 15:24 End: 17:25 Status: We have identified the issue affecting DNS resolution. We are no longer seeing DNS resolution issues. We will continue to monitor Previous Impact: DNS timeouts and slowed responses for users for affected regions. Previously Affected Regions: Dallas, Chicago, Washington DC. We have closed the audio bridge now that the issue is resolved.

September 26, 2024 6:14PM UTC
[Monitoring] Start: 15:24 End: 17:25 Status: We have identified the issue affecting DNS resolution. We are no longer seeing DNS resolution issues. We will continue to monitor Previous Impact: DNS timeouts and slowed responses for users for affected regions. Previously Affected Regions: Dallas, Chicago, Washington DC. Listen Only Bridge: Dial in manually +18722428905 (Toll) Get local numbers Conference ID: 143 246 63

September 26, 2024 6:00PM UTC
[Identified] Start: 15:24 End: 17:25 Status: We have identified the issue affecting DNS resolution. We are no longer seeing DNS resolution issues. Start: 15:24 UTC End Time: 17:25 UTC Previous Impact: DNS timeouts and slowed responses for users for affected regions. Regions Affected: Dallas, Chicago, Washington DC. Listen Only Bridge: Dial in manually +18722428905 (Toll) Get local numbers Conference ID: 143 246 63#

September 26, 2024 5:54PM UTC
[Identified] Status: We have identified the issue affecting DNS resolution. We are not currently seeing DNS resoltuion issues but are still monitoring. Start: 15:24 UTC End Time: 17:25 UTC Impact: DNS timeouts and slowed responses for users for affected regions. Regions Affected: Dallas, Chicago, Washington DC. Listen Only Bridge: Dial in manually +18722428905 (Toll) Get local numbers Conference ID: 143 246 63#

September 26, 2024 5:48PM UTC
[Identified] Status: We have identified the issue affecting DNS resolution. We see improvements in the US East but are still working on the issues affecting US East and US Central. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses for users for affected regions. Regions Affected: Dallas, Chicago, Washington DC. Listen Only Bridge: Dial in manually +18722428905 (Toll) Get local numbers Conference ID: 143 246 63#

September 26, 2024 5:26PM UTC
[Identified] Status: We have identified the issue affecting DNS resolution. We see improvements in the US East but are still working on the issues affecting US East and US Central. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses for users for affected regions. Regions Affected: Dallas, Chicago, Washington DC.

September 26, 2024 5:04PM UTC
[Identified] Status: We have identified the issue affecting DNS resolution. We see improvements in the US East but are still working on the issues affecting US East and US Central. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses for users for affected regions. Regions Affected: Dallas, Chicago, Washington DC.

September 26, 2024 4:50PM UTC
[Identified] Status: We have identified the issue affecting DNS resolution. We see improvements in the US East but are still working on the issues affecting US East and US Central. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses for users for affected regions. Regions Affected: Dallas, Chicago, Washington DC.

September 26, 2024 4:28PM UTC
[Identified] Status: We have identified the issue affecting DNS resolution. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses for users (primarily US East and US Central) routed to the regions affected. Regions Affected: Dallas, Chicago, Washington DC.

September 26, 2024 4:11PM UTC
[Identified] Start Time: 15:24 UTC Status: We have identified the issue affecting DNS resolution. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses for users (primarily US East and US Central) routed to the regions affected. Regions Affected: Dallas, Chicago, Washington DC.

September 26, 2024 4:00PM UTC
[Identified] Start Time: 15:24 UTC Status: We have identified the issue affecting DNS resolution. Our teams continue working to restore services as soon as possible. We will provide updates every 15 minutes or until we have an estimated time to repair. Impact: DNS timeouts and slowed responses. Regions Affected: Some US East and US Central Locations

September 26, 2024 3:42PM UTC
[Investigating] Start Time: 15:24 GMT We are actively investigating a DNS resolution issue and are making every effort to restore services as soon as possible. We will provide updates as they occur or every 15 minutes, whichever comes first.