Resolved -
Clearance as at 21/11 14:18:
Technical teams have confirmed there has been no errors since the rollback yesterday. All systems are now functioning as per BAU. Ticket has been raised to investigate the root cause.
No further updates
Nov 21, 14:30 NZDT
Monitoring -
Update as at 20/11 18:00:
Remedial work has been completed . Technical team will investigate the logs tomorrow during the peak traffic time .
Next update will be around 21/11 12pm or earlier if available.
Nov 20, 18:06 NZDT
Update -
Update as at 20/11 14:34:
Technical team will be carrying out remedial work from 17:00 to 17:30 tonight
Next update as soon as available.
Nov 20, 14:37 NZDT
Update -
Update as at 19/11 18:04:
Technical team will continue to investigate the logs tomorrow during the peak traffic time .
Next update will be around 20/11 12pm or earlier if available.
Nov 19, 18:15 NZDT
Update -
We are currently experiencing an issue with the Assurance website & Assurance API, since 15/11 09:56
Retail Service Provider (RSP – Customer) Impacts:
Testing of fibre and Copper ASIDs may intermittently return an error message stating "we are currently experiencing an outage and this data is not currently available", asking end users to try again.
Please Note: this is also impacting the ONT to RGW connectivity
Service Company Impacts:
Testing of fibre and copper ASIDs may return an error message .
Please Note: this is also impacting the ONT to RGW connectivity test.
Update as at 19/11 17:24:
Technical teams deployed a potential fix into the production environment. The errors have reduced but continue to be present. Technical teams continue to investigate.
Next update as soon as available.
Nov 19, 17:39 NZDT
Update -
Update as at 19/11 12:36:
Technical teams continue to investigate further.
Next update as soon as available.
Nov 19, 12:44 NZDT
Identified -
Update as at 18/11 17:28:
The issue has reoccurred. This has been escalated to technical teams to investigate further.
Next update as soon as available.
Nov 18, 17:37 NZDT
Update -
Users may have noticed a recurrence of this issue between 12:20-12:48 today. However it self-resolved and subsequent testing has been carried out successfully.
This incident will remain under monitoring until COB today at this stage.
Nov 18, 16:41 NZDT
Update -
The number of errors observed has now dropped off to a very low level.
This incident will remain under monitoring over the weekend, and technical teams will follow up and review on Monday morning.
Please Note: this is only affecting the ONT to RGW connectivity test.
Nov 15, 19:31 NZDT
Monitoring -
Functionality to all impacted applications/systems has been restored.
Monitoring will continue until COB today to confirm stability.
Nov 15, 14:54 NZDT
Update -
Technical teams have restarted a backend server but systems continue to be impacted.
Investigations are continuing.
Next update as soon as available.
Nov 15, 14:14 NZDT
Investigating -
We are currently experiencing issues with the following systems/applications, since 09:56 this morning:
Retail Service Provider (RSP – Customer) Impacts:
Assurance website & Assurance API: Testing of fibre ASIDs may return an error message stating "we are currently experiencing an outage and this data is not currently available", asking end users to try again.
Service Company Impacts:
TechMate: Testing of fibre ASIDs may return an error message stating "error 429 - too many requests".
Technical teams are urgently investigating.
Nov 15, 12:23 NZDT