With further work completed last night to the fibre back-end system (i.e. the inclusion of two more interface daemons) Chorus are resolving this issue.
The temporary measure are still in place (i.e. extending the timeout response) as Chorus and support partners continue to monitor the situation given the issues to-date.
Posted Jul 11, 2019 - 10:06 NZST
Update
We are continuing to investigate this issue.
Posted Jul 10, 2019 - 10:02 NZST
Update
Though the experience of the timeouts decreased dramatically yesterday (compared to the issues experienced on Monday 9 July) Last night, increases to the response timeouts were implemented on the API’s consumed by Techmate & Checkmate, including the line test API, when testing the fibre service.
This is only a temporary measure (as Chorus and support patterns continue their investigation) to provide as much time for the caching to occur as we noticed a behaviour from the field service techs of constantly re-running tests, which in turns adds more load to the back-end system.
What is the experience a person using Techmate, Checkmate or consuming the Line Test API be? When the back end system is under-load, a test in flight will timeout at 120seconds. When the timeout occurs, the person will receive an error message and on re-try hit the cache and be provided the information they need. This will be the same for the line test api.
Please note: The 120secs is only associated for timeouts, not for the length of time for every test that is triggered.
Posted Jul 10, 2019 - 08:47 NZST
Investigating
Technical teams are currently investigating large amounts of time outs occurring when calling Line Test API for Fibre products as well as slower than usual responses to Fibre Tests.
This impact also includes Tech Mate and Check Mate.
Posted Jul 08, 2019 - 15:10 NZST
This incident affected: Service Partner Websites and APIs (Tech Mate (Testing Fibre Services)).