Incident Report - January 29-30, 2013


Server issues and fixes. View this email in your browser
You're receiving this email because you opted to be notified of incidents related to the processing of your forms.

Between 12:44 a.m. and 2:31 p.m. EST on Jan. 29, and again on Jan. 30 between 10:05 a.m. and 1:32 p.m. EST, parts of our service were sporadically unavailable as our server capacity was overloaded due to unusually high traffic. 
 
During that time, respondents experienced slow form loading and many failed submissions due to timeouts. Affected customers were users on our Starter, Pay-As-You-Go, Basic, and Professional plans. Customers on our Enterprise Plan were not affected.
 
The incident on the 30th was more severe and sustained than the one on the previous day (Uptime Report).
 
Following the incident on the 29th, our first effort went into fixing a server configuration issue: the server was accepting more concurrent connections than it could handle, and ended up saturating RAM and CPU resources. The heavy traffic reoccurred on the 30th before we had time to make further progress.
 
On the 30th, after spending some time monitoring traffic and server capacity, we decided to bring our standby server online and redirect some traffic to that server. This returned us to normal operations at around 1:30 p.m.
 
Overall, we can't be too satisfied with how things went. The traffic spike caught us with our guard down and we took too long to bring additional capacity online. 
 
Some of the issues that we're going to address immediately:
  1. More RAM. The memory profile of each HTTP request depends on what each form is configured to do. File uploads or Salesforce integration, for instance, consume more memory than other requests. When traffic is high, this can tip a server over capacity, so we need to account for this more carefully.
  2. Add an additional spare server to handle peak load. We'll make an announcement when the server is ready. Customers who use IP whitelisting in Salesforce (or other services) will need to update their configuration accordingly.
Beyond this, we will continue to work on further infrastructure improvements. We're fortunate to have a growing customer base, and it's our responsibility to make sure we deliver a top class service.
 
You can always check our service status at http://status.formassembly.com or follow @FormAssembly on Twitter for updates.
 
We apologize for the downtime and the inconvenience caused. We're incredibly grateful for your patience and support. Please let us know if you have any feedback or comments regarding this issue.
 
The FormAssembly Team
Blog
Twitter
Copyright © 2013 Veer West LLC / FormAssembly, All rights reserved.
You're receiving this message because you opted to receive updates from FormAssembly. If you'd rather not hear from us, you can update your notification preferences in your FormAssembly account.

Our mailing address is:
Veer West LLC / FormAssembly
101 W. Kirkwood Ave. Suite 224
Bloomington, IN 47404

Add us to your address book
unsubscribe from this list   update subscription preferences 


0 comentarios:

Publicar un comentario

"No dudes en preguntar, intentaré ayudarte. Tus comentarios son importantes y si te gustó esta entrada, no olvides suscribirte a mi feed.

Algunos consejos.

- No escribas en mayúsculas, en un blog significa "gritar".

- Esto es un blog, no un móvil, por lo tanto no escribas tipo sms.

- No hago intercambio de links.

- Evita comentarios tipo "Hola, gran blog mi blog es..."

- Ten habilitado tu perfil y podré visitarte.

- Antes de preguntar asegúrate de que no se haya contestado o explicado antes, usa el buscador.

Siento deciros que si no se siguen, los comentarios serán eliminados.



 

OnlinEstrenos Copyright © 2010