-
Improvement
-
Resolution: Won't Do
-
Minor
-
None
As I'm following the development quite a bit, I noticed you are using an official AWS AMI as agent.
I just want to make you aware of an issue I have been debugging with cloudbees stuff for a couple of months until Yoann Dubreuil finally found the issue...
I was having issues to execute long running DB update tasks from within jenkins, everytime the tasks ran into a timeout...
This was caused by a timeout on the NAT gateway.
You can read more about it here:
- http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html#nat-gateway-troubleshooting-timeout
- http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-firewall-guidance.html#connecting-firewall-guidance.change-tcpip-settings
The issue could finally be fixed with the following settings (placed in a file in `/etc/sysctl.d`):
net.ipv4.tcp_keepalive_time=200 net.ipv4.tcp_keepalive_intvl=200 net.ipv4.tcp_keepalive_probes=5
As I'm very interested in using evergreen myself one day, I would love to see this find place in the AWS evergreen setup.
Here is an other article describing the same issue: https://www.vertica.com/blog/configuring-tcp-idle-settings-long-running-idle-sessions/
[JENKINS-54317] change NAT timeout
| Summary | Original: increase NAT timeout | New: change NAT timeout |
| Description |
Original:
As I'm following the development quite a bit, I noticed you are using an official AWS AMI as agent. I just want to make you aware of an issue I have been debugging with cloudbees stuff for a couple of months until Yoann Dubreuil finally found the issue... I was having issues to execute long running DB update tasks from within jenkins, everytime the tasks ran into a timeout... This was caused by a timeout on the NAT gateway. You can read more about it here: * [http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html#nat-gateway-troubleshooting-timeout] * [http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-firewall-guidance.html#connecting-firewall-guidance.change-tcpip-settings] The issue could finally be fixed with the following settings (placed in a file in `/etc/sysctl.d`): {code:java} net.ipv4.tcp_keepalive_time=200 net.ipv4.tcp_keepalive_intvl=200 net.ipv4.tcp_keepalive_probes=5{code} As I'm very interested in using evergreen myself one day, I would love to see this find place in the AWS evergreen setup. |
New:
As I'm following the development quite a bit, I noticed you are using an official AWS AMI as agent. I just want to make you aware of an issue I have been debugging with cloudbees stuff for a couple of months until Yoann Dubreuil finally found the issue... I was having issues to execute long running DB update tasks from within jenkins, everytime the tasks ran into a timeout... This was caused by a timeout on the NAT gateway. You can read more about it here: * [http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html#nat-gateway-troubleshooting-timeout] * [http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-firewall-guidance.html#connecting-firewall-guidance.change-tcpip-settings] The issue could finally be fixed with the following settings (placed in a file in `/etc/sysctl.d`): {code:java} net.ipv4.tcp_keepalive_time=200 net.ipv4.tcp_keepalive_intvl=200 net.ipv4.tcp_keepalive_probes=5{code} As I'm very interested in using evergreen myself one day, I would love to see this find place in the AWS evergreen setup. Here is an other article describing the same issue: [https://www.vertica.com/blog/configuring-tcp-idle-settings-long-running-idle-sessions/] |
| Assignee | Original: R. Tyler Croy [ rtyler ] |
| Resolution | New: Won't Do [ 10001 ] | |
| Status | Original: Open [ 1 ] | New: Closed [ 6 ] |