Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61167

Inedo ProGet plugin when uploading Package file to proget I'm getting 504 Gateway Time-out Error

      Hi all,

      I'm having declarative pipeline in my Jenkinsfile code.  I changed my Jenkinsfile code as below, because I wanted to archive my artifact file by using Inedo ProGet plugin into my Proget site.   I already configured Proget in Jenkins with API key and installed the plugin. 

      I have added the ProGet API Key settings into the Jenkins' system config settings and it's testing the connection successfully. 

       

      Below is the code change I added in my Jenkinsfile.

      So as above script underline shows the way I'm getting my build artifact's Version Number. 

      Also below is the new .upack Package file I'm creating with the PackageName "JenkinsPackage.upack" for the purpose of upload to the Proget site.
       
      But here the issue I'm facing is when I'm running my build job and it's not success and failing the Proget Upload command with below error log message from Console-Output. 

       

       

       Please anyone assist me on this ?  What's the issue I'm having here, what blocking me to uploading my package file into Proget site ?

        1. proget_22Capture.PNG
          proget_22Capture.PNG
          63 kB
        2. proget1_Capture.PNG
          proget1_Capture.PNG
          162 kB
        3. Capture.PNG
          Capture.PNG
          97 kB
        4. image-2020-02-20-08-24-40-897.png
          image-2020-02-20-08-24-40-897.png
          37 kB
        5. image-2020-02-20-08-21-46-822.png
          image-2020-02-20-08-21-46-822.png
          223 kB
        6. image-2020-02-20-08-19-00-759.png
          image-2020-02-20-08-19-00-759.png
          114 kB
        7. image-2020-02-20-08-10-02-266.png
          image-2020-02-20-08-10-02-266.png
          31 kB
        8. image-2020-02-20-08-02-45-754.png
          image-2020-02-20-08-02-45-754.png
          124 kB

          [JENKINS-61167] Inedo ProGet plugin when uploading Package file to proget I'm getting 504 Gateway Time-out Error

          Andrew Sumner added a comment - - edited

          I will have a look at this over the weekend and get back to you as soon as I can. 

          Could you let me know the versions you are using for the following:

          • Jenkins
          • Jenkins ProGet Plugin
          • ProGet application

          I know there is an issue with large files (over 200MB) that I'm hoping to look into soon, however I don't think this would explain your issue.

          Andrew Sumner added a comment - - edited I will have a look at this over the weekend and get back to you as soon as I can.  Could you let me know the versions you are using for the following: Jenkins Jenkins ProGet Plugin ProGet application I know there is an issue with large files (over 200MB) that I'm hoping to look into soon, however I don't think this would explain your issue.

          Amila Gunathilake added a comment - - edited

          Hi Mr. andrewsumner,

           * My Current Jenkins Version - Jenkins ver. 2.220
           * Jenkins ProGet Plugin Version -  1.3
           * ProGet application  -  Version 5.2.24 (Build 6)

          I don't think file size will be a problem here because my package file (JenkinsPackage.upack) size is just 58 mb.

          also is there any issue with the way I'm getting the $FINANCIALVERSION number here?  Because I'm using a customize version number instead of using ${BUILD_NUMBER} as version. .  

          Amila Gunathilake added a comment - - edited Hi Mr.  andrewsumner ,  * My Current Jenkins Version -  Jenkins ver. 2.220  * Jenkins ProGet Plugin Version -  1.3  * ProGet application  -  Version 5.2.24 (Build 6) I don't think file size will be a problem here because my package file (JenkinsPackage.upack) size is just 58 mb. also is there any issue with the way I'm getting the $FINANCIALVERSION number here?  Because I'm using a customize version number instead of using ${BUILD_NUMBER} as version. .  

          Andrew Sumner added a comment - - edited

          I haven't been able to duplicate this issue on my setup - the best I have come up with is to get a 400 level errors when I put in invalid parameters for ProGet.

          Can you create a new pipeline with the following code (you may need to tweak some values) as this works for me, I just want to understand if it's a problem with what you are sending or an access issue. 

           
          pipeline {
            agent any
            stages {
              stage('Main') {
                steps {
                  bat '''DEL *.TXT /Q
                      if not exist "subfolder" mkdir subfolder
                      DEL *.upack /Q
                      ECHO Build Tag: %BUILD_TAG% > subfolder/Example.txt'''
                 
                  script {
                      VERSION = "1.1.${BUILD_NUMBER}"
                  }
                 
                  echo "version = '${VERSION}'"
                 
                  uploadProgetPackage artifacts: 'subfolder/Example.txt', feedName: 'Example1', groupName: 'jenkins/pipleline', packageName: 'JenkinsPackage', version: "${VERSION}"
                }
              }
            }
          }
          Sorry I don't have a better answer for you, however I'm sure we can get to the bottom of this.

          Is there anything else you can tell me about your setup - ProGet or the Jenkins build? eg:

          1. I assume they are on the same network - no firewalls in they way
          2. Is the jenkins job running on an agent or on the Jenkins master node
          3. ?

           

           

          Andrew Sumner added a comment - - edited I haven't been able to duplicate this issue on my setup - the best I have come up with is to get a 400 level errors when I put in invalid parameters for ProGet. Can you create a new pipeline with the following code (you may need to tweak some values) as this works for me, I just want to understand if it's a problem with what you are sending or an access issue.    pipeline {   agent any   stages {     stage('Main') {       steps {         bat '''DEL *.TXT /Q             if not exist "subfolder" mkdir subfolder             DEL *.upack /Q             ECHO Build Tag: %BUILD_TAG% > subfolder/Example.txt'''                 script {             VERSION = "1.1.${BUILD_NUMBER}"         }                 echo "version = '${VERSION}'"                 uploadProgetPackage artifacts: 'subfolder/Example.txt', feedName: 'Example1', groupName: 'jenkins/pipleline', packageName: 'JenkinsPackage', version: "${VERSION}"       }     }   } } Sorry I don't have a better answer for you, however I'm sure we can get to the bottom of this. Is there anything else you can tell me about your setup - ProGet or the Jenkins build? eg: I assume they are on the same network - no firewalls in they way Is the jenkins job running on an agent or on the Jenkins master node ?    

          Amila Gunathilake added a comment - - edited

          Hi Mr. andrewsumner sorry for the late reply, I'm getting the same error after your requested code change as well.  But I had a doubt with my uploading file (financialscripts.7z) size because after creating "JenkinsPackage.upack" file it has almost 58 mb in size.  Please see the below screenshot.

           

          So what I did is instead of financialscripts.7z file I used a small .txt file to upload which only has 3 mb of data and it uploaded successfully to proget site. 

          So I'm 100% sure the issue is with the size of "JenkinsPackage.upack" file which includes financialscripts.7z with almost 58 mb of data.

          Also there are no any Firewall issues between Jenkins and Proget.  Yes Jenkins build (workspace) happening in Jenkins's builder node (agent).

          I hope these information will help you. How can I sort out this filesize issue and is there any limitation of file size when uploading files into Proget andrewsumner ?

          Thanks,
          Amila

           

          Amila Gunathilake added a comment - - edited Hi Mr.  andrewsumner sorry for the late reply, I'm getting the same error after your requested code change as well.  But I had a doubt with my uploading file (financialscripts.7z) size because after creating "JenkinsPackage.upack" file it has almost 58 mb in size.  Please see the below screenshot.   So what I did is instead of financialscripts.7z file I used a small .txt file to upload which only has 3 mb of data and it uploaded successfully to proget site.  So I'm 100% sure the issue is with the size of "JenkinsPackage.upack" file which includes financialscripts.7z with almost 58 mb of data. Also there are no any Firewall issues between Jenkins and Proget.  Yes Jenkins build (workspace) happening in Jenkins's builder node (agent). I hope these information will help you. How can I sort out this filesize issue and is there any limitation of file size when uploading files into Proget  andrewsumner  ? Thanks, Amila  

          Amila Gunathilake added a comment - - edited

          Mr.  Sir andrewsumner this document maybe help-full for us to solve this matter. 
           
          https://forums.inedo.com/topic/137/proget-uploading-larger-files

          Amila Gunathilake added a comment - - edited Mr.  Sir andrewsumner this document maybe help-full for us to solve this matter.    https://forums.inedo.com/topic/137/proget-uploading-larger-files

          Andrew Sumner added a comment -

          I don't think that is the problem - that is quite an old issue and I'm able to upload 200MB files to my ProGet instance.

          There is an issue with this plugin (rather than proget) around file size but I thought that was well in excess of 58 MB and throws an out of memory exceptions rather than Bad Gateway.  I've been meaning to fix this for a while so will look for a solution. Seems to be an issue running under Jenkins as the code doesn't have this problem when run as a unit test.

          If you can't wait for this there is a [command line tool|https://docs.inedo.com/docs/upack/tools-and-libraries/upack-cli] for ProGet that you could use instead of this plugin. Obviously not as handy but would get your pipeline running while I work on the plugin.

          Andrew Sumner added a comment - I don't think that is the problem - that is quite an old issue and I'm able to upload 200MB files to my ProGet instance. There is an issue with this plugin (rather than proget) around file size but I thought that was well in excess of 58 MB and throws an out of memory exceptions rather than Bad Gateway.  I've been meaning to fix this for a while so will look for a solution. Seems to be an issue running under Jenkins as the code doesn't have this problem when run as a unit test. If you can't wait for this there is a [command line tool| https://docs.inedo.com/docs/upack/tools-and-libraries/upack-cli ] for ProGet that you could use instead of this plugin. Obviously not as handy but would get your pipeline running while I work on the plugin.

          hi andrewsumner my concern is when I'm uploading a 3 mb .txt file the plugin is working but when I'm uploading 58 mb .7z file its giving this bad gateway error. So if it's not a file size issue is it an issue with .7z file extension  ? 

          I'm sure it's not an network issue. In the mean time I will try to set up the UPack CLI tool. 

          Amila

           

          Amila Gunathilake added a comment - hi andrewsumner my concern is when I'm uploading a 3 mb .txt file the plugin is working but when I'm uploading 58 mb .7z file its giving this bad gateway error. So if it's not a file size issue is it an issue with .7z file extension  ?  I'm sure it's not an network issue. In the mean time I will try to set up the UPack CLI tool.  Amila  

          Andrew Sumner added a comment -

          It shouldn't be an issue with the .7z extension as this is packed into a universal package before being uploaded to proget, however I will do some testing to rule that out.

          Andrew Sumner added a comment - It shouldn't be an issue with the .7z extension as this is packed into a universal package before being uploaded to proget, however I will do some testing to rule that out.

          Amila Gunathilake added a comment - - edited

          Hi Mr andrewsumner it's seems like uploading my JenkinsPackage.upack package file including .7z file inside into the Proget site.  It's actually uploaded. But after uploading Jenkins gives  504: Gateway Time-out error message and existing my Jenkins build job with that error.  Please see below screenshots. 

          actually my artifact uploaded to proget correctly but its taking more than 5 minutes to upload to the proget site and that's where I'm getting the 504 error. Because I noticed even after I got the Jenkins Console output error I have to wait another 5 minutes to get the uploaded file in Proget site.

          as you know after that gives the below error and existing Jenkins build job:

          Amila Gunathilake added a comment - - edited Hi Mr  andrewsumner it's seems like uploading my JenkinsPackage.upack package file including .7z file inside into the Proget site.  It's actually uploaded. But after uploading Jenkins gives  504: Gateway Time-out error message and existing my Jenkins build job with that error.  Please see below screenshots.  actually my artifact uploaded to proget correctly but its taking more than 5 minutes to upload to the proget site and that's where I'm getting the 504 error . Because I noticed even after I got the Jenkins Console output error I have to wait another 5 minutes to get the uploaded file in Proget site. as you know after that gives the below error and existing Jenkins build job:

          Andrew Sumner added a comment -

          I have tried a few more things, still can't get the gateway error though.

          • 7zip file worked fine on a medium sized file of 56 MB
          • On a larger file (155 MB) I get "java.lang.OutOfMemoryError: Java heap space" - this is running on master which is a Windows Server 2016 with 4 GB RAM
          • I then ran build on an agent (Windows Server with 24 GB RAM) and the larger file was uploaded just fine.

          I cannot say for certain that your issue is memory related, but as it's an issue I need to fix I will and keep my fingers crossed it also fixes your issue.

          Andrew Sumner added a comment - I have tried a few more things, still can't get the gateway error though. 7zip file worked fine on a medium sized file of 56 MB On a larger file (155 MB) I get "java.lang.OutOfMemoryError: Java heap space" - this is running on master which is a Windows Server 2016 with 4 GB RAM I then ran build on an agent (Windows Server with 24 GB RAM) and the larger file was uploaded just fine. I cannot say for certain that your issue is memory related, but as it's an issue I need to fix I will and keep my fingers crossed it also fixes your issue.

          Andrew Sumner added a comment -

          Sorry, didn't see your post before I sent mine.

          The length of time you're stating is awfully long for a 58 MB file, none of my uploads have taken anywhere near that time so this is possibly hitting some sort of limit within ProGet and you will need to approach Inedo for help on that. But before you go down that route it would be good to prove that this is the case...

          Have you tried the command line client? You wouldn't even need to run it under Jenkins, in-fact I would prefer to take Jenkins out of the mix at first and only when that is working replicate that in Jenkins.

          If you CAN upload the file using the command line tool (both via Jenkins and direct), it at least proves there is a problem with this plugin.
          If not CANNOT upload the file from the command line tool then you will need to raise an issue via the support channel (and also let me know)

          Andrew Sumner added a comment - Sorry, didn't see your post before I sent mine. The length of time you're stating is awfully long for a 58 MB file, none of my uploads have taken anywhere near that time so this is possibly hitting some sort of limit within ProGet and you will need to approach Inedo for help on that. But before you go down that route it would be good to prove that this is the case... Have you tried the command line client? You wouldn't even need to run it under Jenkins, in-fact I would prefer to take Jenkins out of the mix at first and only when that is working replicate that in Jenkins. If you CAN upload the file using the command line tool (both via Jenkins and direct), it at least proves there is a problem with this plugin. If not CANNOT upload the file from the command line tool then you will need to raise an issue via the support channel (and also let me know)

          Andrew Sumner added a comment -

          I have released version 1.4 of this plugin to fix the out of memory error I have experianced - it should be available in the next hour or so.

          Based on your description where the upload is taking a long time (> 5 minutes) for moderately sized files (approx 50 MB) I think the issue maybe either networking or on the ProGet end so I have my doubts that this will fix your issue but definately worth a go.

          On my system using a 385 MB file it took around 1 minute to pack the file and another minute to upload it.

          Andrew Sumner added a comment - I have released version 1.4 of this plugin to fix the out of memory error I have experianced - it should be available in the next hour or so. Based on your description where the upload is taking a long time (> 5 minutes) for moderately sized files (approx 50 MB) I think the issue maybe either networking or on the ProGet end so I have my doubts that this will fix your issue but definately worth a go. On my system using a 385 MB file it took around 1 minute to pack the file and another minute to upload it.

          Amila Gunathilake added a comment - - edited

          hi Mr. andrewsumner thanks for the update I have installed new inedo proget 1.4 version but still 504 error I'm getting.  I'm actually in the middle of setting up upack cli tool in my powershell scripts. I will get back to you with more updates in next week. 

          Thanks
          Amila

          Amila Gunathilake added a comment - - edited hi Mr. andrewsumner thanks for the update I have installed new inedo proget 1.4 version but still 504 error I'm getting.  I'm actually in the middle of setting up upack cli tool in my powershell scripts. I will get back to you with more updates in next week.  Thanks Amila

          Amila Gunathilake added a comment - - edited

          Hi Mr Andrew Sumner,
          Actually my customer requirement is changed now. Now my customer wants to package and publish my artifact .7z file as a .nupkg file not as a .unpack extension file in to the Proget site.
          So it will be a Nuget package and not a Universal package. For this requirement can I still use the Inedo-Proget plugin? Because in default Inedo plugin create a .upack file no?

           

          Thanks 

          Amila

           

          Amila Gunathilake added a comment - - edited Hi Mr Andrew Sumner, Actually my customer requirement is changed now. Now my customer wants to package and publish my artifact .7z file as a .nupkg file not as a .unpack extension file in to the Proget site. So it will be a Nuget package and not a Universal package. For this requirement can I still use the Inedo-Proget plugin? Because in default Inedo plugin create a .upack file no?   Thanks  Amila  

          Hi andrewsumner is there any update sir ? 

          Thank you
          Amila

          Amila Gunathilake added a comment - Hi andrewsumner is there any update sir ?  Thank you Amila

          Andrew Sumner added a comment -

          Sorry, have been busy moving house so haven't had access.  This plugin was specifically designed for universal packages and there are no plans at present to support nuget (however you're welcome to raise an issue with Inedo and if approved I would work on it). 

          A quick google found this article which might give you some guidance.

          Andrew Sumner added a comment - Sorry, have been busy moving house so haven't had access.  This plugin was specifically designed for universal packages and there are no plans at present to support nuget (however you're welcome to raise an issue with Inedo and if approved I would work on it).  A quick google found this article which might give you some guidance.

          Amila Gunathilake added a comment - - edited

          hi andrewsumner, just forget previous comments, below is my final requirement and what I need now regarding this issue: 

          • I had use upack cli command line tool for package my artifacts as a universal package .upack file.  But when I'm uploading upack file into proget site by using the upack "push" command its still giving the 504 error and stopping the Jenkinspipeline. Any idea how to skip this 504 error from the Jenkinsfile? 
          • When I'm using upack "push"  command in my powershell scripts its skipping the 504 error and keep building the project. But when I'm using upack "push" command in-side Jenkinsfile only it's stopping the build.  So now I want to know is how to continue building the job with this 504 error in Jenkinsfile?  because I prefer to use upack "push" command inside my Jenkinsfile rather than inside powershell script.

          Any ideas andrewsumner?

          Thanks,
          Jenkins Customer South Asia

          Amila

          Amila Gunathilake added a comment - - edited hi andrewsumner , just forget previous comments, below is my final requirement and what I need now regarding this issue:  I had use upack cli command line tool for package my artifacts as a universal package .upack file.  But when I'm uploading upack file into proget site by using the upack "push" command its still giving the 504 error and stopping the Jenkinspipeline. Any idea how to skip this 504 error from the Jenkinsfile?  When I'm using upack "push"  command in my powershell scripts its skipping the 504 error and keep building the project. But when I'm using upack "push" command in-side Jenkinsfile only it's stopping the build.  So now I want to know is how to continue building the job with this 504 error in Jenkinsfile?  because I prefer to use upack "push" command inside my Jenkinsfile rather than inside powershell script. Any ideas andrewsumner ? Thanks, Jenkins Customer South Asia Amila

          Andrew Sumner added a comment -

          First of all, I am quite concerned that you're getting the Gateway Timeout error (good news for my plugin as it proves that it's not faulty). Please raise a support request with Inedo to get to the bottom of that at https://my.inedo.com/tickets/new.

          To ignore the error and continue you could use a try-catch block (see https://jenkins.io/doc/book/pipeline/syntax/). This is not something I would recommend except as a temporary measure while you're getting to the bottom of what is causing the gateway timeout. Even then I would investigate to see if you can catch only the 504 Gateway Timeout error.

           

           

           

          Andrew Sumner added a comment - First of all, I am quite concerned that you're getting the Gateway Timeout error (good news for my plugin as it proves that it's not faulty). Please raise a support request with Inedo to get to the bottom of that at  https://my.inedo.com/tickets/new . To ignore the error and continue you could use a try-catch block (see https://jenkins.io/doc/book/pipeline/syntax/ ). This is not something I would recommend except as a temporary measure while you're getting to the bottom of what is causing the gateway timeout. Even then I would investigate to see if you can catch only the 504 Gateway Timeout error.      

            andrewsumner Andrew Sumner
            amila_devops Amila Gunathilake
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: