[JENKINS-15222] FTP Publisher broken for concurrent builds

Type: Bug
Resolution: Unresolved
Priority: Critical
Component/s: ftppublisher-plugin
Labels:
None
Environment:
Linux, Jenkins 1.482

Similar Issues:
Powered by SuggestiMate

Show

Job confinuration
[x] "Execute concurrent builds if necessary"
Restrict where this project can be run: MyNode
Post Build: Send build artifacts over FTP "*.tar.bz2"

The "Send build artifacts over FTP" postbuild-step appears to be broken with concurrent builds. It appears to waits until all concurrent builds are done before any FTP upload happens. However, the workspace can be overwritten by another concurrent build, so the wrong data gets uploaded.
See logs, below, we end up with "JOB2/JOB3.tar.bz2" on the FTP server
As you can see from the logs, the postbuild steps in general appears to be ok, as the Groovy postbuild step happens in time, just the FtpPublisher is waiting for every concurrent build to finish before continuing

Job 1: takes very long
16:32:49 Building remotely on Machine1 in workspace "Test"
16:32:49 this job takes very long
16:43:31 done creaated JOB1.tar.bz2
16:43:31 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step: JOB1/*.tar.bz2
16:43:31 Finished: SUCCESS

Job 2: fast job on 2nd workspace
16:32:57 Building remotely on Machine1 in workspace "Test@2"
16:32:57 fast job
16:33:04 done creaated JOB2.tar.bz2
16:33:04 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step: JOB2/*.tar.bz2
16:43:31 Finished: SUCCESS

Job 3 another fast job on the 2nd workspace
16:33:05 Building remotely on Machine1 in workspace "Test@2"
16:33:05 fast job on the
16:33:09 done, created JOB3.tar.bz2
16:33:10 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step JOB3/*.tar.bz2
16:43:32 Finished: SUCCESS

On the FTP server these files end up
JOB1/JOB1.tar.bz2
JOB2/JOB3.tar.bz2
JOB3/JOB3.tar.bz2

Axel Heider created issue - 2012-09-18 15:51

Axel Heider added a comment - 2012-09-18 16:11

Seeing a similar behavior with the "Publish Over FTP Plugin" and the "FTP-Publisher Plugin".

Axel Heider added a comment - 2012-09-18 16:11 Seeing a similar behavior with the "Publish Over FTP Plugin" and the "FTP-Publisher Plugin".

Axel Heider made changes - 2012-09-18 16:13

Component/s

New: publish-over-ftp [ 15791 ]

Axel Heider added a comment - 2012-09-18 19:19

Update to latest Jenkins 1.482 and using the setup from ~~JENKINS-14283~~ shows the same problem.

Axel Heider added a comment - 2012-09-18 19:19 Update to latest Jenkins 1.482 and using the setup from JENKINS-14283 shows the same problem.

Axel Heider made changes - 2012-09-18 19:20

Affects Version/s		New: current [ 10162 ]
Environment	Original: Linux, jenkins 1.461	New: Linux, Jenkins 1.482
Priority	Original: Blocker [ 1 ]	New: Critical [ 2 ]
Summary	Original: Concurrent Build Workspaces overlap	New: FTP Publisher broken for concurrent builds

Axel Heider made changes - 2012-09-18 19:23

Description

Original: Job confinuration
  [x] "Execute concurrent builds if necessary"
  Restrict where this project can be run: MyNode
  Post Build: Send build artifacts over FTP "*.tar.bz2"

The "Send build artifacts over FTP" postbuild-step appears to be broken with concurrent builds. It appears to waits until all concurrent builds are done before any FTP upload happens. However, wthe workspace can be overwritten by another concurrent build, so the wrong data gets uploaded.
See logs, below, we end up with "JOB2/JOB3.tar.bz2" on the FTP server
As you can see from the logs, the postbuild step itself appear to be ok, as the Groovy Postbuild step happens in time, just the FtpPublisher is waiting for every concurrent build

Job 1: takes very long
16:32:49 Building remotely on Machine1 in workspace "Test"
16:32:49 this job takes very long
16:43:31 done creaated JOB1.tar.bz2
16:43:31 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step: JOB1/*.tar.bz2
16:43:31 Finished: SUCCESS

Job 2: fast job on 2nd workspace
16:32:57 Building remotely on Machine1 in workspace "Test@2"
16:32:57 fast job
16:33:04 done creaated JOB2.tar.bz2
16:33:04 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step: JOB2/*.tar.bz2
16:43:31 Finished: SUCCESS

Job 3 another fast job on the 2nd workspace
16:33:05 Building remotely on Machine1 in workspace "Test@2"
16:33:05 fast job on the
16:33:09 done, created JOB3.tar.bz2
16:33:10 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step JOB3/*.tar.bz2
16:43:32 Finished: SUCCESS

On the FTP server these files end up
  JOB1/JOB1.tar.bz2
  JOB2/JOB3.tar.bz2
  JOB3/JOB3.tar.bz2

New: Job confinuration
  [x] "Execute concurrent builds if necessary"
  Restrict where this project can be run: MyNode
  Post Build: Send build artifacts over FTP "*.tar.bz2"

The "Send build artifacts over FTP" postbuild-step appears to be broken with concurrent builds. It appears to waits until all concurrent builds are done before any FTP upload happens. However, the workspace can be overwritten by another concurrent build, so the wrong data gets uploaded.
See logs, below, we end up with "JOB2/JOB3.tar.bz2" on the FTP server
As you can see from the logs, the postbuild steps in general appears to be ok, as the Groovy postbuild step happens in time, just the FtpPublisher is waiting for every concurrent build to finish before continuing

Job 1: takes very long
16:32:49 Building remotely on Machine1 in workspace "Test"
16:32:49 this job takes very long
16:43:31 done creaated JOB1.tar.bz2
16:43:31 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step: JOB1/*.tar.bz2
16:43:31 Finished: SUCCESS

Job 2: fast job on 2nd workspace
16:32:57 Building remotely on Machine1 in workspace "Test@2"
16:32:57 fast job
16:33:04 done creaated JOB2.tar.bz2
16:33:04 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step: JOB2/*.tar.bz2
16:43:31 Finished: SUCCESS

Job 3 another fast job on the 2nd workspace
16:33:05 Building remotely on Machine1 in workspace "Test@2"
16:33:05 fast job on the
16:33:09 done, created JOB3.tar.bz2
16:33:10 Groovy Postbuild step
16:43:31 Send build artifacts over FTP Step JOB3/*.tar.bz2
16:43:32 Finished: SUCCESS

On the FTP server these files end up
  JOB1/JOB1.tar.bz2
  JOB2/JOB3.tar.bz2
  JOB3/JOB3.tar.bz2

Axel Heider added a comment - 2012-09-19 11:52

My guess would be that this is going on internally:

Job1 starts and acquires Jenkins ressources internally. The FTP publisher seems to be such ressource.
Further jobs have to wait for Job1 to release the ressources

So the solution should be that the ressource FTP publisher should be acquired when actually needed only.

Axel Heider added a comment - 2012-09-19 11:52 My guess would be that this is going on internally: Job1 starts and acquires Jenkins ressources internally. The FTP publisher seems to be such ressource. Further jobs have to wait for Job1 to release the ressources So the solution should be that the ressource FTP publisher should be acquired when actually needed only.

R. Tyler Croy made changes - 2016-07-25 23:50

Workflow

Original: JNJira [ 145944 ]

New: JNJira + In-Review [ 176487 ]

Alex Earl made changes - 2017-12-22 14:00

Component/s

Original: publish-over-ftp-plugin [ 15791 ]

Assignee:: benjaminjaton

Reporter:: Axel Heider

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Created:: 2012-09-18 15:51

Updated:: 2017-12-22 14:00

Jenkins

Details

Description

Attachments

Activity

Collapse comment: Axel Heider added a comment - 2012-09-18 16:11

Expand comment: Axel Heider added a comment - 2012-09-18 16:11

Collapse comment: Axel Heider added a comment - 2012-09-18 19:19

Expand comment: Axel Heider added a comment - 2012-09-18 19:19

Collapse comment: Axel Heider added a comment - 2012-09-19 11:52

Expand comment: Axel Heider added a comment - 2012-09-19 11:52

People

Dates