Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-13128

Artifacts Permissions Stripped

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: core
    • Labels:
      None
    • Environment:
      Centos 5.6, 32-bit, JDK 7u3, Jenkins 1.455
    • Similar Issues:

      Description

      I've seen several related and/or resolved issue on this but I wasn't sure which one was relevant for re-opening.

      I have a very basic job (name DEV) that polls a git repository, deploys the files to a local webserver via a shell script, then archives the workspace. A second job then picks up the last successful archive w/ the Copy Artifact plugin and attempts to run the same deploy script and receives an error due to missing executable permissions. The intent is to put a skeleton together of a pipeline for DEV->QA->PROD, each deploying to a local folder (symlinked to a directory apache is aware of), the file permissions are deployment scripts are managed in the repository with the source for the site.

      I initially thought this was an issue with the Copy Artifact plugin, but after writing a basic shell script to manually copy the files and then looking directly at the archive folders in the job folders, it appears to be the Archive operation itself.

      It appears that all the file and folder permissions are being set/cleared when the archive occurred. The contents of the DEV workspace has original permissions after a run, but the archive contents (jenkins\jobs\DEV\builds##\archive) are missing their permissions. All files have been set to 644, all directories to 755 (>2000 files in multiple subdirectories, although to be fair I've only checked 5 or 6 subdirectories).

      This is similar to the related set of tickets under JENKINS-9397 but I wasn't sure which (if any) was best to reopen.

        Attachments

          Issue Links

            Activity

            tarwn Eli Weinstock-Herman created issue -
            Hide
            evernat evernat added a comment -

            Is it reproduced with a recent Jenkins version?

            Show
            evernat evernat added a comment - Is it reproduced with a recent Jenkins version?
            Hide
            br0ch0n Brandon Rochon added a comment -

            I can confirm that this is still an issue on v1.523

            Show
            br0ch0n Brandon Rochon added a comment - I can confirm that this is still an issue on v1.523
            Hide
            danielbeck Daniel Beck added a comment -

            Would be interesting to see whether this occurs only when building on master, or also when building on a slave node. (A similar issue is known to occur with file modification time stamps and that changes behavior there.)

            Show
            danielbeck Daniel Beck added a comment - Would be interesting to see whether this occurs only when building on master, or also when building on a slave node. (A similar issue is known to occur with file modification time stamps and that changes behavior there.)
            danielbeck Daniel Beck made changes -
            Field Original Value New Value
            Link This issue is related to JENKINS-23645 [ JENKINS-23645 ]
            Hide
            tony7682 Tony Panza added a comment -

            @Daniel This occurs on slave nodes as well. On a slave with Ubuntu 10.04.3 LTS (2.6.32-64-generic-pae #128-Ubuntu SMP Tue Jul 15 08:49:38 UTC 2014 i686 GNU/Linux), Jenkins v1.436.

            Show
            tony7682 Tony Panza added a comment - @Daniel This occurs on slave nodes as well. On a slave with Ubuntu 10.04.3 LTS (2.6.32-64-generic-pae #128-Ubuntu SMP Tue Jul 15 08:49:38 UTC 2014 i686 GNU/Linux), Jenkins v1.436.
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 143591 ] JNJira + In-Review [ 175948 ]
            danielbeck Daniel Beck made changes -
            Link This issue is duplicated by JENKINS-38274 [ JENKINS-38274 ]
            danielbeck Daniel Beck made changes -
            Link This issue is duplicated by JENKINS-14269 [ JENKINS-14269 ]
            aarondmarasco_vsi Aaron D. Marasco made changes -
            Link This issue is related to JENKINS-9397 [ JENKINS-9397 ]
            Hide
            aarondmarasco_vsi Aaron D. Marasco added a comment -

            Pushing six years now... I use a workaround where I use "find" to find any executable flagged scripts and then store off their names into a file. On the downstream jobs I then use that list to set executable flags where needed.

            Show
            aarondmarasco_vsi Aaron D. Marasco added a comment - Pushing six years now... I use a workaround where I use "find" to find any executable flagged scripts and then store off their names into a file. On the downstream jobs I then use that list to set executable flags where needed.
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is duplicated by JENKINS-48966 [ JENKINS-48966 ]
            Hide
            ovidiub13 Ovidiu-Florin Bogdan added a comment -

            Jenkins 2.89.4 here and this issue is present. Workarounds are to archive the artifact, and then extract it, or to reset the permissions after fetching it.

            Show
            ovidiub13 Ovidiu-Florin Bogdan added a comment - Jenkins 2.89.4 here and this issue is present. Workarounds are to archive the artifact, and then extract it, or to reset the permissions after fetching it.
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Unfinished pull request is here: https://github.com/jenkinsci/jenkins/pull/1576 . Everybody is welcome to take it and finalize it

            Show
            oleg_nenashev Oleg Nenashev added a comment - Unfinished pull request is here: https://github.com/jenkinsci/jenkins/pull/1576 . Everybody is welcome to take it and finalize it
            oleg_nenashev Oleg Nenashev made changes -
            Remote Link This issue links to "https://github.com/jenkinsci/jenkins/pull/1576 (Web Link)" [ 20216 ]
            Hide
            ovidiub13 Ovidiu-Florin Bogdan added a comment -

            Oleg Nenashev I see that it's stuck in a merge conflict. Even if that conflict would be solved, will that ever get merged in?

            IMO the proper solution is to simply NOT STRIP file properties, and just keep them as they are. It should assume that the developer who made the job knows what they are doing, and expects things to happen as he set them.

            Show
            ovidiub13 Ovidiu-Florin Bogdan added a comment - Oleg Nenashev I see that it's stuck in a merge conflict. Even if that conflict would be solved, will that ever get merged in? IMO the proper solution is to simply NOT STRIP file properties, and just keep them as they are. It should assume that the developer who made the job knows what they are doing, and expects things to happen as he set them.
            Hide
            danielbeck Daniel Beck added a comment -

            Even if that conflict would be solved, will that ever get merged in?

            I'm with Oliver (see comments there) on the proposed change, there seems to be no real use case to not preserve attributes, so the options seem unnecessary. Should be straightforward to rip that part out of the PR, resolve the conflicts, and resubmit for review.

            Show
            danielbeck Daniel Beck added a comment - Even if that conflict would be solved, will that ever get merged in? I'm with Oliver (see comments there) on the proposed change, there seems to be no real use case to not preserve attributes, so the options seem unnecessary. Should be straightforward to rip that part out of the PR, resolve the conflicts, and resubmit for review.
            Hide
            ovidiub13 Ovidiu-Florin Bogdan added a comment -

            If the patch in question is done in the right place, then it seems to me that we are just using a function from ANT to copy the file. Isn't that the function that strips the permissions?

            I don't see other places that could mess with the file properties.

            Show
            ovidiub13 Ovidiu-Florin Bogdan added a comment - If the patch in question is done in the right place, then it seems to me that we are just using a function from ANT to copy the file. Isn't that the function that strips the permissions? I don't see other places that could mess with the file properties.
            jvz Matt Sicker made changes -
            Assignee Matt Sicker [ jvz ]
            Hide
            jvz Matt Sicker added a comment -

            The simple approach to fixing this is to preserve posix file permissions by default along with mtimes. A umask option could also be added to allow for reverting to the old behavior (umask of 133). Whether or not attempting to preserve file and group ownership is worthwhile may be another story.

            Show
            jvz Matt Sicker added a comment - The simple approach to fixing this is to preserve posix file permissions by default along with mtimes. A umask option could also be added to allow for reverting to the old behavior (umask of 133). Whether or not attempting to preserve file and group ownership is worthwhile may be another story.
            Hide
            jvz Matt Sicker added a comment -

            I should be more specific about the umask. The way file creation works by default generally means working with a directory umask of 022 and "file umask" (not really a thing) of 133. The difference, of course, is because an executable directory means that it can be listed, not executed.

            Show
            jvz Matt Sicker added a comment - I should be more specific about the umask. The way file creation works by default generally means working with a directory umask of 022 and "file umask" (not really a thing) of 133. The difference, of course, is because an executable directory means that it can be listed, not executed.
            Hide
            jvz Matt Sicker added a comment -

            Also important to note: this bug seems to only affect local->local file copying. Any local->remote or remote->local file copying happens via tar archives which preserve file permissions and are already handled as such.

            Show
            jvz Matt Sicker added a comment - Also important to note: this bug seems to only affect local->local file copying. Any local->remote or remote->local file copying happens via tar archives which preserve file permissions and are already handled as such.
            jvz Matt Sicker made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            jvz Matt Sicker added a comment -

            Added a PR.

            Show
            jvz Matt Sicker added a comment - Added a PR.
            jvz Matt Sicker made changes -
            Remote Link This issue links to "Pull request (Web Link)" [ 20479 ]
            jvz Matt Sicker made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jamesdumay James Dumay made changes -
            Remote Link This issue links to "CloudBees Internal OSS-2688 (Web Link)" [ 20533 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Matt Sicker
            Path:
            core/src/main/java/hudson/FilePath.java
            core/src/main/java/hudson/model/ItemGroupMixIn.java
            core/src/test/java/hudson/FilePathTest.java
            http://jenkins-ci.org/commit/jenkins/e229f37dd921812e213e34913c94b632c6e54299
            Log:
            JENKINS-13128: Preserve copied file permissions and mtime (#3400)

            This fixes a bug where files copied locally do not preserve file permissions or last modification time.

            • Use IO utility methods for exception handling
            • Fix invalid path exception propagation
            • Revert hudson.Util

            *NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

            Functionality will be removed from GitHub.com on January 31st, 2019.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Matt Sicker Path: core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/model/ItemGroupMixIn.java core/src/test/java/hudson/FilePathTest.java http://jenkins-ci.org/commit/jenkins/e229f37dd921812e213e34913c94b632c6e54299 Log: JENKINS-13128 : Preserve copied file permissions and mtime (#3400) JENKINS-13128 : Preserve copied file permissions and mtime This fixes a bug where files copied locally do not preserve file permissions or last modification time. Use IO utility methods for exception handling Fix invalid path exception propagation Revert hudson.Util * NOTE: * This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.
            Hide
            dnusbaum Devin Nusbaum added a comment -

            Fixed in Jenkins 2.120.

            Show
            dnusbaum Devin Nusbaum added a comment - Fixed in Jenkins 2.120 .
            dnusbaum Devin Nusbaum made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]
            zbynek Zbynek Konecny made changes -
            Link This issue is related to JENKINS-52325 [ JENKINS-52325 ]

              People

              Assignee:
              jvz Matt Sicker
              Reporter:
              tarwn Eli Weinstock-Herman
              Votes:
              7 Vote for this issue
              Watchers:
              12 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: