Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71605

Out-Of-Box Red Hat Install wont start on stock RHEL 8

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • packaging
    • RHEL 8.8 All downloads from Red Hat or from Jenkins, nothing else on system.

      Issue is in Red Hat/.rpm packaging for Jenkins, not Jenkins itself.

      Brand-new, standard RHEL8.8 install.  Followed instructions at https://pkg.jenkins.io/redhat-stable/ then startup via systemd per Handbook.  This is what a newbie does.

      Wont start.

      RH standard Java is Java 8, and installing either java-11-openjdk or java-17-openjdk doesn't change system default "java", so documented systemd startup dies w/ "wrong java".  The smarter code in init.d/jenkins doesn't seem to be involved in this path. Messages saying so are hidden, so an inexperienced installer would be clueless and frustrated.

      Recommendations:

      1. Set JAVA_HOME=/usr/lib/jre-11 in the distributed jenkins.service and sysconfig/jenkins files
      2. Remove java 8 choices from init.d/jenkins, none of them will work.
      3. Add dependencies on "jre-11" and "fontconfig" to the .rpm
      4. Somewhat larger change: Merge the SysV init.d/jenkins and systemd methods so both pull config from a single file (probably sysconfig/jenkins) and set it there.

       

          [JENKINS-71605] Out-Of-Box Red Hat Install wont start on stock RHEL 8

          Mark Waite added a comment - - edited

          Thanks for the suggestions.

          I think we'll be better served by removing those short instructions on https://pkg.jenkins.io/redhat-stable/ and replacing them with a link to the detailed instructions at https://www.jenkins.io/doc/book/installing/linux/#red-hat-centos . The short instructions worked reasonably well when we weren't having to deal with multiple operating systems (Fedora, Red Hat), multiple versions of those operating systems (Fedora 37, Fedora 38, Red Hat Enterprise Linux 8, Red Hat Enterprise Linux 9), and multiple Java versions (Java 11, Java 17).

          We won't switch to read configuration form sysconfig/jenkins, since that is not where systemd reads configuration. We switched in March 2022 to systemd on RPM and DEB installers. In the future, we expect to remove all references to System V initialization from the RPM based installers. A pull request is in progress now to remove those references.

          In the past we've intentionally not included a dependency on a JDK package provided by the operating system. The original reason was due to Java packaging issues on some platforms. Those have been resolved, but I believe we will continue that because the user needs to choose their JDK, whether Java 11 or Java 17 or in the future Java 21.

          Mark Waite added a comment - - edited Thanks for the suggestions. I think we'll be better served by removing those short instructions on https://pkg.jenkins.io/redhat-stable/ and replacing them with a link to the detailed instructions at https://www.jenkins.io/doc/book/installing/linux/#red-hat-centos . The short instructions worked reasonably well when we weren't having to deal with multiple operating systems (Fedora, Red Hat), multiple versions of those operating systems (Fedora 37, Fedora 38, Red Hat Enterprise Linux 8, Red Hat Enterprise Linux 9), and multiple Java versions (Java 11, Java 17). We won't switch to read configuration form sysconfig/jenkins, since that is not where systemd reads configuration. We switched in March 2022 to systemd on RPM and DEB installers. In the future, we expect to remove all references to System V initialization from the RPM based installers. A pull request is in progress now to remove those references. In the past we've intentionally not included a dependency on a JDK package provided by the operating system. The original reason was due to Java packaging issues on some platforms. Those have been resolved, but I believe we will continue that because the user needs to choose their JDK, whether Java 11 or Java 17 or in the future Java 21.

          John "Woody" Woodruff added a comment -

          Thanks for looking at it, Mark.

          All I think we need to do is ensure that, in standard cases, Jenkins comes up out-of-the box w/ no twiddling of config files.  If such twiddling is truly unavoidable, the newbie instructions need to include that.  I also think a modest degree of smart coding to adapt to frequently-encountered environment issues is reasonable.

          I applaud separating SysV from systemd setups. But w/r/t "where systemd reads configurations", the current setup has systemd running a sizable script that has the opportunity to do smart defaults like searching for a JVM if none was specified - but it doesn't (and I'd move it out of /usr/bin, too...)  Finally, keep  systemctl --user (RHEL >=8) in mind - I'm thinking the systemd setup should be under ~jenkins/.config/systemd/user instead of assuming root.

          RHEL comes Out-of-the-Box w/ firewalling enabled.  I see it's already covered in the instructions - so we already acknowledge that it's frequently encountered.  Perhaps we can fit a firewall-cmd --add-port-8080/tcp in somewhere?

          I had to include -Djava.security.disableSystemPropertiesFile=true and -Dcom.redhat.fips=false . I agree that neither should be the default; but perhaps mentioning them in the comments where we set JAVA_OPTS would be appropriate.  Is there a better place in the documentation for "hints to make it work in nonstandard environments"?

          Finally, a closely related, but different, question: Both SysV and Systemd implementations currently pass everything through environment variables, including leaving the SSL cert passphrase visible in 'ps', which IMHO is a huge security hole.  AFAIK. the only good way to avoid that is to write it to a new winstone.properties that can be 600 jenkins.  Should I raise that as a different Jira issue?

          Last question: Did I open this in the right place?  Is there a specific project for Linux or RPM Linux packaging?  I want to use Jira correctly...

          John "Woody" Woodruff added a comment - Thanks for looking at it, Mark. All I think we need to do is ensure that, in standard cases, Jenkins comes up out-of-the box w/ no twiddling of config files.  If such twiddling is truly unavoidable, the newbie instructions need to include that.  I also think a modest degree of smart coding to adapt to frequently-encountered environment issues is reasonable. I applaud separating SysV from systemd setups. But w/r/t "where systemd reads configurations", the current setup has systemd running a sizable script that has the opportunity to do smart defaults like searching for a JVM if none was specified - but it doesn't (and I'd move it out of /usr/bin, too...)  Finally, keep  systemctl --user (RHEL >=8) in mind - I'm thinking the systemd setup should be under ~jenkins/.config/systemd/user instead of assuming root. RHEL comes Out-of-the-Box w/ firewalling enabled.  I see it's already covered in the instructions - so we already acknowledge that it's frequently encountered.  Perhaps we can fit a firewall-cmd --add-port-8080/tcp in somewhere? I had to include -Djava.security.disableSystemPropertiesFile=true  and -Dcom.redhat.fips=false . I agree that neither should be the default; but perhaps mentioning them in the comments where we set JAVA_OPTS would be appropriate.  Is there a better place in the documentation for "hints to make it work in nonstandard environments"? Finally, a closely related, but different, question: Both SysV and Systemd implementations currently pass everything through environment variables, including leaving the SSL cert passphrase visible in 'ps' , which IMHO is a huge security hole.  AFAIK. the only good way to avoid that is to write it to a new winstone.properties that can be 600 jenkins.  Should I raise that as a different Jira issue? Last question: Did I open this in the right place?  Is there a specific project for Linux or RPM Linux packaging?  I want to use Jira correctly...

          Mark Waite added a comment -

          Issues with the RPM and DEB installers are best reported in the packaging repository where the scripts are maintained and enhanced. Packaging changes are implemented there as pull requests. Pull request 409 is an in-progress change that removes the remaining System V init components that were retained during the RPM transition to systemd.

          I had to include -Djava.security.disableSystemPropertiesFile=true and -Dcom.redhat.fips=false . I agree that neither should be the default; but perhaps mentioning them in the comments where we set JAVA_OPTS would be appropriate. Is there a better place in the documentation for "hints to make it work in nonstandard environments"?

          Jenkins does not test with Red Hat Enterprise Linux FIPS configurations. There are several known issues related to attempts to run Jenkins on systems with FIPS enabled. If you'd like to propose automated tests that confirm Jenkins installs on a FIPS mode RHEL, the tests are stored in the packaging repository. Test execution instructions are in the README.

          Mark Waite added a comment - Issues with the RPM and DEB installers are best reported in the packaging repository where the scripts are maintained and enhanced. Packaging changes are implemented there as pull requests. Pull request 409 is an in-progress change that removes the remaining System V init components that were retained during the RPM transition to systemd. I had to include -Djava.security.disableSystemPropertiesFile=true and -Dcom.redhat.fips=false . I agree that neither should be the default; but perhaps mentioning them in the comments where we set JAVA_OPTS would be appropriate. Is there a better place in the documentation for "hints to make it work in nonstandard environments"? Jenkins does not test with Red Hat Enterprise Linux FIPS configurations. There are several known issues related to attempts to run Jenkins on systems with FIPS enabled. If you'd like to propose automated tests that confirm Jenkins installs on a FIPS mode RHEL, the tests are stored in the packaging repository. Test execution instructions are in the README.

            Unassigned Unassigned
            woodrufj John "Woody" Woodruff
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: