Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-73639

Release API plugin(s) for AWS SDK for Java 2.x

    • 2.29.21-5.v31a_f7d69ea_f9

        [JENKINS-73639] Release API plugin(s) for AWS SDK for Java 2.x

        Matthew Webber added a comment - - edited

        Note the following announcements from AWS:

        The AWS SDK for Java 1.x is in maintenance mode, effective July 31, 2024

        Preview Release of the Migration Tool for the AWS SDK for Java 2.x

        AWS recommend that users migrate to AWS SDK for Java 2.x.

        This ticket is required so that plugins that use the AWS SDK can migrate.

        Matthew Webber added a comment - - edited Note the following announcements from AWS: The AWS SDK for Java 1.x is in maintenance mode, effective July 31, 2024 Preview Release of the Migration Tool for the AWS SDK for Java 2.x AWS recommend that users migrate to AWS SDK for Java 2.x. This ticket is required so that plugins that use the AWS SDK can migrate.

        Mark Waite added a comment -

        Thanks for those links mwebber. I've placed those links in the descriptions.

        Mark Waite added a comment - Thanks for those links mwebber . I've placed those links in the descriptions.

        Mushkoor added a comment - - edited

        Any timeline finalized to move Jenkins ecosystem and plugins to work with AWS SDK v2?

        Mushkoor added a comment - - edited Any timeline finalized to move Jenkins ecosystem and plugins to work with AWS SDK v2?

        Mark Waite added a comment - - edited

        Any timeline finalized to move Jenkins ecosystem and plugins to work with AWS SDK v2?

        No timeline other than that we need to finish the migration by 31 Dec 2025.

        Mark Waite added a comment - - edited Any timeline finalized to move Jenkins ecosystem and plugins to work with AWS SDK v2? No timeline other than that we need to finish the migration by 31 Dec 2025.

        Laszlo Gaal added a comment -

        With the latest plugin update the current AWS SDK for Java plugin now logs the following exception when invoked from the ec2 plugin:

        Nov 04, 2024 4:00:18 PM WARNING com.amazonaws.util.VersionInfoUtils printDeprecationAnnouncement
        The AWS SDK for Java 1.x entered maintenance mode starting July 31, 2024 and will reach end of support on December 31, 2025. For more information, see https://aws.amazon.com/blogs/developer/the-aws-sdk-for-java-1-x-is-in-maintenance-mode-effective-july-31-2024/
        You can print where on the file system the AWS SDK for Java 1.x core runtime is located by setting the AWS_JAVA_V1_PRINT_LOCATION environment variable or aws.java.v1.printLocation system property to 'true'.
        This message can be disabled by setting the AWS_JAVA_V1_DISABLE_DEPRECATION_ANNOUNCEMENT environment variable or aws.java.v1.disableDeprecationAnnouncement system property to 'true'.
        The AWS SDK for Java 1.x is being used here:
        at java.base/java.lang.Thread.getStackTrace(Thread.java:1602)
        at PluginClassLoader for aws-java-sdk-minimal//com.amazonaws.util.VersionInfoUtils.printDeprecationAnnouncement(VersionInfoUtils.java:81)
        at PluginClassLoader for aws-java-sdk-minimal//com.amazonaws.util.VersionInfoUtils.<clinit>(VersionInfoUtils.java:59)
        at PluginClassLoader for aws-java-sdk-minimal//com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:95)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.createClientConfiguration(EC2Cloud.java:1006)
        at PluginClassLoader for ec2//hudson.plugins.ec2.util.AmazonEC2FactoryImpl.connect(AmazonEC2FactoryImpl.java:17)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.reconnectToEc2(EC2Cloud.java:984)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.connect(EC2Cloud.java:998)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:511)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.getPossibleNewSlavesCount(EC2Cloud.java:694)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.getNewOrExistingAvailableSlave(EC2Cloud.java:712)
        at PluginClassLoader for ec2//hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:752)
        at hudson.slaves.Cloud.provision(Cloud.java:227)
        at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:726)
        at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:325)
        at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:823)
        at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:92)
        at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

        looking at the ChangeLog of the AWS SDK project it is clear that the v1 SDK stopped being updated for new features. This means that e.g. the new Graviton 4 instance types announced back in August 2024 (c8g, m8g, r8g, x8g) are not exposed from the v1 SDK, so the ec2 plugin won't be able to offer them as build hosts. This is regrettable.

        Laszlo Gaal added a comment - With the latest plugin update the current AWS SDK for Java plugin now logs the following exception when invoked from the ec2 plugin: Nov 04, 2024 4:00:18 PM WARNING com.amazonaws.util.VersionInfoUtils printDeprecationAnnouncement The AWS SDK for Java 1.x entered maintenance mode starting July 31, 2024 and will reach end of support on December 31, 2025. For more information, see https: //aws.amazon.com/blogs/developer/the-aws-sdk- for -java-1-x-is-in-maintenance-mode-effective-july-31-2024/ You can print where on the file system the AWS SDK for Java 1.x core runtime is located by setting the AWS_JAVA_V1_PRINT_LOCATION environment variable or aws.java.v1.printLocation system property to ' true ' . This message can be disabled by setting the AWS_JAVA_V1_DISABLE_DEPRECATION_ANNOUNCEMENT environment variable or aws.java.v1.disableDeprecationAnnouncement system property to ' true ' . The AWS SDK for Java 1.x is being used here: at java.base/java.lang. Thread .getStackTrace( Thread .java:1602) at PluginClassLoader for aws-java-sdk-minimal //com.amazonaws.util.VersionInfoUtils.printDeprecationAnnouncement(VersionInfoUtils.java:81) at PluginClassLoader for aws-java-sdk-minimal //com.amazonaws.util.VersionInfoUtils.<clinit>(VersionInfoUtils.java:59) at PluginClassLoader for aws-java-sdk-minimal //com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:95) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.createClientConfiguration(EC2Cloud.java:1006) at PluginClassLoader for ec2 //hudson.plugins.ec2.util.AmazonEC2FactoryImpl.connect(AmazonEC2FactoryImpl.java:17) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.reconnectToEc2(EC2Cloud.java:984) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.connect(EC2Cloud.java:998) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.countCurrentEC2Slaves(EC2Cloud.java:511) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.getPossibleNewSlavesCount(EC2Cloud.java:694) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.getNewOrExistingAvailableSlave(EC2Cloud.java:712) at PluginClassLoader for ec2 //hudson.plugins.ec2.EC2Cloud.provision(EC2Cloud.java:752) at hudson.slaves.Cloud.provision(Cloud.java:227) at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:726) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:325) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:823) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:92) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang. Thread .run( Thread .java:829) looking at the ChangeLog of the AWS SDK project it is clear that the v1 SDK stopped being updated for new features. This means that e.g. the new Graviton 4 instance types announced back in August 2024 (c8g, m8g, r8g, x8g) are not exposed from the v1 SDK, so the ec2 plugin won't be able to offer them as build hosts. This is regrettable.

        Basil Crow added a comment -

        I packaged the new SDK in https://github.com/jenkinsci/aws-java-sdk2-plugin (to be released once the authentication token is recreated). Like v1, I am using the default Apache Commons HTTP Client 4.x, using apache-httpcomponents-client-4-api to provide the JARs. This is the traditional synchronous mode.

        One of the new features in v2 is an asynchronous mode, which I have not attempted to package. The Java version uses Netty, which we would need to package as a separate library plugin like apache-httpcomponents-client-4-api. Amazon also now offers native performance-optimized HTTP clients for both synchronous and asynchronous mode in the AWS Common Runtime (CRT). This embeds native code for several supported architectures:

        linux/armv6/glibc/libaws-crt-jni.so
        linux/armv7/glibc/libaws-crt-jni.so
        linux/armv7/musl/libaws-crt-jni.so
        linux/armv8/glibc/libaws-crt-jni.so
        linux/armv8/musl/libaws-crt-jni.so
        linux/x86_32/glibc/libaws-crt-jni.so
        linux/x86_64/glibc/libaws-crt-jni.so
        linux/x86_64/musl/libaws-crt-jni.so
        osx/armv8/cruntime/libaws-crt-jni.dylib
        osx/x86_64/cruntime/libaws-crt-jni.dylib
        windows/x86_32/cruntime/aws-crt-jni.dll
        windows/x86_64/cruntime/aws-crt-jni.dll
        

        In my limited testing of the CRT client in asynchronous mode in Jenkins, I ran into some classloading problems (which could be worked around with SetContextClassLoader). I also noticed that these binaries are quite large (18 MiB). Since we don't need asynchronous mode today, I simply dispensed with packaging CRT. But if there is some desire to write asynchronous AWS code in the future, I believe that either the Netty-based or CRT-based asynchronous runtime could be packaged with a little more effort.

        I did only the most light testing (listing S3 buckets and restarting an EC2 VM) in the context of a toy Jenkins plugin, and the synchronous client based on Commons HttpClient 4.x worked just fine without any class loading issues. More advanced scenarios might need further iteration in the packaging, but this seems like as good a place as any to start.

        Basil Crow added a comment - I packaged the new SDK in https://github.com/jenkinsci/aws-java-sdk2-plugin (to be released once the authentication token is recreated). Like v1, I am using the default Apache Commons HTTP Client 4.x, using apache-httpcomponents-client-4-api to provide the JARs. This is the traditional synchronous mode. One of the new features in v2 is an asynchronous mode, which I have not attempted to package. The Java version uses Netty, which we would need to package as a separate library plugin like apache-httpcomponents-client-4-api . Amazon also now offers native performance-optimized HTTP clients for both synchronous and asynchronous mode in the AWS Common Runtime (CRT). This embeds native code for several supported architectures: linux/armv6/glibc/libaws-crt-jni.so linux/armv7/glibc/libaws-crt-jni.so linux/armv7/musl/libaws-crt-jni.so linux/armv8/glibc/libaws-crt-jni.so linux/armv8/musl/libaws-crt-jni.so linux/x86_32/glibc/libaws-crt-jni.so linux/x86_64/glibc/libaws-crt-jni.so linux/x86_64/musl/libaws-crt-jni.so osx/armv8/cruntime/libaws-crt-jni.dylib osx/x86_64/cruntime/libaws-crt-jni.dylib windows/x86_32/cruntime/aws-crt-jni.dll windows/x86_64/cruntime/aws-crt-jni.dll In my limited testing of the CRT client in asynchronous mode in Jenkins, I ran into some classloading problems (which could be worked around with SetContextClassLoader ). I also noticed that these binaries are quite large (18 MiB). Since we don't need asynchronous mode today, I simply dispensed with packaging CRT. But if there is some desire to write asynchronous AWS code in the future, I believe that either the Netty-based or CRT-based asynchronous runtime could be packaged with a little more effort. I did only the most light testing (listing S3 buckets and restarting an EC2 VM) in the context of a toy Jenkins plugin, and the synchronous client based on Commons HttpClient 4.x worked just fine without any class loading issues. More advanced scenarios might need further iteration in the packaging, but this seems like as good a place as any to start.

        Basil Crow added a comment -

        Basil Crow added a comment - Initial release: https://github.com/jenkinsci/aws-java-sdk2-plugin/releases/tag/2.29.21-5.v31a_f7d69ea_f9

          basil Basil Crow
          markewaite Mark Waite
          Votes:
          0 Vote for this issue
          Watchers:
          11 Start watching this issue

            Created:
            Updated:
            Resolved: