Some ideas that may help as you explore the failure.

The failing console output reports that port 51001 is not reachable, while in the successful case it reports that agent port 52142 is used. That may indicate that Jenkins is configured to use a random port for inbound agents. It might also indicate that Jenkins was configured to open port 51001 but the port is somehow blocked on that computer.

The remoting version on the agent is 3.29. That was released in Jan 2019. Jenkins 2.249.1 includes remoting 4.5, released two months ago. There may be an incompatibility between the port negotiation being done by remoting 3.29 and Jenkins 2.249.1. You could try upgrading the agent.jar file from the older version you're running to see if that helps.

The Java version 8u121 was released in 2017. Oracle expired that release with the April 2017 release of the next Oracle critical patch update. The current AdoptOpenJDK release is 8u262.

So, I updated the agent.jar and got same result.

I added in two remoting logs .... look at them.  I see a difference, the new version of Jenkins is the remoting.log.4 and the old version is remoting.log.0.

The difference is 

New : INFO: Locating server among https://jenkins.cdl.af.mil/

Old: INFO: Locating server among https://jenkins.cdl.af.mil/, http://10.101.0.203:8090/

it has two URL's and the second is the one that connects.  We need to connect via IP based on location of the worker server and not come in the front door based on security.

INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Sep 12, 2020 12:10:57 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver isPortVisible
WARNING: connect timed out
Sep 12, 2020 12:10:57 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Sep 12, 2020 12:10:57 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: 10.101.0.203
Agent port: 61926
Identity: 40:97:48:b3:eb:09:11:33:91:c9:18:ab:c5:f2:91:25 

Notice it fails on first URL but success on second.  The new version only has the one URL and it just fails like old but does not have second URL to try.

So is there anything we can do?  Or does it need a Jenkins change?

I think that the comment from jglick in JENKINS-63014 hints that there may be a different set of arguments you can pass to agent.jar that will cause it to connect to the 10.101.0.203 address and port.

You might try configuring your Jenkins server to use a fixed port for inbound agents ("Manage Jenkins" -> "Configure Global Security"), then use that fixed port in the agent.jar command line.

The port is not the issue.  I need for the URL to be the IP 10.101.0.203 ..... that's whats in the agent config.  How else can I tell it to use the correct URL?

I don't know what Jesse Glick is referring to "you can still do that with TCP agents, using the lower-level and more explicit launch mode" .... any insight?

I think he is referring to the command line argument listed by (with apologies that I deleted most of the output so that the specific options were more visible)

mwaite@mark-pc2:~/bin$ java -jar agent-4.5.jar -help
 -auth user:pass                 : If your Jenkins is security-enabled, specify
                                   a valid user name and password.
 -connectTo HOST:PORT            : make a TCP connection to the given host and
                                   port, then start communication.
 -secret HEX_SECRET              : Agent connection secret to use instead of
                                   -jnlpCredentials.

So it looks like this worked .... in the config for the agent, I added the following under the advanced 

That's great charbl2007. I'll include that information in the Jenkins 2.249.1 upgrade guide.

Documented in the Jenkins 2.249.1 upgrade guide.

Thanks for your help understanding the issue charbl2007!

I think we have the same problem reported in JENKINS-63671

Resolved by an entry in the Jenkins 2.249.1 upgrade guide describing the change and how to adapt to the change.