If using SCM sync to store configuration at github, then I need to put private key in jenkins homefolder (~jenkins/.ssh/) and add the public key to github project as a builder key.
Problem is that this system key overrides keys set using ssh-agent / SSH credentials plugin and makes all other repositories at github fail with a no access error since they using the SCM sync key and not the custom for the build. I have not found a way to make SSH ignore the default user key.