Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16332

Leading and trailing underscores are added to commiters email address, unable to send build notification

      Jenkins email-ext plugin fails to send email, the error is

      Failed to create e-mail address for _developer1@xxx.net_
      Failed to create e-mail address for _developer2@xxx.net_
      Failed to create e-mail address for _developer1@xxx.net_
      Failed to create e-mail address for _developer1@xxx.net_

      (Names and domain are changed)

      Jan 13, 2013 7:15:56 PM hudson.plugins.emailext.ExtendedEmailPublisher addAddressesFromRecipientList
      WARNING: Could not create email address.
      javax.mail.internet.AddressException: Domain contains illegal character in string ``_developer1@xxx.net_''
      at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
      at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
      at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
      at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
      at hudson.plugins.emailext.EmailRecipientUtils.convertRecipientString(EmailRecipientUtils.java:60)
      at hudson.plugins.emailext.ExtendedEmailPublisher.addAddressesFromRecipientList(ExtendedEmailPublisher.java:687)
      at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:508)
      at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:297)
      at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:289)
      at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:249)
      at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:810)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perJan 13, 2013 7:15:56 PM hudson.plugins.emailext.ExtendedEmailPublisher addAddressesFromRecipientList
      WARNING: Could not create email address.
      javax.mail.internet.AddressException: Domain contains illegal character in string ``_developer1@xxx.net_''
      at javax.mail.internet.InternetAddress.checkAddress(InternetAddress.java:1269)
      at javax.mail.internet.InternetAddress.parse(InternetAddress.java:1091)
      at javax.mail.internet.InternetAddress.parse(InternetAddress.java:633)
      at javax.mail.internet.InternetAddress.<init>(InternetAddress.java:111)
      at hudson.plugins.emailext.EmailRecipientUtils.convertRecipientString(EmailRecipientUtils.java:60)
      at hudson.plugins.emailext.ExtendedEmailPublisher.addAddressesFromRecipientList(ExtendedEmailPublisher.java:687)
      at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:508)
      at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:297)
      at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:289)
      at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:249)
      at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
      at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:810)
      at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:785)
      at hudson.model.Build$BuildExecution.cleanUp(Build.java:192)
      at hudson.model.Run.execute(Run.java:1587)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:236)formAllBuildSteps(AbstractBuild.java:785)
      at hudson.model.Build$BuildExecution.cleanUp(Build.java:192)
      at hudson.model.Run.execute(Run.java:1587)
      at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      at hudson.model.ResourceController.execute(ResourceController.java:88)
      at hudson.model.Executor.run(Executor.java:236)

      First I thought is the problem is in email-ext plugin, but when I went to "Changes" tab and opened the page

      http://host/jenkins/user/Developer%201%20__developer1@xxx.net_/
      (hostname changed)

      it showed

      Jenkins User Id: Developer 1 _developer1@xxx.net_

      For unknown reason Jenkins adds a leading and trailing underscore to email address

      Environment
      Jenkins ver. 1.498
      Jenkins Mercurial plugin 1.42
      Jenkins Email Extension Plugin 2.25
      Mercurial Distributed SCM (version 1.4.3)
      Ubuntu 10.04.4 LTS
      Java(TM) SE Runtime Environment (build 1.6.0_32-b05)
      Java HotSpot(TM) 64-Bit Server VM (build 20.7-b02, mixed mode)

      We have a build cluster with a master and two slaves, this problem happened on the slave machine.

      When I type hg log on Jenkins job's workspace, then all changesets have
      correct email address without underscores, so it is not Mercurial configuration problem.

          [JENKINS-16332] Leading and trailing underscores are added to commiters email address, unable to send build notification

          Jerry Maloney added a comment -

          I believe there may be an issue in MercurialChangeLogParser method MercurialChangeSetList parse(AbstractBuild build, File changelogFile). If I pass in a changelog that looks like the following:

          changelog.xml
          <?xml version='1.0' encoding='UTF-8'?>
          <changesets>
          <changeset author='joe.schmo &lt;joe.schmo@example.com&gt;'/> <!-- note that those escape characters are on purpose and not some relic of posting in Jira -->
          </changesets>
          

          I get a changeset author with the ID and fullname "joe.schmo _joe.schmo@example.com_", which is the error reported in this ticket. I believe the Digester class from Apache Commons is taking the escaped XML characters in the changesets/changeset/@author attribute and rendering them as underscores.

          I pushed a unit test showing this to my fork of mercurial-plugin, but not requesting a pull because I don't have a fix yet.

          Jerry Maloney added a comment - I believe there may be an issue in MercurialChangeLogParser method MercurialChangeSetList parse(AbstractBuild build, File changelogFile) . If I pass in a changelog that looks like the following: changelog.xml <?xml version= '1.0' encoding= 'UTF-8' ?> <changesets> <changeset author= 'joe.schmo &lt;joe.schmo@example.com&gt;' /> <!-- note that those escape characters are on purpose and not some relic of posting in Jira --> </changesets> I get a changeset author with the ID and fullname "joe.schmo _joe.schmo@example.com_", which is the error reported in this ticket. I believe the Digester class from Apache Commons is taking the escaped XML characters in the changesets/changeset/@author attribute and rendering them as underscores. I pushed a unit test showing this to my fork of mercurial-plugin , but not requesting a pull because I don't have a fix yet.

          Jesse Glick added a comment -

          @peterbecker wrote:

          In one build two out of three email addresses failed. They all look the same in hg, the one big different is that the two failing ones belong to users with Jenkins accounts, the one that works belongs to a user that never created a Jenkins account.

          which I suspect is important, though I still do not know how to reproduce this.

          Jesse Glick added a comment - @peterbecker wrote: In one build two out of three email addresses failed. They all look the same in hg, the one big different is that the two failing ones belong to users with Jenkins accounts, the one that works belongs to a user that never created a Jenkins account. which I suspect is important, though I still do not know how to reproduce this.

          cowwoc added a comment -

          Jesse,

          If you provide me with a patched plugin with extra logging I will help you figure out what is going on.

          cowwoc added a comment - Jesse, If you provide me with a patched plugin with extra logging I will help you figure out what is going on.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/test/java/hudson/plugins/mercurial/MercurialChangeLogParserTest.java
          http://jenkins-ci.org/commit/mercurial-plugin/319048bbd4de54f8badb19ff57d6475df6b193b1
          Log:
          JENKINS-16332 Demonstrating that MercurialChangeLogParser is not at fault.
          https://github.com/jerrymaloney/mercurial-plugin/commit/037b5b1ac5a3eac18dceb58fa509973d8de40cad
          was misleading because it checked getId, whereas getFullName is what is supposed to keep the email address.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/test/java/hudson/plugins/mercurial/MercurialChangeLogParserTest.java http://jenkins-ci.org/commit/mercurial-plugin/319048bbd4de54f8badb19ff57d6475df6b193b1 Log: JENKINS-16332 Demonstrating that MercurialChangeLogParser is not at fault. https://github.com/jerrymaloney/mercurial-plugin/commit/037b5b1ac5a3eac18dceb58fa509973d8de40cad was misleading because it checked getId, whereas getFullName is what is supposed to keep the email address.

          Jerry Maloney added a comment -

          On https://github.com/jenkinsci/mercurial-plugin/commit/319048bbd4de54f8badb19ff57d6475df6b193b1 Jesse said:
          > getFullName is what is supposed to keep the email address.

          It's been a while since I looked at this, but if I recall there are instances where the results of getId are returned to the emailer. From what you say above, this should never happen, so if it is happening that could be the problem. If I get time today I'll try to recreate the situation that leads me to believe this.

          Jerry Maloney added a comment - On https://github.com/jenkinsci/mercurial-plugin/commit/319048bbd4de54f8badb19ff57d6475df6b193b1 Jesse said: > getFullName is what is supposed to keep the email address. It's been a while since I looked at this, but if I recall there are instances where the results of getId are returned to the emailer. From what you say above, this should never happen, so if it is happening that could be the problem. If I get time today I'll try to recreate the situation that leads me to believe this.

          cowwoc added a comment - - edited

          I just discovered something important. The auto-created user (from commit logs) has an email address with underscores (same as the user's full name). This is probably why hudson.tasks.MailAddressResolver.resolve(u) returns the same value as the full name.

          Who is creating these users from commit logs? Which part of the code is populating the email address for those users? That's where we should be looking!

          cowwoc added a comment - - edited I just discovered something important. The auto-created user (from commit logs) has an email address with underscores (same as the user's full name). This is probably why hudson.tasks.MailAddressResolver.resolve(u) returns the same value as the full name. Who is creating these users from commit logs? Which part of the code is populating the email address for those users? That's where we should be looking!

          Jesse Glick added a comment -

          Who is creating these users from commit logs?

          MercurialChangeSet.getAuthor should call User.get("Name <a@ai>") which should result in a User with an id of Name _a@ai_ and a fullName of Name <a@ai>. If it is calling User.get with a value that already contains underscores, that may be a problem in the Mercurial plugin (or the Digester library it uses, as @jerrymaloney claimed). If some other code (which?) is calling User.get with an ID rather than a full name, then that code would presumably be to blame. Or perhaps the User is created correctly but for some reason the mailer or the MailAddressResolver is ignoring the full name and picking up the ID instead.

          Jesse Glick added a comment - Who is creating these users from commit logs? MercurialChangeSet.getAuthor should call User.get("Name <a@ai>" ) which should result in a User with an id of Name _a@ai_ and a fullName of Name <a@ai> . If it is calling User.get with a value that already contains underscores, that may be a problem in the Mercurial plugin (or the Digester library it uses, as @jerrymaloney claimed). If some other code (which?) is calling User.get with an ID rather than a full name, then that code would presumably be to blame. Or perhaps the User is created correctly but for some reason the mailer or the MailAddressResolver is ignoring the full name and picking up the ID instead.

          cowwoc added a comment -

          Jesse,

          Okay, so to reiterate (I think you already understand, but just in case): fullName being populated with an incorrect value (it contains underscores) and id is correct (containing underscores). I am trying to create a minimal testcase for you now.

          cowwoc added a comment - Jesse, Okay, so to reiterate (I think you already understand, but just in case): fullName being populated with an incorrect value (it contains underscores) and id is correct (containing underscores). I am trying to create a minimal testcase for you now.

          cowwoc added a comment -

          This bug is a pain in the ass. I couldn't reproduce it in a minimal testcase.

          1. The next thing I did was scan my build logs. Looking at jobs/name/builds/date/build.xml section <build><culprits><string> is the value supposed to be an email address or user id.
          2. If I blow away all past builds, the next time Jenkins builds will it add People based on just the latest commit or will it scan the repository's entire history?

          cowwoc added a comment - This bug is a pain in the ass. I couldn't reproduce it in a minimal testcase. 1. The next thing I did was scan my build logs. Looking at jobs/name/builds/date/build.xml section <build><culprits><string> is the value supposed to be an email address or user id. 2. If I blow away all past builds, the next time Jenkins builds will it add People based on just the latest commit or will it scan the repository's entire history?

          Jesse Glick added a comment -

          is the value supposed to be an email address or user id

          ID, so should use underscores.

          will it add People based on just the latest commit or will it scan the repository's entire history?

          Neither; the People link shows authors of changesets mentioned in the changelogs of all build records kept in Jenkins.

          Jesse Glick added a comment - is the value supposed to be an email address or user id ID, so should use underscores. will it add People based on just the latest commit or will it scan the repository's entire history? Neither; the People link shows authors of changesets mentioned in the changelogs of all build records kept in Jenkins.

          cowwoc added a comment - - edited

          Neither; the People link shows authors of changesets mentioned in the changelogs of all build records kept in Jenkins.

          So I should be able to blow away the build records (e.g. jenkins/jobs/boost/2013-09-02_22-47-27) and then sit back waiting for an invalid user to get created, right? Perhaps we're chasing our tails here, looking for a bug that was fixed in the past but still shows up due to old build logs... By blowing away these records, I'll be able to confirm that the issue exists in the latest version. What do you think?

          cowwoc added a comment - - edited Neither; the People link shows authors of changesets mentioned in the changelogs of all build records kept in Jenkins. So I should be able to blow away the build records (e.g. jenkins/jobs/boost/2013-09-02_22-47-27) and then sit back waiting for an invalid user to get created, right? Perhaps we're chasing our tails here, looking for a bug that was fixed in the past but still shows up due to old build logs... By blowing away these records, I'll be able to confirm that the issue exists in the latest version. What do you think?

          Jesse Glick added a comment -

          To anyone who has encountered this bug, be sure to check your $JENKINS_HOME/users/*/config.xml for old user records that might already contain the wrong fullName (or even hudson.tasks.Mailer.UserProperty.emailAddress). If you do, there is nothing wrong with the current Jenkins (or plugin) code: it is just reading bad data. You might have saved the user record in some old version of Jenkins (or with an old buggy plugin TBD) which accidentally used an ID as a full name.

          There are other ways the problem might have happened in principle. Say that in one Jenkins session a User record is created with the correct id and fullName, from a changelog for example; but the user configuration file is not saved. Then a hyperlink is generated, or some kind of serialized record refers to this user (a build cause, etc.), which would use only the id. Then in another Jenkins session this link is followed to /jenkins/user/Some%20Body%20_somebody@nowhere.net_/, which calls User.get on the id; and then some change is made to the record (a new API token, etc.). Now a bogus config.xml will be saved.

          In either case, correcting the problem for a few users is trivial: go to each user’s configuration screen and fix their email address. Whether Jenkins can defend against this problem in general is another matter; I think it would have to eagerly save config.xml whenever get is called with a fullName which gets translated to a different id.

          Jesse Glick added a comment - To anyone who has encountered this bug, be sure to check your $JENKINS_HOME/users/*/config.xml for old user records that might already contain the wrong fullName (or even hudson.tasks.Mailer.UserProperty.emailAddress ). If you do, there is nothing wrong with the current Jenkins (or plugin) code: it is just reading bad data. You might have saved the user record in some old version of Jenkins (or with an old buggy plugin TBD) which accidentally used an ID as a full name. There are other ways the problem might have happened in principle. Say that in one Jenkins session a User record is created with the correct id and fullName , from a changelog for example; but the user configuration file is not saved. Then a hyperlink is generated, or some kind of serialized record refers to this user (a build cause, etc.), which would use only the id . Then in another Jenkins session this link is followed to /jenkins/user/Some%20Body%20_somebody@nowhere.net_/ , which calls User.get on the id ; and then some change is made to the record (a new API token, etc.). Now a bogus config.xml will be saved. In either case, correcting the problem for a few users is trivial: go to each user’s configuration screen and fix their email address. Whether Jenkins can defend against this problem in general is another matter; I think it would have to eagerly save config.xml whenever get is called with a fullName which gets translated to a different id .

          cowwoc added a comment -

          Jesse,

          1. My jenkins/users directory does not contain any bogus users, only the ones who manually created their accounts. Does this tell you anything?
          2. Adding asserts that fail-fast when a bogus config.xml is being created will definitely help us track this to its source.

          cowwoc added a comment - Jesse, My jenkins/users directory does not contain any bogus users, only the ones who manually created their accounts. Does this tell you anything? Adding asserts that fail-fast when a bogus config.xml is being created will definitely help us track this to its source.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/User.java
          http://jenkins-ci.org/commit/jenkins/0ee5f00858f47a9f6ae08d48accbc78c6bf16838
          Log:
          JENKINS-16332 Trying to determine if CanonicalIdResolver is implicated.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/User.java http://jenkins-ci.org/commit/jenkins/0ee5f00858f47a9f6ae08d48accbc78c6bf16838 Log: JENKINS-16332 Trying to determine if CanonicalIdResolver is implicated.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/User.java
          http://jenkins-ci.org/commit/jenkins/e07aac61efc6a77fe58dbd99dca28a51504897b8
          Log:
          JENKINS-16332 More logging at FINE in hudson.model.User.
          Running hypothesis: something historically saved a user configuration file with a bogus fullName;
          thereafter anyone calling User.get(correctFullName) will get a User with the old fullName.
          This would make the bug unreproducible in fresh Jenkins installations.

          Compare: https://github.com/jenkinsci/jenkins/compare/b70f4c07f298...e07aac61efc6

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/User.java http://jenkins-ci.org/commit/jenkins/e07aac61efc6a77fe58dbd99dca28a51504897b8 Log: JENKINS-16332 More logging at FINE in hudson.model.User. Running hypothesis: something historically saved a user configuration file with a bogus fullName; thereafter anyone calling User.get(correctFullName) will get a User with the old fullName. This would make the bug unreproducible in fresh Jenkins installations. Compare: https://github.com/jenkinsci/jenkins/compare/b70f4c07f298...e07aac61efc6

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2877
          JENKINS-16332 Trying to determine if CanonicalIdResolver is implicated. (Revision 0ee5f00858f47a9f6ae08d48accbc78c6bf16838)
          JENKINS-16332 More logging at FINE in hudson.model.User. (Revision e07aac61efc6a77fe58dbd99dca28a51504897b8)

          Result = SUCCESS
          Jesse Glick : 0ee5f00858f47a9f6ae08d48accbc78c6bf16838
          Files :

          • core/src/main/java/hudson/model/User.java

          Jesse Glick : e07aac61efc6a77fe58dbd99dca28a51504897b8
          Files :

          • core/src/main/java/hudson/model/User.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2877 JENKINS-16332 Trying to determine if CanonicalIdResolver is implicated. (Revision 0ee5f00858f47a9f6ae08d48accbc78c6bf16838) JENKINS-16332 More logging at FINE in hudson.model.User. (Revision e07aac61efc6a77fe58dbd99dca28a51504897b8) Result = SUCCESS Jesse Glick : 0ee5f00858f47a9f6ae08d48accbc78c6bf16838 Files : core/src/main/java/hudson/model/User.java Jesse Glick : e07aac61efc6a77fe58dbd99dca28a51504897b8 Files : core/src/main/java/hudson/model/User.java

          cowwoc added a comment -

          Bad news. I wiped out my build log yesterday and today I found two new users created: one with a proper email address, and one with an invalid one. Meaning: the bug still exists as of 1.530 and I still don't know how to reproduce it.

          cowwoc added a comment - Bad news. I wiped out my build log yesterday and today I found two new users created: one with a proper email address, and one with an invalid one. Meaning: the bug still exists as of 1.530 and I still don't know how to reproduce it.

          cowwoc added a comment - - edited

          Here is a regex for validating email addresses (it looks for "FirstName LastName <email@address.com>"):

          (.*\s)([^_@][\w.]+@[\w.]+[a-zA-Z0-9]([>\s]|$))

          Please commit a runtime in this right place for inbound addresses before they get saved. This should allow us to detect who is trying to save bad addresses.

          cowwoc added a comment - - edited Here is a regex for validating email addresses (it looks for "FirstName LastName <email@address.com>"): (.*\s)([^_@][\w.]+@[\w.]+[a-zA-Z0-9]([>\s]|$)) Please commit a runtime in this right place for inbound addresses before they get saved. This should allow us to detect who is trying to save bad addresses.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/User.java
          http://jenkins-ci.org/commit/jenkins/7ca07b5d4b2280d29c1ab92fa83e1391574b1a69
          Log:
          JENKINS-16332 Further diagnostics. Note that you must enable a hudson.model.User logger at FINE or below to use.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/User.java http://jenkins-ci.org/commit/jenkins/7ca07b5d4b2280d29c1ab92fa83e1391574b1a69 Log: JENKINS-16332 Further diagnostics. Note that you must enable a hudson.model.User logger at FINE or below to use.

          cowwoc added a comment -

          Any reason for using a low logging level? Shouldn't we use a higher logging level in the hopes of getting users to provide more information? Also, I would use a more proactive message: "Suspicious fullName being stored: " + fullName + ". Please report to https://issues.jenkins-ci.org/browse/JENKINS-16332"

          cowwoc added a comment - Any reason for using a low logging level? Shouldn't we use a higher logging level in the hopes of getting users to provide more information? Also, I would use a more proactive message: "Suspicious fullName being stored: " + fullName + ". Please report to https://issues.jenkins-ci.org/browse/JENKINS-16332 "

          Jesse Glick added a comment -

          Unless it is entirely sure that this case is definitely a sign of a bug, it is better not to spam log files which potentially a large number of messages. People already watching this bug should enable this logger.

          Jesse Glick added a comment - Unless it is entirely sure that this case is definitely a sign of a bug, it is better not to spam log files which potentially a large number of messages. People already watching this bug should enable this logger.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2919
          JENKINS-16332 Further diagnostics. Note that you must enable a hudson.model.User logger at FINE or below to use. (Revision 7ca07b5d4b2280d29c1ab92fa83e1391574b1a69)

          Result = SUCCESS
          Jesse Glick : 7ca07b5d4b2280d29c1ab92fa83e1391574b1a69
          Files :

          • core/src/main/java/hudson/model/User.java

          dogfood added a comment - Integrated in jenkins_main_trunk #2919 JENKINS-16332 Further diagnostics. Note that you must enable a hudson.model.User logger at FINE or below to use. (Revision 7ca07b5d4b2280d29c1ab92fa83e1391574b1a69) Result = SUCCESS Jesse Glick : 7ca07b5d4b2280d29c1ab92fa83e1391574b1a69 Files : core/src/main/java/hudson/model/User.java

          Jim Goodspeed added a comment -

          If it helps I am also seeing this using 1.537 on Windows with RTC as my SCM tool along with the email-ext plugin. Editing the people (these are not real users, they are picked up from SCM changes) to fix the email address seems to fix the issue for a short period of time, but the records are being updated again intermittently.

          Jim Goodspeed added a comment - If it helps I am also seeing this using 1.537 on Windows with RTC as my SCM tool along with the email-ext plugin. Editing the people (these are not real users, they are picked up from SCM changes) to fix the email address seems to fix the issue for a short period of time, but the records are being updated again intermittently.

          Jesse Glick added a comment -

          @jgoodspeed it helps if you can reproduce the problem and are willing to run the logger (FINE in hudson.model.User).

          Jesse Glick added a comment - @jgoodspeed it helps if you can reproduce the problem and are willing to run the logger ( FINE in hudson.model.User ).

          toomasr added a comment - - edited

          Experiencing similar problem. Unable to send out the emails to the build failed culprits. I went checked out the relevant code in Jenkins, Mercurial plugin and the Mailer plugin. Here is my understanding based on the read source code:

          Mercurial states that the author format is FirstName LastName <email@address> ( see http://www.selenic.com/mercurial/hgrc.5.html )

          Now the mercurial plugin sees the author part with that format and asks Jenkins to get a user or create one by indirectly invoking https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/User.java#L298

          During the creation Jenkins will introduce some replacing ( https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/jenkins/model/DefaultUserCanonicalIdResolver.java#L44 ) and all < and > turn into underscores.

          And now we've created a Jenkins user FirstName LastName _email@address_. Not sure why we want to do this but probably a good thing for later to pick up the culprits.

          Now a build fails and Mailer plugin asks Jenkins to give him the culprits. Jenkins has recorded those users for the build and provides the list of FirstName Lastname _email@address_ users.

          Now Mailer asks for the addresses of the users but as those are empty it will use a MailAddressResolver ( https://github.com/jenkinsci/mailer-plugin/blob/master/src/main/java/hudson/tasks/MailAddressResolver.java#L100 - there is a specific subclass for Subversion also but didn't sis Mercurial) and it will do some magic but probably fails because Mailer still tries to send to FirstName Lastname _email@address_.

          I see couple of options here

          • Mailer plugin(s) to detect the emails better when none is defined
          • Mercurial plugin will do the email address extraction itself, as it knows the format and will then set explicitly the email addresses for the users it creates
          • Mercurial plugin will create the user with the email address instead of the FirstName LastName <email@address> format. In this case any dark magic that does email address extraction from user id might succeed.

          Any holes in my logic? Anyone wants to take crack at it? I'm off to vacation without a laptop

          toomasr added a comment - - edited Experiencing similar problem. Unable to send out the emails to the build failed culprits. I went checked out the relevant code in Jenkins, Mercurial plugin and the Mailer plugin. Here is my understanding based on the read source code: Mercurial states that the author format is FirstName LastName <email@address> ( see http://www.selenic.com/mercurial/hgrc.5.html ) Now the mercurial plugin sees the author part with that format and asks Jenkins to get a user or create one by indirectly invoking https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/User.java#L298 During the creation Jenkins will introduce some replacing ( https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/jenkins/model/DefaultUserCanonicalIdResolver.java#L44 ) and all < and > turn into underscores. And now we've created a Jenkins user FirstName LastName _email@address_. Not sure why we want to do this but probably a good thing for later to pick up the culprits. Now a build fails and Mailer plugin asks Jenkins to give him the culprits. Jenkins has recorded those users for the build and provides the list of FirstName Lastname _email@address_ users. Now Mailer asks for the addresses of the users but as those are empty it will use a MailAddressResolver ( https://github.com/jenkinsci/mailer-plugin/blob/master/src/main/java/hudson/tasks/MailAddressResolver.java#L100 - there is a specific subclass for Subversion also but didn't sis Mercurial) and it will do some magic but probably fails because Mailer still tries to send to FirstName Lastname _email@address_. I see couple of options here Mailer plugin(s) to detect the emails better when none is defined Mercurial plugin will do the email address extraction itself, as it knows the format and will then set explicitly the email addresses for the users it creates Mercurial plugin will create the user with the email address instead of the FirstName LastName <email@address> format. In this case any dark magic that does email address extraction from user id might succeed. Any holes in my logic? Anyone wants to take crack at it? I'm off to vacation without a laptop

          cowwoc added a comment -

          Jesse,

          I just reproduced the problem. Unfortunately I didn't have the logger enabled so we missed an opportunity.

          I did notice something very interesting though:

          1. A build failed and tried to send an email to an auto-generated user with underscores
          2. I went into Jenkins -> People and sure enough the user's email address had underscores
          3. I rebooted Jenkins and went into Manage Jenkins -> Manage Users and all of a sudden that same user no longer had underscores in the email address.

          It's not clear whether the reboot or Manage Users fixed the email address, but that's another clue of what's going on. I've enabled the logger, so the next time this happens I will catch it.

          cowwoc added a comment - Jesse, I just reproduced the problem. Unfortunately I didn't have the logger enabled so we missed an opportunity. I did notice something very interesting though: 1. A build failed and tried to send an email to an auto-generated user with underscores 2. I went into Jenkins -> People and sure enough the user's email address had underscores 3. I rebooted Jenkins and went into Manage Jenkins -> Manage Users and all of a sudden that same user no longer had underscores in the email address. It's not clear whether the reboot or Manage Users fixed the email address, but that's another clue of what's going on. I've enabled the logger, so the next time this happens I will catch it.

          Jesse Glick added a comment -

          If you figure out how to reproduce it from scratch then I can fix it. I am pretty sure the problem somehow lies with a userId getting passed from one place to another without a User context, and then Jenkins serializing the user record at the wrong time, but I do not know the sequence of events that makes this happen to some people some times.

          Jesse Glick added a comment - If you figure out how to reproduce it from scratch then I can fix it. I am pretty sure the problem somehow lies with a userId getting passed from one place to another without a User context, and then Jenkins serializing the user record at the wrong time, but I do not know the sequence of events that makes this happen to some people some times.

          cowwoc added a comment -

          Jesse,

          I triggered a build that failed, and tried emailing the author:

          Invalid address: First Last _first.last@gmail.com_

          I opened the logger and saw these two lines:

          Jan 08, 2014 6:36:06 PM FINE hudson.model.User
          hudson.model.User$FullNameIdResolver@934e9b mapped First Last _first.last@gmail.com_ to First Last _first.last@gmail.com_
          Jan 08, 2014 6:36:06 PM FINE hudson.model.User
          hudson.model.User$FullNameIdResolver@934e9b mapped First Last _first.last@gmail.com_ to First Last _first.last@gmail.com_
          

          (I've replaced the person's actual name with "First Last" but the rest of the log is identical)

          Does this help? Where do we go from here?

          cowwoc added a comment - Jesse, I triggered a build that failed, and tried emailing the author: Invalid address: First Last _first.last@gmail.com_ I opened the logger and saw these two lines: Jan 08, 2014 6:36:06 PM FINE hudson.model.User hudson.model.User$FullNameIdResolver@934e9b mapped First Last _first.last@gmail.com_ to First Last _first.last@gmail.com_ Jan 08, 2014 6:36:06 PM FINE hudson.model.User hudson.model.User$FullNameIdResolver@934e9b mapped First Last _first.last@gmail.com_ to First Last _first.last@gmail.com_ (I've replaced the person's actual name with "First Last" but the rest of the log is identical) Does this help? Where do we go from here?

          Jesse Glick added a comment -

          @cowwoc so something called

          User.get("First Last _first.last@gmail.com_")

          which is OK, but there was already a User which had that not only as an id but also as a fullName, which is wrong. The question is how and when that User got created, which this log snippet does not indicate. To learn that, we would need an (earlier) log message starting with

          [JENKINS-16332] Suspicious fullName being stored: First Last _first.last@gmail.com_

          which includes a stack trace possibly indicating the culprit.

          Jesse Glick added a comment - @cowwoc so something called User.get( "First Last _first.last@gmail.com_" ) which is OK, but there was already a User which had that not only as an id but also as a fullName , which is wrong. The question is how and when that User got created, which this log snippet does not indicate. To learn that, we would need an (earlier) log message starting with [JENKINS-16332] Suspicious fullName being stored: First Last _first.last@gmail.com_ which includes a stack trace possibly indicating the culprit.

          cowwoc added a comment - - edited

          I didn't see such a log entry but I will keep my eye open if this happens again.

          The EMail-ext plugin just fixed an interesting bug: https://issues.jenkins-ci.org/browse/JENKINS-20215

          Do you think this could have been at fault?

          cowwoc added a comment - - edited I didn't see such a log entry but I will keep my eye open if this happens again. The EMail-ext plugin just fixed an interesting bug: https://issues.jenkins-ci.org/browse/JENKINS-20215 Do you think this could have been at fault?

          Jesse Glick added a comment -

          Offhand I would guess that is not related.

          Jesse Glick added a comment - Offhand I would guess that is not related.

          cowwoc added a comment -

          Okay, so to reiterate, I did not see

          [JENKINS-16332] Suspicious fullName being stored: First Last _first.last@gmail.com_

          in the log file even though the problem occurred. Doesn't that imply that we need to add more logs?

          cowwoc added a comment - Okay, so to reiterate, I did not see [JENKINS-16332] Suspicious fullName being stored: First Last _first.last@gmail.com_ in the log file even though the problem occurred. Doesn't that imply that we need to add more logs?

          Jesse Glick added a comment -

          Doesn't that imply that we need to add more logs?

          Not necessarily; you might have turned on logging after this user record ($JENKINS_HOME/users/*.xml) was created with the faulty information.

          Jesse Glick added a comment - Doesn't that imply that we need to add more logs? Not necessarily; you might have turned on logging after this user record ( $JENKINS_HOME/users/*.xml ) was created with the faulty information.

          cowwoc added a comment -

          @Jesse,

          I enabled this log a few weeks ago, so that's unlikely. That being said, the log did seem pretty short (2 pages long).

          1. Does Jenkins truncate the log on startup?
          2. Does it begin logging automatically or do I need to intervene manually every startup?
          3. Does it log to disk or just to memory?

          cowwoc added a comment - @Jesse, I enabled this log a few weeks ago, so that's unlikely. That being said, the log did seem pretty short (2 pages long). Does Jenkins truncate the log on startup? Does it begin logging automatically or do I need to intervene manually every startup? Does it log to disk or just to memory?

          Jesse Glick added a comment -

          The log displayed in /log/all is just the last hundred or so messages in memory (since Jenkins was started). Depending on how you launch Jenkins, typically a log file will be written somewhere to disk.

          Jesse Glick added a comment - The log displayed in /log/all is just the last hundred or so messages in memory (since Jenkins was started). Depending on how you launch Jenkins, typically a log file will be written somewhere to disk.

          Jesse Glick added a comment -

          And custom loggers are normally not written to disk at all. I have an unreleased change in the Support Core plugin which writes custom logs to a location inside the Jenkins home dir (rotated after restart). Plan to release that update soonish.

          Jesse Glick added a comment - And custom loggers are normally not written to disk at all. I have an unreleased change in the Support Core plugin which writes custom logs to a location inside the Jenkins home dir (rotated after restart). Plan to release that update soonish.

          cowwoc added a comment -

          Jesse,

          I am now 100% I am seeing:

          Jan 08, 2014 6:36:06 PM FINE hudson.model.User
          hudson.model.User$FullNameIdResolver@934e9b mapped First Last _first.last@gmail.com_ to First Last _first.last@gmail.com_
          

          without:

          [JENKINS-16332] Suspicious fullName being stored: First Last _first.last@gmail.com_

          ever being logged. It looks like you'll need to add more aggressive logging for us to track this down.

          cowwoc added a comment - Jesse, I am now 100% I am seeing: Jan 08, 2014 6:36:06 PM FINE hudson.model.User hudson.model.User$FullNameIdResolver@934e9b mapped First Last _first.last@gmail.com_ to First Last _first.last@gmail.com_ without: [JENKINS-16332] Suspicious fullName being stored: First Last _first.last@gmail.com_ ever being logged. It looks like you'll need to add more aggressive logging for us to track this down.

          Jesse Glick added a comment -

          If that never gets logged, and you are still unsure how to reproduce the problem, then I am out of ideas for the moment.

          Jesse Glick added a comment - If that never gets logged, and you are still unsure how to reproduce the problem, then I am out of ideas for the moment.

          Alex Earl added a comment -

          Could this have anything to do with this code? https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/AbstractBuild.java#L728 It's using String's instead of User's

          Alex Earl added a comment - Could this have anything to do with this code? https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/AbstractBuild.java#L728 It's using String's instead of User's

          Alex Earl added a comment -

          Alex Earl added a comment - I think this code in Mailer could be an issue as well. https://github.com/jenkinsci/mailer-plugin/blob/41de8d6ada29693073b2c122a367be112ae55663/src/main/java/hudson/tasks/MailAddressResolver.java#L137

          cowwoc added a comment -

          Guys,

          I've managed to produce a minimal testcase for this bug.

          1. Extract testcase.zip into $JENKINS_HOME/jobs
          2. Run the job

          You should get this output:

          Started by user Gili Tzabari
          [EnvInject] - Loading node environment variables.
          Building remotely on MacOS X 10.8 in workspace /Users/builds/.jenkins/workspace/Test
          [Test] $ /usr/local/bin/hg showconfig paths.default
          ERROR: Workspace reports paths.default as https://bbs.darktech.org/code/test/
          which looks different than https://cowwoc@bitbucket.org/cowwoc/test
          so falling back to fresh clone rather than incremental update
          $ /usr/local/bin/hg clone --rev default --noupdate https://cowwoc@bitbucket.org/cowwoc/test /Users/builds/.jenkins/workspace/Test
          
          adding changesets
          adding manifests
          adding file changes
          added 19 changesets with 19 changes to 2 files
          [Test] $ /usr/local/bin/hg update --rev default
          2 files updated, 0 files merged, 0 files removed, 0 files unresolved
          [Test] $ /usr/local/bin/hg log --rev . --template {node}
          [Test] $ /usr/local/bin/hg log --rev . --template {rev}
          [Test] $ /usr/local/bin/hg log --rev 6ddb7626f1d05f3e67a4c92fa255ba32a5406020
          changeset:   18:6ddb7626f1d0
          tag:         tip
          user:        Getcho Getchev <getcho.getchev@gmail.com>
          date:        Fri Feb 21 23:40:41 2014 +0200
          summary:     added line 10
          
          [Test] $ /usr/local/bin/hg log --template "<changeset node='{node}' author='{author|xmlescape}' rev='{rev}' date='{date}'><msg>{desc|xmlescape}</msg><added>{file_adds|stringify|xmlescape}</added><deleted>{file_dels|stringify|xmlescape}</deleted><files>{files|stringify|xmlescape}</files><parents>{parents}</parents></changeset>\n" --rev default:0 --follow --prune 6ddb7626f1d05f3e67a4c92fa255ba32a5406020 --encoding UTF-8 --encodingmode replace
          
          [Test] $ /bin/sh -xe /var/folders/ht/zq3sdm6s55j4cv6gvjtkc8vm0000gp/T/hudson4438504154854395225.sh
          + ./test.sh
          /var/folders/ht/zq3sdm6s55j4cv6gvjtkc8vm0000gp/T/hudson4438504154854395225.sh: line 2: ./test.sh: No such file or directory
          Build step 'Execute shell' marked build as failure
          Invalid address: Getcho Getchev _getcho.getchev@gmail.com_
          Invalid address: Gili Tzabari _gili.tzabari@gmail.com_
          An attempt to send an e-mail to empty list of recipients, ignored.
          
          Finished: FAILURE
          

          cowwoc added a comment - Guys, I've managed to produce a minimal testcase for this bug. Extract testcase.zip into $JENKINS_HOME/jobs Run the job You should get this output: Started by user Gili Tzabari [EnvInject] - Loading node environment variables. Building remotely on MacOS X 10.8 in workspace /Users/builds/.jenkins/workspace/Test [Test] $ /usr/local/bin/hg showconfig paths. default ERROR: Workspace reports paths. default as https: //bbs.darktech.org/code/test/ which looks different than https: //cowwoc@bitbucket.org/cowwoc/test so falling back to fresh clone rather than incremental update $ /usr/local/bin/hg clone --rev default --noupdate https: //cowwoc@bitbucket.org/cowwoc/test /Users/builds/.jenkins/workspace/Test adding changesets adding manifests adding file changes added 19 changesets with 19 changes to 2 files [Test] $ /usr/local/bin/hg update --rev default 2 files updated, 0 files merged, 0 files removed, 0 files unresolved [Test] $ /usr/local/bin/hg log --rev . --template {node} [Test] $ /usr/local/bin/hg log --rev . --template {rev} [Test] $ /usr/local/bin/hg log --rev 6ddb7626f1d05f3e67a4c92fa255ba32a5406020 changeset: 18:6ddb7626f1d0 tag: tip user: Getcho Getchev <getcho.getchev@gmail.com> date: Fri Feb 21 23:40:41 2014 +0200 summary: added line 10 [Test] $ /usr/local/bin/hg log --template "<changeset node= '{node}' author= '{author|xmlescape}' rev= '{rev}' date= '{date}' ><msg>{desc|xmlescape}</msg><added>{file_adds|stringify|xmlescape}</added><deleted>{file_dels|stringify|xmlescape}</deleted><files>{files|stringify|xmlescape}</files><parents>{parents}</parents></changeset>\n" --rev default :0 --follow --prune 6ddb7626f1d05f3e67a4c92fa255ba32a5406020 --encoding UTF-8 --encodingmode replace [Test] $ /bin/sh -xe / var /folders/ht/zq3sdm6s55j4cv6gvjtkc8vm0000gp/T/hudson4438504154854395225.sh + ./test.sh / var /folders/ht/zq3sdm6s55j4cv6gvjtkc8vm0000gp/T/hudson4438504154854395225.sh: line 2: ./test.sh: No such file or directory Build step 'Execute shell' marked build as failure Invalid address: Getcho Getchev _getcho.getchev@gmail.com_ Invalid address: Gili Tzabari _gili.tzabari@gmail.com_ An attempt to send an e-mail to empty list of recipients, ignored. Finished: FAILURE

          Alex Earl added a comment -

          I submitted a pull request for "fixing" this.

          https://github.com/jenkinsci/mailer-plugin/pull/11

          I'm also planning on making a pull request for the Mercurial plugin that would create user accounts a little differently.

          Alex Earl added a comment - I submitted a pull request for "fixing" this. https://github.com/jenkinsci/mailer-plugin/pull/11 I'm also planning on making a pull request for the Mercurial plugin that would create user accounts a little differently.

          Jesse Glick added a comment -

          Your test case is reproducible but it does not really tell us anything, because the malformed culprits are simply being copied from earlier builds (#2 & #5). What is needed is a test case which demonstrates fresh additions to the culprits list with the malformed names; specifically the call stack of User.<init> with an incorrect fullName. Note that the Mercurial changelog.xml in these builds show the correct text.

          Jesse Glick added a comment - Your test case is reproducible but it does not really tell us anything, because the malformed culprits are simply being copied from earlier builds (#2 & #5). What is needed is a test case which demonstrates fresh additions to the culprits list with the malformed names; specifically the call stack of User.<init> with an incorrect fullName . Note that the Mercurial changelog.xml in these builds show the correct text.

          Alex Earl added a comment -

          @Jesse, I agree it should be calling it with the <>'s, but I think because of this code in AbstractBuild.java, it will call it with the _'s

          https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/AbstractBuild.java#L729

          Alex Earl added a comment - @Jesse, I agree it should be calling it with the <>'s, but I think because of this code in AbstractBuild.java, it will call it with the _'s https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/AbstractBuild.java#L729

          Jesse Glick added a comment -

          Still working on reproducing from scratch. I think I may see what the issue is. If I am right, it is a bug in core as I suspected on Dec 19.

          Jesse Glick added a comment - Still working on reproducing from scratch. I think I may see what the issue is. If I am right, it is a bug in core as I suspected on Dec 19.

          Jesse Glick added a comment -

          OK, finally reproduced. The mentioned line 729 is innocent, and really AbstractBuild as a whole is doing nothing wrong; nor, as far as I can tell, are the Mailer or Mercurial plugins.

          The problem (as I surmised on Sep 16) is in User itself. When you run a build that fails, and it has a changelog mentioning at least one user with full name & email address who does not yet have any permanent record in $JENKINS_HOME/users/$id/config.xml, the build stores a culprits field listing the user’s ID, which correctly includes underscores. So long as Jenkins remains running, the loaded User object is fine, and mailing would work. However if you restart Jenkins, and then run a new build that again fails, but with no changelog this time, getCulprits inherits the culprits list from the previous build. This time User.get(id) is called and since there is no persistent record for that user, there is no way of reconstructing the proper fullName—we only have the sanitized ID. Therefore the User is constructed with a bogus fullName and mailing fails.

          Looking into a fix.

          Jesse Glick added a comment - OK, finally reproduced. The mentioned line 729 is innocent, and really AbstractBuild as a whole is doing nothing wrong; nor, as far as I can tell, are the Mailer or Mercurial plugins. The problem (as I surmised on Sep 16) is in User itself. When you run a build that fails, and it has a changelog mentioning at least one user with full name & email address who does not yet have any permanent record in $JENKINS_HOME/users/$id/config.xml , the build stores a culprits field listing the user’s ID, which correctly includes underscores. So long as Jenkins remains running, the loaded User object is fine, and mailing would work. However if you restart Jenkins, and then run a new build that again fails, but with no changelog this time, getCulprits inherits the culprits list from the previous build. This time User.get(id) is called and since there is no persistent record for that user, there is no way of reconstructing the proper fullName —we only have the sanitized ID. Therefore the User is constructed with a bogus fullName and mailing fails. Looking into a fix.

          Alex Earl added a comment -

          Jesse, I just like to poke at these until you come up with the correct solution

          Alex Earl added a comment - Jesse, I just like to poke at these until you come up with the correct solution

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/hudson/model/User.java
          test/src/test/java/hudson/model/UserTest.java
          http://jenkins-ci.org/commit/jenkins/fdda9ca65689423794a8378c1f28faca7dffddf0
          Log:
          [FIXED JENKINS-16332] Ensure that User records are saved when the fullName is not recoverable from the id.
          Otherwise User.get(id).getFullName() after restarting Jenkins will produce the wrong result.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/hudson/model/User.java test/src/test/java/hudson/model/UserTest.java http://jenkins-ci.org/commit/jenkins/fdda9ca65689423794a8378c1f28faca7dffddf0 Log: [FIXED JENKINS-16332] Ensure that User records are saved when the fullName is not recoverable from the id. Otherwise User.get(id).getFullName() after restarting Jenkins will produce the wrong result.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3192
          [FIXED JENKINS-16332] Ensure that User records are saved when the fullName is not recoverable from the id. (Revision fdda9ca65689423794a8378c1f28faca7dffddf0)

          Result = SUCCESS
          Jesse Glick : fdda9ca65689423794a8378c1f28faca7dffddf0
          Files :

          • test/src/test/java/hudson/model/UserTest.java
          • changelog.html
          • core/src/main/java/hudson/model/User.java

          dogfood added a comment - Integrated in jenkins_main_trunk #3192 [FIXED JENKINS-16332] Ensure that User records are saved when the fullName is not recoverable from the id. (Revision fdda9ca65689423794a8378c1f28faca7dffddf0) Result = SUCCESS Jesse Glick : fdda9ca65689423794a8378c1f28faca7dffddf0 Files : test/src/test/java/hudson/model/UserTest.java changelog.html core/src/main/java/hudson/model/User.java

          cowwoc added a comment -

          Jesse,

          Will this fix jobs where malformed culprits were being copied from earlier builds, or do we need to manually "flush" the job history somehow?

          cowwoc added a comment - Jesse, Will this fix jobs where malformed culprits were being copied from earlier builds, or do we need to manually "flush" the job history somehow?

          Jesse Glick added a comment -

          This fix will not handle all cases involving historical records. If a new build is run after upgrade whose changelog mentions the given committer’s full name, that name will be properly registered and you should not see the problem for that user again. The main problem I can foresee is a build which was failing at the time of upgrade, and continues to fail without new changes being committed (to that or other jobs developed by the same people), and is using an email trigger configured to repeatedly send mail for every unstable build.

          Again there are no “malformed” culprits fields here. The culprits list is correct; the problem only arises when the culprits list contains the only record of a given user in the system. You ought to be able to work around the problem for a given user by visiting that user’s page as hyperlinked from a changelog (logged in as an administrator), and explicitly setting a full name (and/or email address).

          Jesse Glick added a comment - This fix will not handle all cases involving historical records. If a new build is run after upgrade whose changelog mentions the given committer’s full name, that name will be properly registered and you should not see the problem for that user again. The main problem I can foresee is a build which was failing at the time of upgrade, and continues to fail without new changes being committed (to that or other jobs developed by the same people), and is using an email trigger configured to repeatedly send mail for every unstable build. Again there are no “malformed” culprits fields here. The culprits list is correct; the problem only arises when the culprits list contains the only record of a given user in the system. You ought to be able to work around the problem for a given user by visiting that user’s page as hyperlinked from a changelog (logged in as an administrator), and explicitly setting a full name (and/or email address).

          cowwoc added a comment -

          I see. So just to make sure I understand correctly:

          Until a user makes a new commit, any existing jobs will keep on using the incorrect email address. Once a user makes a new commit, his email address will get corrected system-wide across all jobs. Alternatively, deleting the contents of the "builds" directory will remove all changelogs and force re-evaluation of the email address.

          Is that correct?

          cowwoc added a comment - I see. So just to make sure I understand correctly: Until a user makes a new commit, any existing jobs will keep on using the incorrect email address. Once a user makes a new commit, his email address will get corrected system-wide across all jobs. Alternatively, deleting the contents of the "builds" directory will remove all changelogs and force re-evaluation of the email address. Is that correct?

          Jesse Glick added a comment -

          Until a user makes a new commit, any existing jobs will keep on using the incorrect email address. Once a user makes a new commit, his email address will get corrected system-wide across all jobs.

          Right. You can simply look in $JENKINS_HOME/users/ to see whether the user has been defined or not.

          deleting the contents of the "builds" directory will remove all changelogs and force re-evaluation of the email address.

          No. If you deleted all historical builds, then either the next build has commits from the relevant user(s), in which case their email addresses will be recorded; or it does not, in which case the culprits list for that build will simply be empty and no email will be sent (unless you have an unconditional recipient configured in the job). In the former case, deleting the old build records does not improve your situation at all. In the latter case, deleting them could make the new build’s status be “unstable” due to test failures or whatever, rather than “failed” due to an attempt to send mail to bogus addresses, but the email addresses of users mentioned in those old builds would still be missing.

          Jesse Glick added a comment - Until a user makes a new commit, any existing jobs will keep on using the incorrect email address. Once a user makes a new commit, his email address will get corrected system-wide across all jobs. Right. You can simply look in $JENKINS_HOME/users/ to see whether the user has been defined or not. deleting the contents of the "builds" directory will remove all changelogs and force re-evaluation of the email address. No. If you deleted all historical builds, then either the next build has commits from the relevant user(s), in which case their email addresses will be recorded; or it does not, in which case the culprits list for that build will simply be empty and no email will be sent (unless you have an unconditional recipient configured in the job). In the former case, deleting the old build records does not improve your situation at all. In the latter case, deleting them could make the new build’s status be “unstable” due to test failures or whatever, rather than “failed” due to an attempt to send mail to bogus addresses, but the email addresses of users mentioned in those old builds would still be missing.

          Fixed in 1.554. No need to backport.

          Oliver Gondža added a comment - Fixed in 1.554. No need to backport.

            jglick Jesse Glick
            sgr Sergey Grigoriev
            Votes:
            6 Vote for this issue
            Watchers:
            20 Start watching this issue

              Created:
              Updated:
              Resolved: