[JENKINS-48648] Violation Comments to Bitbucket Server plugin does not add comments to files with non-ASCII names on Windows

Type: Bug
Resolution: Fixed
Priority: Minor
Component/s: violation-comments-to-github-plugin
Labels:
- encoding
- windows
Environment:
Bitbucket Server on Windows
Jenkins 2.89.2 on Windows
Violation Comments to Bitbucket Server Plugin 1.58

Similar Issues:
Powered by SuggestiMate

Show

I deliberately changed three files so that the build system generates warnings that the Violation Comments to Bitbucket Server Plugin should report as comments to a pull request:

mobiilirajapinta/json-dateandtime.md
mobiilirajapinta/json-nimeämiskäytäntö.md
sanasto.md

The plugin does add comments about the other two files but not about "mobiilirajapinta/json-nimeämiskäytäntö.md", whose name contains non-ASCII characters. This happens on Windows.

The console output says:

Found 3 violations from ViolationConfig [pattern=.*\\docfx\.log\.xml, reporter=DocFX, parser=JCREPORT].

Jenkins loggers say:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

Found 1 reports:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

C:\JenkTst\agent1\workspace\n-mobiilin-dokumentit_PR-16-XBYT654LGQHOZLTISQHSND3OR52VOZRVTZYYQSSUQPHLAN2AU6IQ\obj\docfx.log.xml

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

Found 3 violations:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-nimeämiskäytäntö.md 7 -> 7

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) sanasto.md 63 -> 63

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-dateandtime.md 18 -> 18

joulukuuta 20, 2017 12:58:13 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator 

3 violations.

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

Asking BitbucketServerCommentsProvider to comment:

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=sanasto.md, specifics=[]] 63 

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=mobiilirajapinta/json-dateandtime.md, specifics=[]] 18

That is, the plugin did not realize that the pull request included a change to mobiilirajapinta/json-nimeämiskäytäntö.md, so it did not report any violations in that file. However, the web UI of Bitbucket Server shows that the pull request includes such a change, and so does the REST API.

I'm guessing the problem is in BitbucketServerInvoker.invokeUrl:

      reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));

This calls public InputStreamReader(InputStream in), which uses the default charset of Java, likely Windows-1252 on Windows; but the JSON HTTP response from the server seems to be always UTF-8.

I'm filing this under the violation-comments-to-github-plugin component because there is no violation-comments-to-stash-plugin component and both plugins have the same author.

Kalle Niemitalo created issue - 2017-12-20 11:13

Kalle Niemitalo made changes - 2017-12-20 12:03

Description

Original: I deliberately changed three files so that the build system generates warnings that the Violation Comments to Bitbucket Server Plugin should report as comments to a pull request:
* mobiilirajapinta/json-dateandtime.md
* mobiilirajapinta/json-nimeämiskäytäntö.md
* sanasto.md

The plugin does add comments about the other two files but not about "mobiilirajapinta/json-nimeämiskäytäntö.md", whose name contains non-ASCII characters. This happens on Windows.

The console output says:
{noformat}
Found 3 violations from ViolationConfig [pattern=.*\\docfx\.log\.xml, reporter=DocFX, parser=JCREPORT].{noformat}
Jenkins loggers say:
{noformat}
joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

Found 1 reports:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

C:\JenkTst\agent1\workspace\n-mobiilin-dokumentit_PR-16-XBYT654LGQHOZLTISQHSND3OR52VOZRVTZYYQSSUQPHLAN2AU6IQ\obj\docfx.log.xml

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

Found 3 violations:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-nimeämiskäytäntö.md 7 -> 7

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) sanasto.md 63 -> 63

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-dateandtime.md 18 -> 18

joulukuuta 20, 2017 12:58:13 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator

3 violations.

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

Asking BitbucketServerCommentsProvider to comment:

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=sanasto.md, specifics=[]] 63

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=mobiilirajapinta/json-dateandtime.md, specifics=[]] 18
{noformat}
That is, the plugin did not realize that the pull request included a change to mobiilirajapinta/json-nimeämiskäytäntö.md, so it did not report any violations in that file. However, the web UI of Bitbucket Server shows that the pull request includes such a change, and so does the REST API.

I'm guessing the problem is in [BitbucketServerInvoker.invokeUrl|https://github.com/tomasbjerre/violation-comments-to-bitbucket-server-lib/blob/1.34/src/main/java/se/bjurr/violations/comments/bitbucketserver/lib/client/BitbucketServerInvoker.java#L51]:
{code:java}
reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
{code}
This calls {{public InputStreamReader(InputStream in)}}, which uses the default charset of Java, likely Windows-1252 on Windows; but the JSON HTTP response from the server seems to be always UTF-8.

I'm filing this under the violation-comments-to-github-plugin component because there is no violation-comments-to-stash component and both plugins have the same author.

New: I deliberately changed three files so that the build system generates warnings that the Violation Comments to Bitbucket Server Plugin should report as comments to a pull request:
* mobiilirajapinta/json-dateandtime.md
* mobiilirajapinta/json-nimeämiskäytäntö.md
* sanasto.md

The plugin does add comments about the other two files but not about "mobiilirajapinta/json-nimeämiskäytäntö.md", whose name contains non-ASCII characters. This happens on Windows.

The console output says:
{noformat}
Found 3 violations from ViolationConfig [pattern=.*\\docfx\.log\.xml, reporter=DocFX, parser=JCREPORT].{noformat}
Jenkins loggers say:
{noformat}
joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

Found 1 reports:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

C:\JenkTst\agent1\workspace\n-mobiilin-dokumentit_PR-16-XBYT654LGQHOZLTISQHSND3OR52VOZRVTZYYQSSUQPHLAN2AU6IQ\obj\docfx.log.xml

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

Found 3 violations:

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-nimeämiskäytäntö.md 7 -> 7

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) sanasto.md 63 -> 63

joulukuuta 20, 2017 12:58:13 IP. FINE ViolationsReporterApi

DocFX INFO (InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save)) mobiilirajapinta/json-dateandtime.md 18 -> 18

joulukuuta 20, 2017 12:58:13 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator

3 violations.

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

Asking BitbucketServerCommentsProvider to comment:

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=sanasto.md, specifics=[]] 63

joulukuuta 20, 2017 12:58:14 IP. FINE se.bjurr.violations.comments.lib.CommentsCreator createSingleFileComments

DocFX INFO InvalidFileLink(BuildCore.Build Document.LinkPhaseHandlerWithIncremental.ConceptualDocumentProcessor.Save) ChangedFile [filename=mobiilirajapinta/json-dateandtime.md, specifics=[]] 18
{noformat}
That is, the plugin did not realize that the pull request included a change to mobiilirajapinta/json-nimeämiskäytäntö.md, so it did not report any violations in that file. However, the web UI of Bitbucket Server shows that the pull request includes such a change, and so does the REST API.

I'm guessing the problem is in [BitbucketServerInvoker.invokeUrl|https://github.com/tomasbjerre/violation-comments-to-bitbucket-server-lib/blob/1.34/src/main/java/se/bjurr/violations/comments/bitbucketserver/lib/client/BitbucketServerInvoker.java#L51]:
{code:java}
reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
{code}
This calls {{public InputStreamReader(InputStream in)}}, which uses the default charset of Java, likely Windows-1252 on Windows; but the JSON HTTP response from the server seems to be always UTF-8.

I'm filing this under the violation-comments-to-github-plugin component because there is no violation-comments-to-stash-plugin component and both plugins have the same author.

Tomas Bjerre made changes - 2017-12-20 20:34

Status

Original: Open [ 1 ]

New: In Progress [ 3 ]

Tomas Bjerre made changes - 2017-12-20 20:35

Resolution		New: Fixed [ 1 ]
Status	Original: In Progress [ 3 ]	New: Resolved [ 5 ]

Kalle Niemitalo made changes - 2017-12-21 09:26

Status

Original: Resolved [ 5 ]

New: Closed [ 6 ]

Assignee:: Tomas Bjerre

Reporter:: Kalle Niemitalo

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: 2017-12-20 11:13

Updated:: 2017-12-21 09:26

Resolved:: 2017-12-20 20:35

Jenkins

Details

Description

Attachments

Activity

People

Dates