Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10160

When the network is broken,then jenkins will query dns info in loop.The log file will grow up very fast.

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • core
    • None
    • Windows XP Pro|SunJDK1.6.0_21|Tomcat 7.0.14|Jenkins1.418
      Debian 6|OpenJDK1.6.0_18|Tomcat7.0.14|Jenkins1.413
      Server 2008 R2 x64|Sun JDK 1.6.0_25 Jenkins 1.411 and Jenkins 1.413

      The network in my company is often broken.When it was broken,the log file of jenkins would grow up very fast until there is no disk space for it.
      Here is the log in my Windows.

      	question:      [DNSQuestion@9657493 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@29482661 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@16787101 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@12328574 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@7738582 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@776128 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@25267887 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@8281222 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@32275542 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      	question:      [DNSQuestion@32262868 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
      

          [JENKINS-10160] When the network is broken,then jenkins will query dns info in loop.The log file will grow up very fast.

          Rob Petti added a comment -

          I've been able to reproduce this in Centos 5.6 x64 as well. -Dhudson.DNSMultiCast.disabled=true seems to make it go away...

          Rob Petti added a comment - I've been able to reproduce this in Centos 5.6 x64 as well. -Dhudson.DNSMultiCast.disabled=true seems to make it go away...

          Rob Petti added a comment -

          Scratch that. It's back again.

          Rob Petti added a comment - Scratch that. It's back again.

          Joseph Lawson added a comment -

          Is anyone running on a virtual machine? I wonder if a vMotion or similar operation is causing the blip in the network adapter which triggers this issue. We are running ESXi 4

          Joseph Lawson added a comment - Is anyone running on a virtual machine? I wonder if a vMotion or similar operation is causing the blip in the network adapter which triggers this issue. We are running ESXi 4

          If anyone sees this, can you please get the thread dump so that we can better understand what's going on?

          Kohsuke Kawaguchi added a comment - If anyone sees this, can you please get the thread dump so that we can better understand what's going on?

          Rob Petti added a comment -

          I'm still running into this in Jenkins 1.446, though this time I'm running in tomcat and I've set -Dhudson.DNSMultiCast.disabled=true in catalina.sh, and it seems to be holding so far. Personally I don't think this is a problem with Jenkins so much as the jmDNS library. If it happens again, I'll be sure to give you a thread dump if possible.

          To (belatedly) answer Joseph's question, I was running in ESXi 4 during my first round of tests, but I'm currently running on physical hardware and still encountering the issue, so I don't think it's related.

          Rob Petti added a comment - I'm still running into this in Jenkins 1.446, though this time I'm running in tomcat and I've set -Dhudson.DNSMultiCast.disabled=true in catalina.sh, and it seems to be holding so far. Personally I don't think this is a problem with Jenkins so much as the jmDNS library. If it happens again, I'll be sure to give you a thread dump if possible. To (belatedly) answer Joseph's question, I was running in ESXi 4 during my first round of tests, but I'm currently running on physical hardware and still encountering the issue, so I don't think it's related.

          BalagiriPrasad S added a comment - - edited

          Thread dump is attached (Find it in the Attachments section).
          At this point tomcat is consuming 993MB of memory.

          BalagiriPrasad S added a comment - - edited Thread dump is attached (Find it in the Attachments section). At this point tomcat is consuming 993MB of memory.

          Looks like the offending thread is this:

          "SocketListener(praba03-ac1.local.)" daemon prio=6 tid=0x3f682400 nid=0xce4 runnable [0x430ef000]
          java.lang.Thread.State: RUNNABLE
          at java.lang.Class.getEnclosingMethod0(Native Method)
          at java.lang.Class.getEnclosingMethodInfo(Class.java:929)
          at java.lang.Class.getEnclosingClass(Class.java:1081)
          at java.lang.Class.getSimpleBinaryName(Class.java:1220)
          at java.lang.Class.getSimpleName(Class.java:1112)
          at javax.jmdns.impl.DNSEntry.toString(DNSEntry.java:279)
          at java.lang.String.valueOf(String.java:2826)
          at java.lang.StringBuilder.append(StringBuilder.java:115)
          at javax.jmdns.impl.DNSIncoming.toString(DNSIncoming.java:474)
          at javax.jmdns.impl.DNSMessage.print(DNSMessage.java:225)
          at javax.jmdns.impl.DNSIncoming.print(DNSIncoming.java:419)
          at javax.jmdns.impl.DNSIncoming.readAnswer(DNSIncoming.java:261)
          at javax.jmdns.impl.DNSIncoming.<init>(DNSIncoming.java:219)
          at javax.jmdns.impl.SocketListener.run(SocketListener.java:50)

          JMDNS appears to have fixed in May 2012 this in r342: "UDP portscan causes JmDNS to log excessively - ID: 3529498"

          Unfortunately, the latest release is 3.4.1, bundled back in Aug 2011, and no "Release" jar has this fix. Perhaps we could ask for a 3.4.2 release, and bundle that?

          Daryl Banttari added a comment - Looks like the offending thread is this: "SocketListener(praba03-ac1.local.)" daemon prio=6 tid=0x3f682400 nid=0xce4 runnable [0x430ef000] java.lang.Thread.State: RUNNABLE at java.lang.Class.getEnclosingMethod0(Native Method) at java.lang.Class.getEnclosingMethodInfo(Class.java:929) at java.lang.Class.getEnclosingClass(Class.java:1081) at java.lang.Class.getSimpleBinaryName(Class.java:1220) at java.lang.Class.getSimpleName(Class.java:1112) at javax.jmdns.impl.DNSEntry.toString(DNSEntry.java:279) at java.lang.String.valueOf(String.java:2826) at java.lang.StringBuilder.append(StringBuilder.java:115) at javax.jmdns.impl.DNSIncoming.toString(DNSIncoming.java:474) at javax.jmdns.impl.DNSMessage.print(DNSMessage.java:225) at javax.jmdns.impl.DNSIncoming.print(DNSIncoming.java:419) at javax.jmdns.impl.DNSIncoming.readAnswer(DNSIncoming.java:261) at javax.jmdns.impl.DNSIncoming.<init>(DNSIncoming.java:219) at javax.jmdns.impl.SocketListener.run(SocketListener.java:50) JMDNS appears to have fixed in May 2012 this in r342: "UDP portscan causes JmDNS to log excessively - ID: 3529498" Unfortunately, the latest release is 3.4.1, bundled back in Aug 2011, and no "Release" jar has this fix. Perhaps we could ask for a 3.4.2 release, and bundle that?

          For people looking for a workaround (and not afraid of command lines):

          1. Download and unzip the latest jmdns snapshot (I used http://sourceforge.net/p/jmdns/code/353/tree/)
          2. Change to that directory, then build by typing: "mvn package" (requires Java developer kit and Maven)
          3. Using your favorite zip tool, extract the "com" directory of jmdns-3.4.0-jenkins-3.jar into a temp dir. Zip just the "com" directory back up as jmdns-tools.jar
          4. Stop tomcat (or whatever container you're using to run Jenkins)
          5. Where Jenkins is deployed, rename ./WEB-INF/lib/jmdns-3.4.0-jenkins-3.jar jmdns-3.4.0-jenkins-3.jar.old
          6. Copy jmdns-3.4.2-SNAPSHOT.jar (will be in the "target" directory below where you extracted the jmdns snapshot) into WEB-INF/lib
          7. Copy jmdns-tools.jar into WEB-INF/lib
          8. Start tomcat (or whatever container you're using to run Jenkins)

          Not sure this will fix the problem, but so far hasn't appeared to hurt.

          Daryl Banttari added a comment - For people looking for a workaround (and not afraid of command lines): 1. Download and unzip the latest jmdns snapshot (I used http://sourceforge.net/p/jmdns/code/353/tree/ ) 2. Change to that directory, then build by typing: "mvn package" (requires Java developer kit and Maven) 3. Using your favorite zip tool, extract the "com" directory of jmdns-3.4.0-jenkins-3.jar into a temp dir. Zip just the "com" directory back up as jmdns-tools.jar 4. Stop tomcat (or whatever container you're using to run Jenkins) 5. Where Jenkins is deployed, rename ./WEB-INF/lib/jmdns-3.4.0-jenkins-3.jar jmdns-3.4.0-jenkins-3.jar.old 6. Copy jmdns-3.4.2-SNAPSHOT.jar (will be in the "target" directory below where you extracted the jmdns snapshot) into WEB-INF/lib 7. Copy jmdns-tools.jar into WEB-INF/lib 8. Start tomcat (or whatever container you're using to run Jenkins) Not sure this will fix the problem, but so far hasn't appeared to hurt.

          Markus Stein added a comment -

          Here, too...

          Sep 18, 2013 9:46:16 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x239
          Sep 18, 2013 9:46:16 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          SEVERE: Could not find record type for index: -1
          Sep 18, 2013 9:46:16 AM javax.jmdns.impl.DNSIncoming readAnswer
          SEVERE: Could not find record type. domain:
          dns[response,x.x.x.x:5353, length=571, id=0x0, flags=0x8000:r, answers=1

          note: ipadress replaced with x.x.x.x

          Jenkins 1.531

          Markus Stein added a comment - Here, too... Sep 18, 2013 9:46:16 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x239 Sep 18, 2013 9:46:16 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex SEVERE: Could not find record type for index: -1 Sep 18, 2013 9:46:16 AM javax.jmdns.impl.DNSIncoming readAnswer SEVERE: Could not find record type. domain: dns[response,x.x.x.x:5353, length=571, id=0x0, flags=0x8000:r, answers=1 note: ipadress replaced with x.x.x.x Jenkins 1.531

          Stephen Feather added a comment - - edited

          This started occuring this week with Jenkins 1.552 on OSX.

          The "when the network is broken" is a bit vague, be nice to know what that actually 'means' to the original poster.

          Sep 30, 2014 7:02:44 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex
          WARNING: Could not find record class for index: -1
          Sep 30, 2014 7:02:44 AM javax.jmdns.impl.DNSIncoming readAnswer
          SEVERE: Could not find record class. domain:  type: TYPE_IGNORE index 0
          dns[response,192.168.1.110:5353, length=303, id=0x0, flags=0x8400:r:aa, answers=1
          answers:
          	[Text@532787820 type: TYPE_TXT index 16, class: CLASS_IN index 1-unique, name: 9C207B8E5E2F@Office Apple TV._raop._tcp.local. ttl: '4499/4500' text: '
          cn=0,1,2,3da=tr...']]
          	answer:        [Text@532787820 type: TYPE_TXT index 16, class: CLASS_IN index 1-unique, name: 9C207B8E5E2F@Office Apple TV._raop._tcp.local. ttl: '4499/4500' text: '
          cn=0,1,2,3da=tr...']
             0: 0000840000000001 000000021c394332 3037423845354532 46404f6666696365     ........ .....9C2 07B8E5E2 F@Office
            20: 204170706c652054 56055f72616f7004 5f746370056c6f63 616c000010800100     .Apple.T V._raop. _tcp.loc al......
            40: 00119400b00a636e 3d302c312c322c33 0764613d74727565 0865743d302c332c     ......cn =0,1,2,3 .da=true .et=0,3,
            60: 351166743d307835 413746464646372c 307845086d643d30 2c312c320d616d3d     5.ft=0x5 A7FFFF7, 0xE.md=0 ,1,2.am=
            80: 4170706c65545633 2c3143706b3d6134 6530613364306633 3236336463393339     AppleTV3 ,1Cpk=a4 e0a3d0f3 263dc939
            a0: 6535393532323434 3135383937613830 6531316136643838 6663346632303264     e5952244 15897a80 e11a6d88 fc4f202d
            c0: 6162613535383330 3438383164330773 663d307834340674 703d55445008766e     aba55830 4881d3.s f=0x44.t p=UDP.vn
            e0: 3d36353533370976 733d3231302e3938 0476763d32000029 05a0000000000018     =65537.v s=210.98 .vv=2..) ........
           100: 0004001400970200 0000000002000000 0000000000000000 00002905a0000000     ........ ........ ........ ..).....
           120: 00000cfdea0008de 47a811e167e381                                         ........ G...g..
          

          Stephen Feather added a comment - - edited This started occuring this week with Jenkins 1.552 on OSX. The "when the network is broken" is a bit vague, be nice to know what that actually 'means' to the original poster. Sep 30, 2014 7:02:44 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex WARNING: Could not find record class for index: -1 Sep 30, 2014 7:02:44 AM javax.jmdns.impl.DNSIncoming readAnswer SEVERE: Could not find record class. domain: type: TYPE_IGNORE index 0 dns[response,192.168.1.110:5353, length=303, id=0x0, flags=0x8400:r:aa, answers=1 answers: [Text@532787820 type: TYPE_TXT index 16, class: CLASS_IN index 1-unique, name: 9C207B8E5E2F@Office Apple TV._raop._tcp.local. ttl: '4499/4500' text: ' cn=0,1,2,3da=tr...']] answer: [Text@532787820 type: TYPE_TXT index 16, class: CLASS_IN index 1-unique, name: 9C207B8E5E2F@Office Apple TV._raop._tcp.local. ttl: '4499/4500' text: ' cn=0,1,2,3da=tr...'] 0: 0000840000000001 000000021c394332 3037423845354532 46404f6666696365 ........ .....9C2 07B8E5E2 F@Office 20: 204170706c652054 56055f72616f7004 5f746370056c6f63 616c000010800100 .Apple.T V._raop. _tcp.loc al...... 40: 00119400b00a636e 3d302c312c322c33 0764613d74727565 0865743d302c332c ......cn =0,1,2,3 .da= true .et=0,3, 60: 351166743d307835 413746464646372c 307845086d643d30 2c312c320d616d3d 5.ft=0x5 A7FFFF7, 0xE.md=0 ,1,2.am= 80: 4170706c65545633 2c3143706b3d6134 6530613364306633 3236336463393339 AppleTV3 ,1Cpk=a4 e0a3d0f3 263dc939 a0: 6535393532323434 3135383937613830 6531316136643838 6663346632303264 e5952244 15897a80 e11a6d88 fc4f202d c0: 6162613535383330 3438383164330773 663d307834340674 703d55445008766e aba55830 4881d3.s f=0x44.t p=UDP.vn e0: 3d36353533370976 733d3231302e3938 0476763d32000029 05a0000000000018 =65537.v s=210.98 .vv=2..) ........ 100: 0004001400970200 0000000002000000 0000000000000000 00002905a0000000 ........ ........ ........ ..)..... 120: 00000cfdea0008de 47a811e167e381 ........ G...g..

          I have found this: http://sourceforge.net/p/jmdns/bugs/130/

          It seems to be quite possible that the problem is caused by malformed DNS responses. Either because of the corrupted UDP datagrams (unlikely, but possible), or by bad DNS server responses, or, more likely, by someone running a scan against the machine.

          We have got this problem too. I am going to apply the recommendation to upgrade jmdns library AND also I am going to start using local caching BIND instance. I believe that BIND might better resist the fake responses.

          Nikolai Grigoriev added a comment - I have found this: http://sourceforge.net/p/jmdns/bugs/130/ It seems to be quite possible that the problem is caused by malformed DNS responses. Either because of the corrupted UDP datagrams (unlikely, but possible), or by bad DNS server responses, or, more likely, by someone running a scan against the machine. We have got this problem too. I am going to apply the recommendation to upgrade jmdns library AND also I am going to start using local caching BIND instance. I believe that BIND might better resist the fake responses.

          Actually, more on this. I have realized that the problem is most likely related to Jenkins multicast DNS support and auto-discovery feature (it listens on two UDP ports). I have disabled both with "-Dhudson.DNSMultiCast.disabled=true -Dhudson.udp=-1" Java options. That should eliminate the problem, I think.

          Nikolai Grigoriev added a comment - Actually, more on this. I have realized that the problem is most likely related to Jenkins multicast DNS support and auto-discovery feature (it listens on two UDP ports). I have disabled both with "-Dhudson.DNSMultiCast.disabled=true -Dhudson.udp=-1" Java options. That should eliminate the problem, I think.

          ngrigoriev that workaround worked for me. Thanks.

          Cleydson José de Figueiredo Júnior added a comment - ngrigoriev that workaround worked for me. Thanks.

          R. Tyler Croy added a comment -

          We use a patched version of jmdns which looks to me as if it is synced with jmdns v3.4.0, but I see in their SourceForge account they've tagged at least 3.4.1.

          dbanttari if you wish to open a pull request to fix our patched version of jmdns for this issue, I can try to help get that merged through and released (if I'm able to convince abayer to jump on board )

          R. Tyler Croy added a comment - We use a patched version of jmdns which looks to me as if it is synced with jmdns v3.4.0, but I see in their SourceForge account they've tagged at least 3.4.1. dbanttari if you wish to open a pull request to fix our patched version of jmdns for this issue, I can try to help get that merged through and released (if I'm able to convince abayer to jump on board )

          Andrew Bayer added a comment -

          That can be arranged. =)

          Andrew Bayer added a comment - That can be arranged. =)

          Cody Scott added a comment -

          It looks like development has moved to https://github.com/jmdns/jmdns with the latest version being 3.5.1 https://github.com/jmdns/jmdns/releases

          Cody Scott added a comment - It looks like development has moved to https://github.com/jmdns/jmdns with the latest version being 3.5.1 https://github.com/jmdns/jmdns/releases

          Zachary LaCelle added a comment - - edited

          EDIT: I misspelled the argument (disable vs disabled). Seems to work.

          PREVIOUS:

          I attempted the fix mentioned by ngrigoriev (-Dhudson.DNSMultiCast.disabled=true -Dhudson.udp=-1), but it doesn't seem to fix the problem.

          Here's the arguments given to Jenkins:

          /usr/bin/java -Dmail.smtp.starttls.enable=true -Dhudson.DNSMultiCast.disable=true -Dhudson.udp=-1 -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080 --httpsPort=-1 --ajp13Port=-1

          Here's what my jenkins.log looks like:

          question: [DNSQuestion@2137451499 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@816664983 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1397021935 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@365274778 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1597387715 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@2142995283 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@94356232 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@118673637 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1585153435 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@943519598 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1812730091 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1440318277 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1220146348 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@134233287 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1038834151 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@802211412 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@904112 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1234766810 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@170379198 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@2106342031 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@998228220 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@163782323 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1134412540 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1672826511 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@339263853 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1631728593 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1129001243 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@575558159 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1850896725 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@618325458 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@1896930720 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@466126220 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          question: [DNSQuestion@784260492 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          0: 6239636662373665 3330666130346538 3138613733653665 3136656538363865 b9cfb76e 30fa04e8 18a73e6e 16ee868e
          20: 6134623363333637 6436613934326566 6634633333626132 3436633762313862 a4b3c367 d6a942ef f4c33ba2 46c7b18b
          40: 6436323135633635 3061656562643332 3935663530636562 3838313030656533 d6215c65 0aeebd32 95f50ceb 88100ee3
          60: 3561346136616264 3335363762323362 6539396236343836 3235323736383161 5a4a6abd 3567b23b e99b6486 2527681a
          80: 3664336130313838 3633656264653235 3433626563366637 6465396636646262 6d3a0188 63ebde25 43bec6f7 de9f6dbb
          a0: 3032396464623739 6638353233393732 6436633133353863 3435326262383438 029ddb79 f8523972 d6c1358c 452bb848
          c0: 6137326462323365 3837333233343234 6138383862666136 6463393963646236 a72db23e 87323424 a888bfa6 dc99cdb6
          e0: 3866623031633737 3238396363393838 3837646462363361 3139666338623333 8fb01c77 289cc988 87ddb63a 19fc8b33
          100: 6261623066306531 6332646137636566 3738666435343530 6637643233616461 bab0f0e1 c2da7cef 78fd5450 f7d23ada
          120: 3561396337636433 3365386361313933 6234636161363661 3633633062666337 5a9c7cd3 3e8ca193 b4caa66a 63c0bfc7

          Any further ideas? This issue means that my log fills up in about 5 days, causing the server to crash.

          Zachary LaCelle added a comment - - edited EDIT: I misspelled the argument (disable vs disabled). Seems to work. PREVIOUS: I attempted the fix mentioned by ngrigoriev (-Dhudson.DNSMultiCast.disabled=true -Dhudson.udp=-1), but it doesn't seem to fix the problem. Here's the arguments given to Jenkins: /usr/bin/java -Dmail.smtp.starttls.enable=true -Dhudson.DNSMultiCast.disable=true -Dhudson.udp=-1 -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080 --httpsPort=-1 --ajp13Port=-1 Here's what my jenkins.log looks like: question: [DNSQuestion@2137451499 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@816664983 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1397021935 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@365274778 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1597387715 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@2142995283 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@94356232 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@118673637 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1585153435 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@943519598 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1812730091 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1440318277 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1220146348 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@134233287 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1038834151 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@802211412 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@904112 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1234766810 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@170379198 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@2106342031 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@998228220 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@163782323 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1134412540 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1672826511 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@339263853 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1631728593 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1129001243 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@575558159 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1850896725 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@618325458 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@1896930720 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@466126220 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] question: [DNSQuestion@784260492 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] 0: 6239636662373665 3330666130346538 3138613733653665 3136656538363865 b9cfb76e 30fa04e8 18a73e6e 16ee868e 20: 6134623363333637 6436613934326566 6634633333626132 3436633762313862 a4b3c367 d6a942ef f4c33ba2 46c7b18b 40: 6436323135633635 3061656562643332 3935663530636562 3838313030656533 d6215c65 0aeebd32 95f50ceb 88100ee3 60: 3561346136616264 3335363762323362 6539396236343836 3235323736383161 5a4a6abd 3567b23b e99b6486 2527681a 80: 3664336130313838 3633656264653235 3433626563366637 6465396636646262 6d3a0188 63ebde25 43bec6f7 de9f6dbb a0: 3032396464623739 6638353233393732 6436633133353863 3435326262383438 029ddb79 f8523972 d6c1358c 452bb848 c0: 6137326462323365 3837333233343234 6138383862666136 6463393963646236 a72db23e 87323424 a888bfa6 dc99cdb6 e0: 3866623031633737 3238396363393838 3837646462363361 3139666338623333 8fb01c77 289cc988 87ddb63a 19fc8b33 100: 6261623066306531 6332646137636566 3738666435343530 6637643233616461 bab0f0e1 c2da7cef 78fd5450 f7d23ada 120: 3561396337636433 3365386361313933 6234636161363661 3633633062666337 5a9c7cd3 3e8ca193 b4caa66a 63c0bfc7 Any further ideas? This issue means that my log fills up in about 5 days, causing the server to crash.

          Same problem here with jenkins 2.25, using -Dhudson.DNSMultiCast.disabled=true

          Oct 11, 2016 1:23:04 PM javax.jmdns.impl.DNSIncoming readQuestion
          SEVERE: Could not find record type: dns[response,10.200.31.173:5353, length=12, id=0x0, flags=0x84f4:r:aa, questions=17623
          questions:
          [DNSQuestion@422648395 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          [DNSQuestion@1723154136 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          etc.

          The log fills several GB/min.

          Maxime Viargues added a comment - Same problem here with jenkins 2.25, using -Dhudson.DNSMultiCast.disabled=true Oct 11, 2016 1:23:04 PM javax.jmdns.impl.DNSIncoming readQuestion SEVERE: Could not find record type: dns[response,10.200.31.173:5353, length=12, id=0x0, flags=0x84f4:r:aa, questions=17623 questions: [DNSQuestion@422648395 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] [DNSQuestion@1723154136 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] etc. The log fills several GB/min.

          mirabilos added a comment -

          Just got bitten from that, too. Unfortunately, the network is now non-broken again (save for an empty resolv.conf, which did not trigger the bug), so I am uncertain as to the precise cause.

          mirabilos added a comment - Just got bitten from that, too. Unfortunately, the network is now non-broken again (save for an empty resolv.conf, which did not trigger the bug), so I am uncertain as to the precise cause.

          Lior Goikhburg added a comment - - edited

          Wow, this is craptacular!

          This has just filled up 100gig of my rootfs.

          Lior Goikhburg added a comment - - edited Wow, this is craptacular! This has just filled up 100gig of my rootfs.

          Happening to me with hudson.DNSMultiCast.disable=true
          MacOSX 10.12.2
          defaults read /Library/Preferences/org.jenkins-ci
          {
          heapSize = 512m;
          httpPort = 8080;
          "hudson.DNSMultiCast.disabled" = true;
          minHeapSize = 256m;
          minPermGen = 256m;
          permGen = 512m;
          tmpdir = "/Users/Shared/Jenkins/tmp";
          }

          Clara Mckenzie added a comment - Happening to me with hudson.DNSMultiCast.disable=true MacOSX 10.12.2 defaults read /Library/Preferences/org.jenkins-ci { heapSize = 512m; httpPort = 8080; "hudson.DNSMultiCast.disabled" = true; minHeapSize = 256m; minPermGen = 256m; permGen = 512m; tmpdir = "/Users/Shared/Jenkins/tmp"; }

          Shevek . added a comment -

          I just got 90Gb of jenkins log in 18 hours, all caused by this bug. Each time it logs, it logs one more question: line.

          Shevek . added a comment - I just got 90Gb of jenkins log in 18 hours, all caused by this bug. Each time it logs, it logs one more question: line.

          Shevek . added a comment -

          My network is generally reliable; I don't know where these DNS packets are coming from. However, I've disabled all dnsmulticast, udp and other features and only allowed internal HTTP to Jenkins. If under these circumstances, it is still possible for a UDP packet to be received by jenkins and perform a remote DOS, then this bug should be marked at LEAST critical.

          Shevek . added a comment - My network is generally reliable; I don't know where these DNS packets are coming from. However, I've disabled all dnsmulticast, udp and other features and only allowed internal HTTP to Jenkins. If under these circumstances, it is still possible for a UDP packet to be received by jenkins and perform a remote DOS, then this bug should be marked at LEAST critical.

          Shevek . added a comment -

          There is also this workaround:

           

          https://stackoverflow.com/questions/31719756/how-to-stop-jenkins-log-from-becoming-huge

           

          Although given that the size of the log messages increases over time, I now expect jenkins to take itself down from heap exhaustion rather than disk exhaustion.

          Shevek . added a comment - There is also this workaround:   https://stackoverflow.com/questions/31719756/how-to-stop-jenkins-log-from-becoming-huge   Although given that the size of the log messages increases over time, I now expect jenkins to take itself down from heap exhaustion rather than disk exhaustion.

          I've generated 80 GB of log in several minutes thanks to this issue, but I haven't found any issues with DNS connectivity during the time that Jenkins logs this.

          Why has this "Major Priority" issue gone unresolved for several years?

          Brigham Campbell added a comment - I've generated 80 GB of log in several minutes thanks to this issue, but I haven't found any issues with DNS connectivity during the time that Jenkins logs this. Why has this "Major Priority" issue gone unresolved for several years?

          Shevek . added a comment -

           [Brigham Campbell] I agree, particularly given that none of the "workarounds" prevent the issue. I simply have to kill it and reboot every so often.

          Shevek . added a comment -   [Brigham Campbell] I agree, particularly given that none of the "workarounds" prevent the issue. I simply have to kill it and reboot every so often.

          Cody Scott added a comment - - edited

          I agree it is a silly bug for a silly feature. Disabling it does work, as others have said

          I've added

          -Dhudson.DNSMultiCast.disabled=true

          to /etc/init.d/jenkins

          here

          $SU -l $JENKINS_USER --shell=/bin/bash -c "$DAEMON $DAEMON_ARGS – $JAVA $JAVA_ARGS -Dhudson.DNSMultiCast.disabled=true -jar $JENKINS_WAR $JENKINS_ARGS" || return 2

          Cody Scott added a comment - - edited I agree it is a silly bug for a silly feature. Disabling it does work, as others have said I've added -Dhudson.DNSMultiCast.disabled= true to /etc/init.d/jenkins here $SU -l $JENKINS_USER --shell=/bin/bash -c  "$DAEMON $DAEMON_ARGS – $JAVA $JAVA_ARGS -Dhudson.DNSMultiCast.disabled=true -jar $JENKINS_WAR $JENKINS_ARGS"   ||  return   2

          Wow, unbelievable, this crap has not been fixed yet. Epic!

          Lior Goikhburg added a comment - Wow, unbelievable, this crap has not been fixed yet. Epic!

          Shevek . added a comment -

          Took our Jenkins down again today.

          Shevek . added a comment - Took our Jenkins down again today.

          Took our Jenkins and gitlab (same server) down :-|

           

          78Gig of logfile.

          Sigi Kiermayer added a comment - Took our Jenkins and gitlab (same server) down :-|   78Gig of logfile.

          Patrick Rose added a comment - - edited

          Our Jenkins serer was also brought down today by this and we're using the workaround that code suggests.

          This is a definitely a critical issue - can even a temporary workaround be included in Jenkins which prevents it logging multiple times per second for this?

          The logs from when this srarted happening are below, and suggest it is a jmdns bug

          Jan 11 07:56:02.188034 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          Jan 11 07:56:02.188639 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b
          Jan 11 07:56:02.189039 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          Jan 11 07:56:02.189474 ci jenkins[4461]: SEVERE: Could not find record type for index: -1
          Jan 11 07:56:02.189846 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion
          Jan 11 07:56:02.190242 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa]
          Jan 11 07:56:02.190648 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w....
          Jan 11 07:56:02.191071 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex
          Jan 11 07:56:02.191479 ci jenkins[4461]: WARNING: Could not find record class for index: -1
          Jan 11 07:56:02.191853 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          Jan 11 07:56:02.192309 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b
          Jan 11 07:56:02.192673 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          Jan 11 07:56:02.193069 ci jenkins[4461]: SEVERE: Could not find record type for index: -1
          Jan 11 07:56:02.193464 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion
          Jan 11 07:56:02.193847 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=1
          Jan 11 07:56:02.194245 ci jenkins[4461]: questions:
          Jan 11 07:56:02.194651 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.195067 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.195485 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]]
          Jan 11 07:56:02.195869 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.196278 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.196677 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.197076 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w....
          Jan 11 07:56:02.197465 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex
          Jan 11 07:56:02.197854 ci jenkins[4461]: WARNING: Could not find record class for index: -1
          Jan 11 07:56:02.198246 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          Jan 11 07:56:02.198679 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b
          Jan 11 07:56:02.199085 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          Jan 11 07:56:02.199487 ci jenkins[4461]: SEVERE: Could not find record type for index: -1
          Jan 11 07:56:02.199887 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion
          Jan 11 07:56:02.200272 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=2
          Jan 11 07:56:02.200697 ci jenkins[4461]: questions:
          Jan 11 07:56:02.201083 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.201497 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.201877 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.202263 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]]
          Jan 11 07:56:02.202649 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.203075 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.203486 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.203923 ci jenkins[4461]: question:      [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.204321 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w....
          Jan 11 07:56:02.204793 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex
          Jan 11 07:56:02.205355 ci jenkins[4461]: WARNING: Could not find record class for index: -1
          Jan 11 07:56:02.205893 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          Jan 11 07:56:02.206434 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b
          Jan 11 07:56:02.206933 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          Jan 11 07:56:02.207489 ci jenkins[4461]: SEVERE: Could not find record type for index: -1
          Jan 11 07:56:02.207967 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion
          Jan 11 07:56:02.208512 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=3
          Jan 11 07:56:02.209005 ci jenkins[4461]: questions:
          Jan 11 07:56:02.209534 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.210043 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.210568 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.211101 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.211651 ci jenkins[4461]: [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]]
          Jan 11 07:56:02.212061 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.212485 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.212876 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.213284 ci jenkins[4461]: question:      [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.213668 ci jenkins[4461]: question:      [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.214107 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w....
          Jan 11 07:56:02.214507 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex
          Jan 11 07:56:02.214880 ci jenkins[4461]: WARNING: Could not find record class for index: -1
          Jan 11 07:56:02.215302 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          Jan 11 07:56:02.215693 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b
          Jan 11 07:56:02.216076 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          Jan 11 07:56:02.216503 ci jenkins[4461]: SEVERE: Could not find record type for index: -1
          Jan 11 07:56:02.216875 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion
          Jan 11 07:56:02.217536 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=4
          Jan 11 07:56:02.218174 ci jenkins[4461]: questions:
          Jan 11 07:56:02.218803 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.219539 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.220253 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.220976 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.221701 ci jenkins[4461]: [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.222396 ci jenkins[4461]: [DNSQuestion@1631326966 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]]
          Jan 11 07:56:02.223104 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.223808 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.224554 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.225242 ci jenkins[4461]: question:      [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.225910 ci jenkins[4461]: question:      [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.226583 ci jenkins[4461]: question:      [DNSQuestion@1631326966 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.227298 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w....
          Jan 11 07:56:02.227989 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex
          Jan 11 07:56:02.228796 ci jenkins[4461]: WARNING: Could not find record class for index: -1
          Jan 11 07:56:02.229476 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName
          Jan 11 07:56:02.230150 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b
          Jan 11 07:56:02.230916 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex
          Jan 11 07:56:02.231599 ci jenkins[4461]: SEVERE: Could not find record type for index: -1
          Jan 11 07:56:02.232270 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion
          Jan 11 07:56:02.232940 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=5
          Jan 11 07:56:02.233625 ci jenkins[4461]: questions:
          Jan 11 07:56:02.234302 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1
          Jan 11 07:56:02.234967 ci jenkins[4461]: Host: www
          Jan 11 07:56:02.235636 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
          Jan 11 07:56:02.236329 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.236995 ci jenkins[4461]: [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]
          Jan 11 07:56:02.237677 ci jenkins[4461]: [DNSQuestion@1631326966 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]

          Patrick Rose added a comment - - edited Our Jenkins serer was also brought down today by this and we're using the workaround that code  suggests. This is a definitely a critical issue - can even a temporary workaround be included in Jenkins which prevents it logging multiple times per second for this? The logs from when this srarted happening are below, and suggest it is a jmdns bug Jan 11 07:56:02.188034 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName Jan 11 07:56:02.188639 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b Jan 11 07:56:02.189039 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex Jan 11 07:56:02.189474 ci jenkins[4461]: SEVERE: Could not find record type for index: -1 Jan 11 07:56:02.189846 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion Jan 11 07:56:02.190242 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa] Jan 11 07:56:02.190648 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w.... Jan 11 07:56:02.191071 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex Jan 11 07:56:02.191479 ci jenkins[4461]: WARNING: Could not find record class for index: -1 Jan 11 07:56:02.191853 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName Jan 11 07:56:02.192309 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b Jan 11 07:56:02.192673 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex Jan 11 07:56:02.193069 ci jenkins[4461]: SEVERE: Could not find record type for index: -1 Jan 11 07:56:02.193464 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion Jan 11 07:56:02.193847 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=1 Jan 11 07:56:02.194245 ci jenkins[4461]: questions: Jan 11 07:56:02.194651 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.195067 ci jenkins[4461]: Host: www Jan 11 07:56:02.195485 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]] Jan 11 07:56:02.195869 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.196278 ci jenkins[4461]: Host: www Jan 11 07:56:02.196677 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.197076 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w.... Jan 11 07:56:02.197465 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex Jan 11 07:56:02.197854 ci jenkins[4461]: WARNING: Could not find record class for index: -1 Jan 11 07:56:02.198246 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName Jan 11 07:56:02.198679 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b Jan 11 07:56:02.199085 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex Jan 11 07:56:02.199487 ci jenkins[4461]: SEVERE: Could not find record type for index: -1 Jan 11 07:56:02.199887 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion Jan 11 07:56:02.200272 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=2 Jan 11 07:56:02.200697 ci jenkins[4461]: questions: Jan 11 07:56:02.201083 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.201497 ci jenkins[4461]: Host: www Jan 11 07:56:02.201877 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.202263 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]] Jan 11 07:56:02.202649 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.203075 ci jenkins[4461]: Host: www Jan 11 07:56:02.203486 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.203923 ci jenkins[4461]: question:      [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.204321 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w.... Jan 11 07:56:02.204793 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex Jan 11 07:56:02.205355 ci jenkins[4461]: WARNING: Could not find record class for index: -1 Jan 11 07:56:02.205893 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName Jan 11 07:56:02.206434 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b Jan 11 07:56:02.206933 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex Jan 11 07:56:02.207489 ci jenkins[4461]: SEVERE: Could not find record type for index: -1 Jan 11 07:56:02.207967 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion Jan 11 07:56:02.208512 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=3 Jan 11 07:56:02.209005 ci jenkins[4461]: questions: Jan 11 07:56:02.209534 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.210043 ci jenkins[4461]: Host: www Jan 11 07:56:02.210568 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.211101 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.211651 ci jenkins[4461]: [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]] Jan 11 07:56:02.212061 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.212485 ci jenkins[4461]: Host: www Jan 11 07:56:02.212876 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.213284 ci jenkins[4461]: question:      [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.213668 ci jenkins[4461]: question:      [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.214107 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w.... Jan 11 07:56:02.214507 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex Jan 11 07:56:02.214880 ci jenkins[4461]: WARNING: Could not find record class for index: -1 Jan 11 07:56:02.215302 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName Jan 11 07:56:02.215693 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b Jan 11 07:56:02.216076 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex Jan 11 07:56:02.216503 ci jenkins[4461]: SEVERE: Could not find record type for index: -1 Jan 11 07:56:02.216875 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion Jan 11 07:56:02.217536 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=4 Jan 11 07:56:02.218174 ci jenkins[4461]: questions: Jan 11 07:56:02.218803 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.219539 ci jenkins[4461]: Host: www Jan 11 07:56:02.220253 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.220976 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.221701 ci jenkins[4461]: [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.222396 ci jenkins[4461]: [DNSQuestion@1631326966 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]] Jan 11 07:56:02.223104 ci jenkins[4461]: question:      [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.223808 ci jenkins[4461]: Host: www Jan 11 07:56:02.224554 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.225242 ci jenkins[4461]: question:      [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.225910 ci jenkins[4461]: question:      [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.226583 ci jenkins[4461]: question:      [DNSQuestion@1631326966 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.227298 ci jenkins[4461]: 0: 474554202f204854 54502f312e310d0a 486f73743a207777 770d0a0d0a           GET./.HT TP/1.1.. Host:.ww w.... Jan 11 07:56:02.227989 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordClass classForIndex Jan 11 07:56:02.228796 ci jenkins[4461]: WARNING: Could not find record class for index: -1 Jan 11 07:56:02.229476 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming$MessageInputStream readName Jan 11 07:56:02.230150 ci jenkins[4461]: SEVERE: bad domain name: possible circular name detected. Bad offset: 0xffffffff at 0x1b Jan 11 07:56:02.230916 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.constants.DNSRecordType typeForIndex Jan 11 07:56:02.231599 ci jenkins[4461]: SEVERE: Could not find record type for index: -1 Jan 11 07:56:02.232270 ci jenkins[4461]: Jan 11, 2018 7:56:02 AM javax.jmdns.impl.DNSIncoming readQuestion Jan 11 07:56:02.232940 ci jenkins[4461]: SEVERE: Could not find record type: dns[query,222.240.222.92:48199, length=29, id=0x4745, flags=0x5420:aa, questions=5 Jan 11 07:56:02.233625 ci jenkins[4461]: questions: Jan 11 07:56:02.234302 ci jenkins[4461]: [DNSQuestion@917158197 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 1 Jan 11 07:56:02.234967 ci jenkins[4461]: Host: www Jan 11 07:56:02.235636 ci jenkins[4461]: ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.] Jan 11 07:56:02.236329 ci jenkins[4461]: [DNSQuestion@73205774 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.236995 ci jenkins[4461]: [DNSQuestion@49574096 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] Jan 11 07:56:02.237677 ci jenkins[4461]: [DNSQuestion@1631326966 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]

          Lior Goikhburg added a comment - - edited

          Just happened to me again!

          Jammed servers disk.

          What should be done to make jenkins maintainers address this issue ?

          Lior Goikhburg added a comment - - edited Just happened to me again! Jammed servers disk. What should be done to make jenkins maintainers address this issue ?

          Cody Scott added a comment -

          drugcrazed what do the relevant lines in /etc/init.d/jenkins look like? A couple of releases ago that file changed so if you took the new version you would need to add the fix again.

           

          Cody Scott added a comment - drugcrazed what do the relevant lines in /etc/init.d/jenkins look like? A couple of releases ago that file changed so if you took the new version you would need to add the fix again.  

          Patrick Rose added a comment -

          code - I can see by using ps -ef that it's definitely set up correctly.

          The sysadmins have it set up so it's done using $JAVA_ARGS instead of editing the init.d file. We're adding the udp workaround as well.

          Patrick Rose added a comment - code - I can see by using ps -ef that it's definitely set up correctly. The sysadmins have it set up so it's done using $ JAVA_ARGS  instead of editing the init.d file. We're adding the udp  workaround as well.

          Just came across exactly the same issue. 47 Gb of 'DNSQuestion' entries.

          Ivan Ponomarev added a comment - Just came across exactly the same issue. 47 Gb of 'DNSQuestion' entries.

          Patrick Rose added a comment -

          I opened a PR to at least get the ball rolling on this, but I'm not really that proficient in what jmdns is actually doing under the hood - this is just me attempting to get this fixed so jmdns could have broken all kinds of things for Jenkins ¯_(ツ)_/¯

          abayer / rtyler - now that jmdns is on github and using git instead of SVN, it might actually be sensible for that to be an actual fork with the abort method added instead of copy-dumping? Then updating might be less of a burden in general.

          Patrick Rose added a comment - I opened a PR to at least get the ball rolling on this , but I'm not really that proficient in what jmdns is actually doing under the hood - this is just me attempting to get this fixed so jmdns could have broken all kinds of things for Jenkins ¯_(ツ)_/¯ abayer / rtyler - now that jmdns is on github and using git instead of SVN, it might actually be sensible for that to be an actual fork with the abort method added instead of copy-dumping? Then updating might be less of a burden in general.

          Furai added a comment - - edited

          I'm running into similar issue right now. My syslog is getting filled with messages from jmdns

          Ubuntu 17.10 x64

          java -version
          openjdk version "1.8.0_151"
          OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.10.2-b12)
          OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

          https://file.furai.pl/2018-02-15-09-18-46.txt
          And from this point onward it just keeps looping over and over again.
          Increasing that question (queue?) by 1 entry.

          Changed my logrotate to mitigate the issue but I'd love an actual fix for this.

          Furai added a comment - - edited I'm running into similar issue right now. My syslog is getting filled with messages from jmdns Ubuntu 17.10 x64 java -version openjdk version "1.8.0_151" OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.17.10.2-b12) OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode) https://file.furai.pl/2018-02-15-09-18-46.txt And from this point onward it just keeps looping over and over again. Increasing that question (queue?) by 1 entry. Changed my logrotate to mitigate the issue but I'd love an actual fix for this.

          Terry Lacy added a comment - - edited

          I'm adding another data point for this. My logs filled up with entries similar to:

          question: [DNSQuestion@1607462970 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ]

          This is on Jenkins 2.89.4 on a Windows Server 2012 R2 VM.

          Are y'all certain that jmdns is the cause? It was not clear to me that jmdns was the source of the entries, but my logs were so large that I couldn't trace what originally triggered the flood of DNSQuestion entries.

          This filled a drive once on my VM. Our SAs, not knowing the source of the problem, enlarged the drive, which promptly filled up again (within less than an hour)!

          In the end logs were almost 100G before I stopped the server.

          This killed a couple of critical production processes.

          I disabled DNS multicast, but as I read the comments here, that does not appear to be a permanent solution. I'm also not clear on the consequences of applying these dubious "fixes". I'm very reluctant to manually patch jmdns since that presumably makes it painful to update Jenkins with security patches, etc. Especially since it's not obvious that it will solve the problem, either.

          My gut feeling, based on the comments here, is that shutting down the server to apply any of these umm...fixes temporarily resolves the issue, maybe for days or weeks, but none of the fixes permanently resolves the problem.

          Terry Lacy added a comment - - edited I'm adding another data point for this. My logs filled up with entries similar to: question: [DNSQuestion@1607462970 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: ] This is on Jenkins 2.89.4 on a Windows Server 2012 R2 VM. Are y'all certain that jmdns is the cause? It was not clear to me that jmdns was the source of the entries, but my logs were so large that I couldn't trace what originally triggered the flood of DNSQuestion entries. This filled a drive once on my VM. Our SAs, not knowing the source of the problem, enlarged the drive, which promptly filled up again (within less than an hour)! In the end logs were almost 100G before I stopped the server. This killed a couple of critical production processes. I disabled DNS multicast, but as I read the comments here, that does not appear to be a permanent solution. I'm also not clear on the consequences of applying these dubious "fixes". I'm very reluctant to manually patch jmdns since that presumably makes it painful to update Jenkins with security patches, etc. Especially since it's not obvious that it will solve the problem, either. My gut feeling, based on the comments here, is that shutting down the server to apply any of these umm...fixes temporarily resolves the issue, maybe for days or weeks, but none of the fixes permanently resolves the problem.

          Patrick Rose added a comment -

           Is it certain that jmdns is the cause? It was not clear to me that jmdns was the source of the messages, but my logs were so large that I couldn't trace where they came from.

          Yes - the first log you get makes it obvious that the is in jmdns, but it's difficult to find that when it's logging multiple times a second.

          Is there anything more I can do to get my PR looked at?

          Patrick Rose added a comment -  Is it certain that jmdns is the cause? It was not clear to me that jmdns was the source of the messages, but my logs were so large that I couldn't trace where they came from. Yes - the first log you get makes it obvious that the is in jmdns, but it's difficult to find that when it's logging multiple times a second. Is there anything more I can do to get my PR looked at?

          Terry Lacy added a comment -

          Thanks for your quick response, Patrick. It looks like there is a problem with your PR? It says "All checks have failed" and "Jenkins — Looks like there's a problem with this pull request". Looks like it broke a build?

          Terry Lacy added a comment - Thanks for your quick response, Patrick. It looks like there is a problem with your PR? It says "All checks have failed" and "Jenkins — Looks like there's a problem with this pull request". Looks like it broke a build?

          Patrick Rose added a comment -

          Considering how there's not been a commit to that repo for 7 years, I'm going to assume that the infrastructure for that Jenkins build is out of date.

          Patrick Rose added a comment - Considering how there's not been a commit to that repo for 7 years, I'm going to assume that the infrastructure for that Jenkins build is out of date.

          Patrick Rose added a comment -

          This brought down our Jenkins instance again. We're using both of -Dhudson.DNSMultiCast.disabled=true and -Dhudson.udp=-1 and this still happens.

          abayer - is there anything I can do to get the above PR looked at? I'd understand this issue not being fixed if there's no time to get the fix together but there's a PR that just needs to be merged and tested that's had no feedback for 2 months!

          Patrick Rose added a comment - This brought down our Jenkins instance again . We're using both of -Dhudson.DNSMultiCast.disabled=true and -Dhudson.udp=-1 and this still happens. abayer - is there anything I can do to get the above PR looked at? I'd understand this issue not being fixed if there's no time to get the fix together but there's a PR that just needs to be merged and tested that's had no feedback for 2 months!

          Furai added a comment -

          I'm sorry to just bump the thread but it happened few times since the last time I've posted and I haven't seen any patch addressing the issue. Is it going to be fixed?

          Furai added a comment - I'm sorry to just bump the thread but it happened few times since the last time I've posted and I haven't seen any patch addressing the issue. Is it going to be fixed?

          Patrick Rose added a comment -

          The pull request I submitted is now being looked at so it might be fixed soon.

          Patrick Rose added a comment - The pull request I submitted is now being looked at so it might be fixed soon.

          Ryan Campbell added a comment -

          drugcrazed, which pull request? I don't see it.

          Ryan Campbell added a comment - drugcrazed , which pull request? I don't see it.

          Patrick Rose added a comment -

          recampbell - this one, there's a compilation error I need to look into but I've not had the chance to yet.

          Patrick Rose added a comment - recampbell - this one , there's a compilation error I need to look into but I've not had the chance to yet.

          Furai added a comment -

          This bug have again brought my machine down. Is there any fix to it or a workaround?

          Furai added a comment - This bug have again brought my machine down. Is there any fix to it or a workaround?

          Oleg Nenashev added a comment -

          As discussed in https://github.com/jenkinsci/jmdns/pull/2, my proposal is to replace the custom library fork by the official release from https://github.com/jmdns/jmdns . All custom patches were before 2010, and I doubt they are really needed anymore

          danielbeck jglick would be nice to get your feedback here

           

          Oleg Nenashev added a comment - As discussed in https://github.com/jenkinsci/jmdns/pull/2 , my proposal is to replace the custom library fork by the official release from  https://github.com/jmdns/jmdns . All custom patches were before 2010, and I doubt they are really needed anymore danielbeck jglick would be nice to get your feedback here  

          Jesse Glick added a comment -

          Just do JENKINS-33596 already so that it becomes someone else’s problem: whoever decides to maintain that plugin.

          Jesse Glick added a comment - Just do JENKINS-33596 already so that it becomes someone else’s problem: whoever decides to maintain that plugin.

          Furai added a comment -

          But if I understood right - people who were running their war directly with

          -Dhudson.DNSMultiCast.disabled=true 
          

          and

          -Dhudson.udp=-1
          

          Were still getting the issue as per this link.  Is this somehow related? Or are this options disabling completely different things?

          Furai added a comment - But if I understood right - people who were running their war directly with -Dhudson.DNSMultiCast.disabled= true and -Dhudson.udp=-1 Were still getting the issue as per this link .  Is this somehow related? Or are this options disabling completely different things?

          The reason why the workaround doesn't work for people, is because of the bad ordering of the parameters.

          The parameters

           -Dhudson.DNSMultiCast.disabled=true -Dhudson.udp=-1
          

          are parameters to java, not jenkins, and therefore should be specified after java command but before 

          -jar /usr/share/jenkins/jenkins.war
          

          All parameters after jenkins.war are parameters to jenkins.

          So, check the process command with 

          pgrep -a java
          

          and you should see something similar to

          6916 /usr/bin/java -Djava.awt.headless=true -Dhudson.DNSMultiCast.disabled=true -Dhudson.udp=-1 -jar /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=-1 --httpListenAddress=192.168.255.1
          

          Hope that helps.

           

           

           

          Lior Goikhburg added a comment - The reason why the workaround doesn't work for people, is because of the bad ordering of the parameters . The parameters  -Dhudson.DNSMultiCast.disabled= true -Dhudson.udp=-1 are parameters to java, not jenkins , and therefore should be specified after java command but before  -jar /usr/share/jenkins/jenkins.war All parameters after jenkins.war are parameters to jenkins . So, check the process command with  pgrep -a java and you should see something similar to 6916 /usr/bin/java -Djava.awt.headless= true -Dhudson.DNSMultiCast.disabled= true -Dhudson.udp=-1 -jar /usr/share/jenkins/jenkins.war --webroot=/ var /cache/jenkins/war --httpPort=8080 --ajp13Port=-1 --httpListenAddress=192.168.255.1 Hope that helps.      

          Furai added a comment -

          What's the status on the fix to that? Recently it brought my machine down yet again. Maybe I should just containerise jenkins instance.

          Might actually add those java parameters mentioned above to prevent it from happening.

          Furai added a comment - What's the status on the fix to that? Recently it brought my machine down yet again. Maybe I should just containerise jenkins instance. Might actually add those java parameters mentioned above to prevent it from happening.

          Patrick Rose added a comment - - edited

          Might actually add those java parameters mentioned above to prevent it from happening.

          We're running with those java args, and it still happens to us (albeit infrequently)

          EDIT: Which makes me wonder if JENKINS-33596 would actually fix the issue? If we're turning off on our end, I'm not sure why we keep getting the issue once every 6 months or so.

          Patrick Rose added a comment - - edited Might actually add those java parameters mentioned above to prevent it from happening. We're running with those java args, and it still happens to us (albeit infrequently) EDIT : Which makes me wonder if JENKINS-33596 would actually fix the issue? If we're turning off on our end, I'm not sure why we keep getting the issue once every 6 months or so.

          Patrick Rose added a comment -

          Thanks to fcojfernandez, this should now be fixed in 2.178!

          Patrick Rose added a comment - Thanks to fcojfernandez , this should now be fixed in 2.178!

          Oleg Nenashev added a comment -

          We believe that JENKINS-25369 fixes it as well. Please reopen if you experience any issues

          Oleg Nenashev added a comment - We believe that  JENKINS-25369 fixes it as well. Please reopen if you experience any issues

            Unassigned Unassigned
            hellyguo Helly Guo
            Votes:
            32 Vote for this issue
            Watchers:
            46 Start watching this issue

              Created:
              Updated:
              Resolved: