Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52870

Make Zookeeper optional for ad hoc topics

    XMLWordPrintable

Details

    Description

      We may add another option which works without Zookeeper.

       

      One suggestion I have is that you might want to remove the requirement of connecting to ZooKeeper, which transitively means you can’t create topics on the fly. Many people, me included, will have locked down Kafka instances with zk totally hidden from us, so creating ad hoc topics just isn’t possible.
       
      After spending almost no time looking at it, I’m guessing you could set a header to whatever you had the ad hoc topic named and then do a if (header[‘agentName’] != ‘myAgentName’) return;
       
      If you’re worried about it being too chatty, you could shard it and create topics jenkins-remote-01 -> jenkins-remote-99 and then hash the agent name into a number to determine which topic to use (you’d still have to filter, but you’d have significantly fewer “false messages”).
      

      Attachments

        Activity

          People

            pvtuan10 Pham Vu Tuan
            pvtuan10 Pham Vu Tuan
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: