Re-design topic, groupID model for the current implementation.

          [JENKINS-51942] Improve Kafka producer-consumer topic model

          Pham Vu Tuan created issue -
          Pham Vu Tuan made changes -
          Epic Link New: JENKINS-51943 [ 191430 ]
          Pham Vu Tuan made changes -
          Priority Original: Major [ 3 ] New: Minor [ 4 ]
          Pham Vu Tuan made changes -
          Description New: We need to:
           * Investigate on consumer pooling, NIO options.
           * Study Kafka to re-design topic, groupID model. For the current model, we need different topics for different master-agent connections, which I am not sure is good enough.
           * Jenkins admin should have opportunities to login to Kafka admin portal to do admin work such as filtering of topics, events, connectivity issues, ...
           * Investigate on tools like [https://github.com/yahoo/kafka-manager] then incorporate into Docker compose.
          Pham Vu Tuan made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          Pham Vu Tuan made changes -
          Status Original: In Progress [ 3 ] New: In Review [ 10005 ]
          Pham Vu Tuan made changes -
          Description Original: We need to:
           * Investigate on consumer pooling, NIO options.
           * Study Kafka to re-design topic, groupID model. For the current model, we need different topics for different master-agent connections, which I am not sure is good enough.
           * Jenkins admin should have opportunities to login to Kafka admin portal to do admin work such as filtering of topics, events, connectivity issues, ...
           * Investigate on tools like [https://github.com/yahoo/kafka-manager] then incorporate into Docker compose.
          New: Re-design topic, groupID model for the current implementation.
          Pham Vu Tuan made changes -
          Summary Original: Improve Kafka producer-consumer model to ensure reliability New: Improve Kafka producer-consumer topic model
          Pham Vu Tuan made changes -
          Resolution New: Done [ 10000 ]
          Status Original: In Review [ 10005 ] New: Resolved [ 5 ]

            pvtuan10 Pham Vu Tuan
            pvtuan10 Pham Vu Tuan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: