Kafka Pausing partition and taking long time (10 min) in resuming it again

I am using spring kafka with following configurations :

props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
environment.getProperty("kafka.url"));
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
KafkaAvroDeserializer.class);
props.put("specific.avro.reader", true);
props.put("schema.registry.url",
environment.getProperty("schema.registry.url"));
props.put("session.timeout.ms", 120000);
props.put("request.timeout.ms", 300000);
props.put("max.partition.fetch.bytes", "10240");
props.put("max.poll.interval.ms", 120000);
props.put("max.poll.records", 100);


And

@Bean
public ConcurrentKafkaListenerContainerFactory<String,
List<AlertNotificationPayload>> kafkaListenerContainerFactory()
ConcurrentKafkaListenerContainerFactory<String,
List<AlertNotificationPayload>> factory = new
ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(consumerFactory());
factory.setConcurrency(10);



and as soon as I start the server, the consumers start consuming the messages but after a while, few partitions gets paused and doesn't resume for almost 9-10 min. The processing time in the consumer thread is in milliseconds and have separate thread for processing. I have tried multiple combinations of configurations but nothing worked.

I have also tried to use the setting factory.getContainerProperties().setPauseEnabled(false);
which stops the pause/resume but than the rebalance starts happening after some time and sooner and later the consumer gets died and after some time the consumer group has no consumer left.

I am using following versions :

<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>1.1.3.RELEASE</version> //tried latest one as well
</dependency>
<dependency>
<groupId>io.confluent.maven</groupId>
<artifactId>kafka-connect-quickstart</artifactId>
<version>0.10.0.0</version>
</dependency>
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-avro-serializer</artifactId>
<version>3.2.0</version>
</dependency>


Please help.

1.3.x

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.