multithreading - When using RabbitMQ as a java work queue, how should you handle concurrency and transient errors? -
i'm considering setting rabbitmq broker handle basic task processing java web application. basic idea producer web server wants speedy processing requests, needs data processing. accomplish this, knows job dtos can serialized amqp messages, , somewhere out there consumer knows how process these jobs. classic.
after reading rabbitmq documentation, i'm still left couple of questions.
i want consumer application utilize cpu processing messages, , i'm wondering if rabbitmq provides worker pool. assuming wanted 4 worker threads, enough register 4 channels 1 consumer each on same connection? in case, work done in handledelivery() , ack sent if completes. or rather, should use 1 consumer , manage pool of workers @ own layer of application?
consumers talking database, means transient errors occur; deadlocks, optimistic locking violations, database server restarts, , on. intended behavior if consumer can't process job? issue nack , wait broker resend job, or delay ack until it's job can't completed? or not matter?
it's worth noting jobs won't take longer few seconds unless database restarting or something. appreciate guidance!
1a. if want fixed number of working threads should open several channels , register 1 consumer per channel. rabbitmq has single dispatcher thread per channel 1 consumer block other consumers registered on same channel.
1b. if need dynamically growing pool of workers might consider using spring-rabbit (assuming implement in java). specifically, simplemessagelistenercontainer manages pool of consumers growing , shrinking dynamically according work load. refer section "listeners concurrency" in spring amqp reference guide.
2a. if know worker has failed can nack , message delivered consumer. message not delivered consumer until consumer acknowledged unless it's "auto ack" mode.
Comments
Post a Comment