Java mail listener using Spring Integration : mail isn't received by multiple app instances

Question

I'm using below code in a Springboot application:

@Bean
    public IntegrationFlow mailListener() {

        return IntegrationFlows.from(Mail.imapInboundAdapter(receiver()), e -> e.poller(Pollers.fixedRate(60000).maxMessagesPerPoll(-1)))
                .<Message>handle(message -> logMail(message)).get();
    }

    private org.springframework.messaging.Message<?> logMail(org.springframework.messaging.Message<?> message) {
        System.out.println("received a mail********** !");
        // System.out.println(message.getPayload());
        // process message
        return message;
    }

    @Bean
    public ImapMailReceiver receiver() {

        ImapMailReceiver receiver = new ImapMailReceiver(
                "imaps://username:pwd@mail.company.com/INBOX");
        receiver.setShouldMarkMessagesAsRead(true);
        receiver.setJavaMailProperties(javaMailProperties());
        return receiver;
    }

    private Properties javaMailProperties() {
        Properties javaMailProperties = new Properties();

        /*
         * javaMailProperties.setProperty("mail.imap.socketFactory.class",
         * "javax.net.ssl.SSLSocketFactory");
         * javaMailProperties.setProperty("mail.imap.socketFactory.fallback","false");
         * javaMailProperties.setProperty("mail.store.protocol","imaps");
         */
        // javaMailProperties.setProperty("mail.debug","true");

        return javaMailProperties;
    }

This springboot applications has been deployed in 2 different servers (dev and stage). Whenever this email ID receives a new mail it will be received by either application that's running on dev server or stage and not both. How do I retrieve new mails in all instances of my applications running on different servers ?

UPDATE : debug logs

DEBUG IMAPS: IMAPProtocol noop
A20 NOOP
A20 OK NOOP completed.
A21 LIST "" INBOX
* LIST (\Marked \HasNoChildren) "/" INBOX
A21 OK LIST completed.
2020-06-05 03:06:39.003 DEBUG 15568 --- [   scheduling-1] o.s.integration.mail.ImapMailReceiver    : opening folder [imaps://user.name%40company.com@mail.ad.company.com/INBOX]
DEBUG IMAPS: connection available -- size: 1
A22 SELECT INBOX
* 1438 EXISTS
* 1 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 228] Is the first unseen message
* OK [UIDVALIDITY 2278797] UIDVALIDITY value
* OK [UIDNEXT 2855] The next unique identifier value
A22 OK [READ-WRITE] SELECT completed.
2020-06-05 03:06:39.752  INFO 15568 --- [   scheduling-1] o.s.integration.mail.ImapMailReceiver    : attempting to receive mail from folder [INBOX]
2020-06-05 03:06:39.752 DEBUG 15568 --- [   scheduling-1] o.s.integration.mail.ImapMailReceiver    : This email server does not support RECENT or USER flags. System flag 'Flag.FLAGGED' will be used to prevent duplicates during email fetch.
A23 SEARCH NOT (ANSWERED) NOT (DELETED) NOT (SEEN) NOT (FLAGGED) ALL
* SEARCH 1438
A23 OK SEARCH completed.
2020-06-05 03:06:40.488 DEBUG 15568 --- [   scheduling-1] o.s.integration.mail.ImapMailReceiver    : found 1 new messages
A24 FETCH 1438 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1438 FETCH (ENVELOPE ("Thu, 4 Jun 2020 17:36:21 -0400" "test message" (("sender,name (Company)" NIL "Sender.name" "company.com")) NIL NIL (("~receiver name" NIL "user.name" "company.com")) NIL NIL "<KL1P101MB0152E33B47someid@KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>" "<KL1P101MB0152E33B4someid@KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>") INTERNALDATE "04-Jun-2020 17:36:28 -0400" RFC822.SIZE 2638 FLAGS (\Recent) BODYSTRUCTURE ("text" "html" ("charset" "us-ascii") NIL NIL "7BIT" 0 0 NIL NIL "en-US" NIL))
A24 OK FETCH completed.
2020-06-05 03:06:41.536 DEBUG 15568 --- [   scheduling-1] o.s.integration.mail.ImapMailReceiver    : Received 1 messages
2020-06-05 03:06:41.536 DEBUG 15568 --- [   scheduling-1] o.s.integration.mail.ImapMailReceiver    : USER flags are not supported by this mail server. Flagging message with system flag
A25 STORE 1438 +FLAGS (\Flagged)
* 1438 FETCH (FLAGS (\Flagged \Recent))
A25 OK STORE completed.
A26 FETCH 1438 (BODY.PEEK[]<0.16384>)
* 1438 FETCH (BODY[]<0> {3198}
Received: from ALPMBHT04.e2k.ad.company.com (3.159.19.197) by
 ALPURAPA30.e2k.ad.company.com (3.159.16.198) with Microsoft SMTP Server (TLS) id
 14.3.487.0; Thu, 4 Jun 2020 17:36:28 -0400
Received: from APC01-HK2-obe.outbound.protection.outlook.com (10.38.143.172)
 by mail.o365.company.com (3.159.19.197) with Microsoft SMTP Server (TLS) id
 14.3.487.0; Thu, 4 Jun 2020 17:36:28 -0400
Received: from KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM (2603:1096:808:3::27) by
 KL1P101MB0151.NAMP101.PROD.OUTLOOK.COM (2603:1096:808:3::26) with Microsoft
 SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.3045.25; Thu, 4 Jun 2020 21:36:21 +0000
Received: from KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM ([129.75.125.219]) by
 KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM ([129.75.125.219]) with mapi id
 15.20.3045.029; Thu, 4 Jun 2020 21:36:21 +0000
From: "sender,name (Company)" <Sender.name@company.com>
To: ~receiver name <user.name@company.com>
Subject: test message
Thread-Topic: test message
Thread-Index: AdY6s3z4MSq0ceMSRqCEtXhoAxXllAABK3Aw
Date: Thu, 4 Jun 2020 17:36:21 -0400
Message-ID:
 <KL1P101MB0152E33B47someid@KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>
References:
 <KL1P101MB0152E33B47someid@KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>
In-Reply-To:
 <KL1P101MB0152E33B47someid@KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM>
Accept-Language: en-IN, en-US
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 04
X-MS-Exchange-Organization-AuthSource: KL1P101MB0152.NAMP101.PROD.OUTLOOK.COM
X-MS-Has-Attach:
X-MS-Exchange-Organization-SCL: -1
X-Message-Flag: Follow up
X-MS-TNEF-Correlator:
Content-Type: text/html; charset="us-ascii"
MIME-Version: 1.0

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
    {font-family:"Cambria Math";
    panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
    {font-family:Calibri;
    panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
    {margin:0cm;
    margin-bottom:.0001pt;
    font-size:11.0pt;
    font-family:"Calibri",sans-serif;
    mso-fareast-language:EN-US;}
.MsoChpDefault
    {mso-style-type:export-only;
    font-size:10.0pt;}
@page WordSection1
    {size:612.0pt 792.0pt;
    margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
    {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-IN" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal">Test content<o:p></o:p></p>
</div>
</body>
</html>
)
A26 OK FETCH completed.
A27 EXAMINE INBOX
* 1438 EXISTS
* 0 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS ()] Permanent flags
* OK [UNSEEN 228] Is the first unseen message
* OK [UIDVALIDITY 2278797] UIDVALIDITY value
* OK [UIDNEXT 2856] The next unique identifier value
A27 OK [READ-ONLY] EXAMINE completed.
A28 CLOSE
A28 OK CLOSE completed.
DEBUG IMAPS: added an Authenticated connection -- size: 1
2020-06-05 03:06:44.788 DEBUG 15568 --- [   scheduling-1] o.s.i.mail.MailReceivingMessageSource    : received mail message [org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage@3904c3]
2020-06-05 03:06:44.796 DEBUG 15568 --- [   scheduling-1] o.s.i.e.SourcePollingChannelAdapter      : Poll resulted in Message: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage@3904c3, headers={id=7c50d96e-36da-b55f-9862-2215117ef304, timestamp=1591306604796}]
2020-06-05 03:06:44.798 DEBUG 15568 --- [   scheduling-1] o.s.integration.channel.DirectChannel    : preSend on channel 'bean 'mailListener.channel#0'; defined in: 'class path resource [com//mail/MailPoller.class]'; from source: 'bean method mailListener'', message: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage@3904c3, headers={id=7c50d96e-36da-b55f-9862-2215117ef304, timestamp=1591306604796}]
got a new mail
2020-06-05 03:06:44.799 DEBUG 15568 --- [   scheduling-1] o.s.integration.channel.DirectChannel    : postSend (sent=true) on channel 'bean 'mailListener.channel#0'; defined in: 'class path resource [com/mail/MailPoller.class]'; from source: 'bean method mailListener'', message: GenericMessage [payload=org.springframework.integration.mail.AbstractMailReceiver$IntegrationMimeMessage@3904c3, headers={id=7c50d96e-36da-b55f-9862-2215117ef304, timestamp=1591306604796}]
DEBUG IMAPS: IMAPProtocol noop

score 1 · Answer 1 · answered Jun 04 '20 at 19:25

1

It's because of receiver.setShouldMarkMessagesAsRead(true);.

If you change it to false, I think both will get it; the problem is then you will have to manually mark it as SEEN somehow to prevent re-fetching it next time the apps start.

You should also be able to use a custom SearchTermStrategy. The default strategy includes

if (supportedFlags.contains(Flags.Flag.SEEN)) {
    NotTerm notSeen = new NotTerm(new FlagTerm(new Flags(Flags.Flag.SEEN), true));
    if (searchTerm == null) {
        searchTerm = notSeen;
    }
    else {
        searchTerm = new AndTerm(searchTerm, notSeen);
    }
}

With a custom search term, you should also be able to use a different userFlag in each instance; enabling each to mark the message as read that way (as long as your IMAP server supports Flag.USER.

See ImapMailReceiver.DefaultSearchTermStrategy for how the default one works.

answered Jun 04 '20 at 19:25

Gary Russell

166,535
14
146
179

Tried receiver.setShouldMarkMessagesAsRead(false); still only one instance receives the new emails – VGH Jun 04 '20 at 19:39
That implies that `Flag.RECENT` is not supported by the server. In that case, the adapter falls back and uses the `Flag.USER` to mark the record (regardless of the boolean). The default user flag is `spring-integration-mail-adapter` which means it will be ignored by the default search strategy. Try setting a different flag value (`setUserFlag`) on one of your environments to see if that works. You can see all the flag activity with `mail.debug=true`. – Gary Russell Jun 04 '20 at 19:44
Tried receiver.setUserFlag("dev-flag"); and receiver.setUserFlag("stage-flag"); for dev and stage applications and still only one instance receives the email . Will paste the debug logs which keeps printing for every 1 minute – VGH Jun 04 '20 at 20:48
Unfortunately that doesn't help; I need to see a log where a message is actually received and flagged. You can redact the contents of the mail itself; I just need to see the IMAP interaction, flagging etc, including the CAPABILITY request/reply near the beginning. – Gary Russell Jun 04 '20 at 20:58
DEBUG logs for `org.springframework.integration.mail` for the same time frame would be useful too. – Gary Russell Jun 04 '20 at 21:08
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/215337/discussion-between-vgh-and-gary-russell). – VGH Jun 04 '20 at 21:26
In the logs looks like flag that'm setting through code hasn't been set however i'm using receiver.setUserFlag("local-flag"); in the code – VGH Jun 04 '20 at 22:00
https://stackoverflow.com/questions/62214324/java-mail-listner-this-email-server-does-not-support-recent-or-user-flags/62216628#62216628 – Gary Russell Jun 05 '20 at 13:37

Java mail listener using Spring Integration : mail isn't received by multiple app instances

1 Answers1

Linked