Top

Test Automation Utility to Verify Email

Test Automation Utility to Verify Email

Many applications send its users emails as part of the process flow but it’s not obvious how to use Selenium WebDriver to verify email. This is of course possible, but automating UI steps of a 3rd-party email application is not the most optimal approach. Because of this, I worked on a project once where we simply skipped the verification of the email receipt. This always bothered me, so I eventually blocked out some time to research how I might be able to automate the verification of email.

I found the JavaMail API. This is an absolute life saver! With this, I’ve been able to not only verify that an email was received, but also verify its contents, open links within the email, extract data sent (temp passwords, verification codes, etc). Someone recently asked me about email verification within automation, so I decided to open source the utility methods I’ve written that use the JavaMail API.

This is written to work for Gmail accounts. I’ve also tweaked this in past projects to work for Outlook accounts as well, so that’s definitely possible.

Downloading the Dependencies

Here are the dependencies that I add to my maven project. These go inside of pom.xml

 

Setting Email Properties

There’s quite a few properties that need to be set. It’s cleaner to put this in a properties file as opposed to directly in the code, but that would work just as well. Here’s my email.properties file which is located in a resources folder in my project.

 

Utility Class

All interactions with the email application are contained in this EmailUtils.java class. Tests can then call any method needed.

I start by defining a private Folder object which represents the current folder that we’re reading from. I also created an enum of possible folders my application’s emails may be found in. This can be expanded to any other folders that exist in the email application.

 

Next, I provide multiple constructors. All of these may not necessary in most cases, but I wanted to be as flexible as possible to allow for the use of another properties file that holds the email account authentication data, and also provide a constructor that allows you to just pass the data in.

If you’d like to store the email credentials in a properties file, you can add methods to be able to retrieve them. These are needed for the second constructor.

Here are methods to interact and/or read info from the email folder

 

Boolean methods helpful for verification

 

Parsing methods can also prove helpful. For example, if your application sends out an authorization code that you need to retrieve to verify or continue on with your testing, you can use the method below to parse it out. Here’s an example where the data you want to parse is on the same line with other text.

Example Email

Subject: Authorization Code
Dear JANE DOE:

The authorization code you requested is listed below.
Authorization code:870632
Please note that this code is temporary. If you try entering the authorization code and it has expired, you’ll need to request a new one.

 

 

There could also be the case where the data you need to parse is on its own line. In that case, the method would be slightly different.

Subject: Authorization Code
Dear JANE DOE:

The authorization code you requested is listed below.
Authorization code:
870632
Please note that this code is temporary. If you try entering the authorization code and it has expired, you’ll need to request a new one.

 

 

EmailUtils.java in its entirety

 

Using the Utility in Tests

All of the following methods are example test methods that would go in another class, such as EmailTests.java

Connect to Email

 

Example of test using verification code

 

Example of test verifying that email contains a specific text

Notice the call to getMessagesBySubject. This utility method allows you to indicate whether you only want to get unread messages or not. This is very helpful for tests that are running repeatedly. This ensures you’re not looking at an older message from a previous run.

The third argument here, 5, is the max number of messages to look through to find this email message. This also helps ensure you’re only looking at the latest messages, and also speeds up the test by limiting the search and avoiding going through the entire folder.

getMessagesBySubject returns an array of Message objects, so the test uses an index to get the first one (which would be the latest one in the email folder)…the one we want.

 

Go to link within email

The utility method getUrlsFromMessage will return a list of links contained within the email. This method can of course be used to verify the link’s existence, but coupling this with Selenium WebDriver, you can also go to the link if you need to do further verification.

 

There are many more methods in the EmailUtils class that can prove useful for your automation needs. The utility class is on Github, so can be downloaded and/or contributed to.

 

Troubleshooting

Gmail has gotten stricter about third party applications accessing their services. I’ve noticed recently, Gmail will block the authorization and you’ll see an error like this

Please log in via your web browser and then try again

 

Additionally, you may also get an email from Gmail alerting you that the API is trying to access your account and has been blocked.

To resolve this, you’ll need to change your settings within Gmail to allow access to less secure accounts. Note that this makes the account more susceptible to attacks, so I wouldn’t recommend doing this on your corporate or even personal account. Ensure you’re using an account set up just for testing which holds no sensitive data.

Angie Jones
46 Comments
  • Alister Scott

    We use a service called Mailosaur which built straight for this purpose and allows full API access to all emails and JSON objects to easily retrieve content. We found using GMail too unreliable due to Google constantly blocking and shutting down the account for API access. See my blog post: https://watirmelon.blog/2016/02/15/testing-email-in-e2e-tests-with-mailosaur/

    January 29, 2018 at 10:07 pm Reply
  • Jithin Mathew

    Hi Angie, Nice tutorial.

    I have a suggestion. A disposable web mail applications will also do the same job. We can get the message content by using their API’s. This one is an open source.

    http://www.inbucket.org.

    January 30, 2018 at 11:26 am Reply
    • Greg

      Thanks for recommending inbucket. I’ll try it out since Gmail sometimes messes up my automation workflow with their random verification of recovery info.

      February 16, 2018 at 11:48 am Reply
    • Rohit

      Hi , How do you actually send Emails to inbuket from your application . We have a application which sends emails via a service(that belongs to some security team) to whatever email we set via the UI . How then can I send emails to inbucket . I tried to ask my IT Team if they could set a autoforward rule in my microsoft exchange so that when my application sends email to me it gets routed to this local smtp server.But they are not agreeing to this ? Is there any other alternative ?

      September 6, 2018 at 6:59 am Reply
  • Luke

    I use Putbox.com which allows me to send an email to anyemail@putbox.com then hit the website itself to grab the email and get/verify whatever I need in it. It’s a nice simple, fast, and free option for verifying email.

    April 20, 2018 at 9:20 pm Reply
  • Vinodh

    Hi Angie, Nice to read this article. Email testing involves 2 step process (viz) triggering email templates for set of users from 3rd party tool & verifying the triggered email in multiple email clients/browsers which involves content check against your template source (HTML/PDF), images, links & alt-tag info. I see this covers part1 only, have you done the other part.

    We have done this end-to-end implementation, just want to know if you have any solution for alignment check visually.

    Our tech stack has BOX API for content check against source, Java and Selenium. Let me know if you have any solution.

    Regards,
    Vinodh
    @vinu015

    June 9, 2018 at 10:32 am Reply
  • siva

    What are the changes need to do in order to work with outlook?

    July 16, 2018 at 7:26 am Reply
  • Mircea Cocosila

    Hi Angie,So nice that you open source your email verification utility. You said ” I’ve also tweaked this in past projects to work for Outlook accounts as well, so that’s definitely possible. ” Could you please also open source the variant that works with Outlook?Thanks,mircea

    July 20, 2018 at 3:57 pm Reply
    • Angie Jones

      Mircea, all you should have to do is change the properties in email.properties. Mine are set for gmail but you would change yours for Outlook. Here’s a resource that shows where to get the Outlook values:

      July 22, 2018 at 6:52 pm Reply
      • Rohit T

        Hi Angie/Mircea,

        Were you able to read the email from exchange service/outlook successfully. As I am unable to that even following above steps. Can someone help me in the same. Thank in advance.
        Regards,
        Rohit
        August 29, 2020 at 8:40 am Reply
  • Veer

    Hi Angie. Thanks a lot for sharing this. Do you have the code in any repositories like Git that you can share?

    September 10, 2018 at 12:04 am Reply
  • Svetlana Petrenko

    Thank you, Angie, you saved my week. Very nice tutorial!

    September 11, 2018 at 10:50 am Reply
  • Ruchi Saklani

    Hi Angie, I am getting following error, can you please help me out – javax.mail.MessagingException: Connection timed out: connect; nested exception is: java.net.ConnectException: Connection timed out: connect at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:670) at javax.mail.Service.connect(Service.java:295) at javax.mail.Service.connect(Service.java:176) at utils.EmailUtils.<init>(EmailUtils.java:85)

    October 8, 2018 at 2:38 am Reply
    • Aathi

      Hey Ruchi,I faced the same issue and later found out to be the network issue, try configuring Fire wall rules (inbound and outbound rules) for the port which you are trying to communicate ( 993 for imap). Thanks

      January 8, 2019 at 4:12 am Reply
      • Shehani Hew

        Hi Aathi,

        I have the same problem.
        I tried to resolve it by adding mail.imap.proxy.host /port to properties file, but it didn’t work. Can you explain a little bit more about how you resolved it?
        May 22, 2019 at 9:02 pm Reply
  • Krishna Chaitanya

    Thanks!!! This article has really increase the scope of automation.

    October 9, 2018 at 12:47 am Reply
  • Dima

    Thank you so much, you’re awesome !!!

    November 21, 2018 at 7:30 am Reply
  • Bhushan

    Hi Angie, Your article is really helpful. I wanted to know why have you used SMTP instead of IMAP.Thanks

    January 4, 2019 at 12:01 pm Reply
  • Aathi

    Hi Angie,I am trying this method Go to link within email, but getting an error – java.lang.ArrayIndexOutOfBoundsException: 0, can you tell me where am i going wrong?

    January 8, 2019 at 4:11 am Reply
  • Bharath

    Hi Angie,java.lang.ArrayIndexOutOfBoundsException: 0I am getting this error inspite of having emails in the mailbox.I want know the reason am I missing on something.Thanks

    January 8, 2019 at 4:34 pm Reply
    • Angie Jones

      Make sure you’re indicating the right folder (e.g. INBOX)

      January 27, 2019 at 8:44 pm Reply
      • Rohit Vatta

        I also saw this error message. The indices is calculated wrongly. I could see count of emails as 62000 but the getTotalMessagesCount comes out as 100000;

        April 24, 2020 at 4:40 am Reply
  • Atif

    Thanks for sharing your solution Angie. Do you have or know of any similar solution for C#?

    January 22, 2019 at 9:32 pm Reply
  • Thomas Knee

    Hi Angie,In the “Connect To Email” code snippet, shouldn’t the host be pop3 or imap instead of smtp as you are retrieving incoming emails rather than sending outgoing ones, or am I totally missing something?Cheers,-Thomas

    April 24, 2019 at 12:09 pm Reply
  • Rodrigo

    Hi Angie:

    I already tried to implement your class, with no success.
    I tried to implement a public static void method that could be accessed from the test class.
    This is the method that I`´ try to implement:
    public static void getCode() throws Exception { Message email = getMessagesBySubject( “Code” , true , 5 )[ 0 ]; BufferedReader reader = new BufferedReader( new InputStreamReader(email.getInputStream())); String line; String prefix = “code ” ;}The expression: getMessagesBySubject gives me error.I want to rescue a security code from
    June 14, 2019 at 12:24 pm Reply
    • Rodrigo

      Hi Again, Angie:

      As I told you before, I need to rescue the value of a security code taken from a Mailtrap mail and pass that value to a String. I want something like invoke a public static void from the test class calling like: Emailutils.getCode();
      This is my code (I have errors in the getMessagesBySubject:
      public static void getCode() throws Exception { Message email = getMessagesBySubject( “Code” , true , 5 )[ 0 ]; BufferedReader reader = new BufferedReader( new InputStreamReader(email.getInputStream())); String line; String prefix = “code ” ;}Can you please help me.
      June 14, 2019 at 12:31 pm Reply
    • Angie Jones

      can you send me an example of the email you’re trying to test and tell me the code you’re trying to extract out?

      June 21, 2019 at 10:31 am Reply
  • satish parimi

    Hi Angie,

    Your article is helping a lot to check inbox for email notification testing. We are unable to login with username and password and google is blocking through script.
    Can you suggest any work around for this.
    [RemoteTestNG] detected TestNG version 6.14.3
    javax.mail.AuthenticationFailedException: [AUTHENTICATIONFAILED] Invalid credentials (Failure)
    at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:732)
    at javax.mail.Service.connect(Service.java:

    August 16, 2019 at 5:46 am Reply
    • Viji

      Please look at troubleshooting section.

      To resolve this, you’ll need to change your settings within Gmail to allow access to less secure accounts . Note that this makes the account more susceptible to attacks, so I wouldn’t recommend doing this on your corporate or even personal account. Ensure you’re using an account set up just for testing which holds no sensitive data.
      October 16, 2019 at 11:56 pm Reply
  • Radha Reddy
    IAM GETTING RESOURCE EMAIL PROPERTIES NOT FOUND ERROR- PLEASE HELP
    java.io.FileNotFoundException: resourcesemail.properties (The system cannot find the file specified)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at test.integration.app.EmailUtils.<init>(EmailUtils.java:76)
    at test.integration.app.EmailTests.connectToEmail(EmailTests.java:22)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    January 6, 2020 at 2:10 am Reply
  • Pipul

    Angie while I’m retrieving the email I’m not getting the exact content. When extracting I’m getting amp sign as well I’m getting extra character when using the same function. Can you tell me a solution for this?

    January 21, 2020 at 3:40 am Reply
  • Asma Razaq
    Hi,
    Can we automate ‘sms noification, like email?
    April 7, 2020 at 8:14 am Reply
    • Angie Jones

      Probably with something like Twilio

      April 7, 2020 at 8:33 am Reply
  • Omprakash

    Good solution, however Google’s OAuth2 is better due to Google’s security restrictions that might break automation. I believe even for OAuth2, you have to authenticate once manually in order to make it work.

    The challenge is when you run this in CI/CD pipeline. Usually the servers are Linux based. There can be practical limitations around authenticating the OAuth2 project manually on these servers.
    To get rid of this, I’m using our internal e-mail server to work with e-mail accounts pro-grammatically (create, fetch, delete, etc.) and it works like charm in CI/CD too.
    I’m wondering if there is an open source e-mail server available for use to general public. I would like to explore that option as well for my personal projects.
    April 10, 2020 at 10:14 pm Reply
  • Dung

    This code is great but I have a question. I’m realizing that this code gets latest emails by the first email in the list of email. However when I want to get the latest message from emails that are sent by only one person, it gets the first email, this is not my expectation. I want to get the latest one. How can I get the latest one in this case?

    May 30, 2020 at 5:35 am Reply
  • Jeff
    This is a rare QA article with usable Java code!
    There are fake email testing services which are more reliable than gmail – which limits daily interactions, inbound and outbound mail, and can hide messages marked spam.
    For example, Mailinator, Mailsac, Mailsaur, trashmail, etc. These are disposable email providers. Certain disposable email services are better or worse for QA. Some have rest APIs so you don’t need to connect via SMTP.
    Often, ISPs or hosting providers block SMTP ports, so using a REST API to check email is more r
    June 1, 2020 at 10:41 am Reply
  • Oleksandr Iavgel

    Hello Angie,

    I am getting this error when trying to connectToEmail ( ) with my Gmail creds:
    java.lang.NoSuchMethodError: com.sun.mail.util.TraceInputStream.<init>(Ljava/io/InputStream;Lcom/sun/mail/util/MailLogger;)V
    Please assist – thank you a lot in advance!
    June 8, 2020 at 12:13 pm Reply
    • Maria Feier

      Hello,

      I faced the same error as you, and it turned out I had the dependency javax.mail-api twice. I had an older version and the newer one 1.5.5.
      I excluded the older dependency, and the error was no longer displayed.
      June 17, 2020 at 2:57 am Reply
  • Asserted

    Hi Angie,

    Thanks for the great write-up. Does your setup here cover waiting for emails?
    Like if you send a request and have to wait x seconds for a mail to arrive and AFTER that do the actions above like get subject and so on?There is a MessageCountListener for MessageCountEvent. Any examples in one of your repos that would be usefull?
    June 19, 2020 at 12:16 pm Reply
  • Vivek Rathod

    Awesome! Thanks for writing this Angie, very helpful article

    June 30, 2020 at 3:44 pm Reply
  • Gummadi Kalyan

    Hi Angie,

    your code really helpful but I am getting below error.
    java.lang.AssertionError: Software caused connection abort: recv failed
    Failing at store.connect(server, username, password);
    Unable to establish the connection. Please give me solution for this.
    July 6, 2020 at 1:39 pm Reply
  • Anisha Raju

    Could you please share this through your Youtube channel. So that we can visualize each steps

    August 10, 2020 at 11:09 pm Reply
  • David Tran

    Hi Angie,

    My system : login and verify code by gmail, I’m applied your code but appeared this error :
    TC_03_CheckLoginWithValidIPIStaffAccount(ipi.script.TestScriptForTechOffersAndTechNeeds) Time elapsed: 18.684 sec <<< FAILURE!
    java.lang.NullPointerException
    at ipi.script.TestScriptForTechOffersAndTechNeeds.TC_03_CheckLoginWithValidIPIStaffAccount(TestScriptForTechOffersAndTechNeeds.java:103)
    Failed tests:
    TestScriptForTechOffersAndTechNeeds.TC_03_CheckLoginWithValidIPIStaffAccount:103 NullPointer
    August 19, 2020 at 9:09 pm Reply
  • Bhavesh Soni
    Message email = emailUtils.getMessagesBySubject(“Fwd: App password created”, false, 100)[0];
    String link = emailUtils.getUrlsFromMessage(email, “recently used devices”).get(0);
    System.out.println(link);
    Message email2 = emailUtils.getMessagesBySubject(“Bem-vindo a PodeSubir”, false, 100)[0];
    String link2 = emailUtils.getUrlsFromMessage(email2, “Verificar email”).get(0);
    System.out.println(link2);
    For first link I am getting correct output however, for 2 email I am getting below error:
    java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
    at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
    at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
    at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
    at java.base/java.util.Objects.checkIndex(Objects.java:372)
    at java.base/java.util.ArrayList.get(ArrayList.java:459)
    at utils.EmailTests.testLink(EmailTests.java:38)
    September 15, 2020 at 10:50 pm Reply
  • Manoj

    I used the below code to login in to my gmail account but getting the exception javax.mail.AuthenticationFailedException: [AUTHENTICATIONFAILED] Invalid credentials (Failure) however when i try to login manually it is working fine.

    Note:- I have replaced my password with ***** here but in real time i am giving my correct password .
    emailUtils = new EmailUtils(“myemail@gmail.com”, “********”, “smtp.gmail.com”, EmailUtils.EmailFolder.INBOX);
    October 9, 2020 at 10:34 am Reply

Post a Reply to Vivek Rathod Cancel Reply