Skip to main content

Gmail internal application, Two-Legged OAuth2, Server to Server authentication, and Google API versions

I am working on a little tool at home in my free time to put some skills into practice.  The general idea (nothing novel) is this:

I have some financial alerts sent to a new email address I have spun up on my domain.  I am creating an AWS Lambda that will wake up on an hourly schedule, read those emails, and publish SNS messages with parsed financial transaction information.  I then will have an SQS queue listen to the SNS message topic that is consumed by a Step Function.  The Step Function will:

  • store the financial transaction information into a database
  • send an SMS to me if the transaction is above a certain threshold.
I could later extend this to do some aggregation reporting, etc if I wanted, too.  This will only work for my own financial transactions, and the data being gathered/stored will be sufficiently vague, so I am not really concerned about financial security for this project.

The biggest hurdle I have run into so far is connecting to Gmail securely.  I was able to quickly connect with JavaMail after I turned off enhanced security for the account.  I don't want to keep that basic authentication in place long term, though, so I started working on how to connect securely using OAuth2.  This is not a typical OAuth implementation for a couple of reasons:
  1. A cron job-like task will check the email account.  This means there will be no pop-up like authentication and authorization screen that google can provide.
  2. The email account is never meant to be checked manually; it is really a poor man's queue for purposes of this project.  That means I only want the app I am building to manage the email; no manual mailbox handling should be necessary.
To elaborate on the hurdle some more... OAuth2 is not new, so this is not a new problem.  So why was this such a hurdle?  The answer is that there is a lot of documentation and examples out there, and much of it is very out of date.  So finding the right library with the right examples took way more time than actually writing the final code that worked.

The final solution turned out to require the following:
  • Create a "Project" in Google's dev console.  Make note of the project name - you will reference it in the code.
  • Generate a "Service Account" allowed to manage the Project in Google's dev console.
  • Generate a JSON key for the service account that is allowed to interact with the Project.
  • Set the Service Account as an Owner role for the Project.
  • Modify Domain Wide Delegation for the JSON key's client Id to have access to desired scope -- particularly "https://mail.google.com/" because the service account should be allowed to do anything in this case.  The name of the API Client for delegation can be anything you want - it is just a useful description of the thing we are creating, we don't have to reference the name from what I can tell.  The client Id is the important part, and this value can easily be found inside the JSON key file.
Once all of this is done, we can write the code.  

import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport
import com.google.api.services.gmail.Gmail
import com.google.auth.http.HttpCredentialsAdapter
import com.google.auth.oauth2.ServiceAccountCredentials
import net.inherency.finances.jsonFactory
import net.inherency.finances.vo.Email
import org.apache.commons.configuration.EnvironmentConfiguration
import java.time.Instant
import java.time.LocalDateTime
import java.time.ZoneOffset


class GmailOauth2MailServiceImpl(
        private val env: EnvironmentConfiguration
): MailService {

    override fun listMessages(): List {
        val gmail = login()
        val list = gmail.users().Messages().list(userId()).execute()
        return list.messages.map { msg->
            val message = gmail.users().Messages().get(userId(), msg.id).execute()
            val messageDate = LocalDateTime.ofInstant(
              Instant.ofEpochMilli(message.internalDate), ZoneOffset.UTC)
                .toLocalDate()
            val fromAddress = message.payload.headers
              .first { it.name == "From" }.value
            val subject = message.payload.headers
              .first { it.name == "Subject" }.value
            Email(messageDate, fromAddress, subject)
        }
    }

    private fun login(): Gmail {
        val serviceAccountCredentials = ServiceAccountCredentials
                .fromStream(credentialsKey().byteInputStream())
                .createScoped(gmailScopes())
                .createDelegated(userId())

        serviceAccountCredentials.refreshAccessToken()

        return Gmail.Builder(
                GoogleNetHttpTransport.newTrustedTransport(),
                jsonFactory(),
                HttpCredentialsAdapter(serviceAccountCredentials))
                .setApplicationName(googleApplicationName())
                .build()
    }


    private fun googleApplicationName(): String {
        return env.getString("app_name")
    }

    private fun userId(): String {
        return env.getString("email_login")
    }

    private fun gmailScopes(): List {
        return listOf("https://mail.google.com/")
    }

    private fun credentialsKey(): String {
        return env.map["auth_json"].toString()
    }
}

Comments

Popular posts from this blog

JHipster, Liquibase, MySQL, and initializing data, including booleans!

When generating a data model from JHipster JDL, we will often declare entities with Boolean fields.  I have so far abandoned H2 as a database because of liquibase issues, and both my dev and production databases will be MySQL.  This is relevant to the Boolean field desire there is a long history in software development of how to store Boolean data types in a SQL database whose standards classically do not support Boolean. In the current JHipster/Liquibase incarnation, tables in MySQL are generated for us, which is really nice.  The Boolean data types are stored as BIT  (1).  This is not a problem so far -- most developers seem to agree now that as a best practice, we should store values in databases as false = 0 and true = 1, and a BIT(1) is a great, simple way to do that. An issue arises when we try to use liquibase to set/update our database to the desired starting state.  For my project, I've chosen gradle instead of maven as a build tool, and gradle has a plugin for liquiba

Deploying Spring Boot talking to MySQL on AWS

In a recent post, I listed some very basic information about running a MySQL database on AWS.  In most cases, we don't want a database alone; we want an application that uses that database for CRUD. I've created a simple Spring Boot application that exposes a REST API to create and manage lists of things.  The list values are all stored in a MySQL database. When I went to deploy the application on AWS using Elastic Beanstalk, there were some really good, automatic things that happened to make my life easy: AWS can deploy a Spring Boot jar very easily by simply uploading the jar during setup. AWS creates security groups on the fly so I don't have to worry about extra security configuration. AWS automatically generates DNS information and provides me a URL for accessing the application. As I deployed the application and saw all of these things, I was pretty excited.  It is nice to have a lot of this stuff taken care of for me. Then, I tried to test my applica

Spring Security - Authority vs Role

I have spent a lot of time recently trying to understand the difference between Authority and Role in Spring Security.  This is a brief review of what I found. When creating a UserDetailsService or overriding configure(AuthenticationManagerBuilder auth) in the security config class that extends WebSecurityConfigurerAdapter, I basically get complete control over what I populate inside of the UserDetails that is used/returned.  This is important because the UserDetails interface really only cares about how to return one thing: Collection<? extends GrantedAuthority> getAuthorities(); A GrantedAuthority just seems like a glorified String wrapper that names some thing.  The question is... what is that thing? This is where the subtle difference between Authority and Role comes into play. I think that Role is an older thought/construct that automatically gets plugged into Authority if we just create a user with a Role.  But completely forget about the code and classes for a mi