Auth and OAuth in Node/React App

This is a high-level summary of how I got auth and oauth working on my node/react app, Quizipedia. My goal was to offer a local login system (where a user could use their email and a custom password) and also a social login system (where a user could use their google and/or facebook credentials to log in – aka OAuth).

Quick summary of the app. There are 2.5 parts. First, there is a Node API sitting at the backend which connects with a postgres database. From an auth point of view, this is where user info gets stored, including hashed passwords, and it can be accessed using various calls to the API. This is also where all the calls and logic for the rest of the app functionality lives, but we aren’t concerned with that today. Above the API is the node-react-app. This was created with create-react-app, which is why it counts for 1.5 parts: It has both server code and client code, but it is in the same project, and when it is deployed to production (using Heroku), they get combined into one piece. Ergo 1.5 parts. In this post I will be discussing the server parts.

Backend API

The auth-related code in the backend API is probably the simplest to understand. It is just a set of various calls to get and post information about a user, such as:

  • Get user by id (This is a unique GUID generated in the code)
  • Get user by email
  • Get user by social network and token (More on this later)
  • Post login credentials in an attempt to login
  • Post a new user
  • Post a new social user

So there are two resources working here: the user and the social user. Someone may only want to use the local login system without any ties to a social network. This means they will just get an entry in the ‘users’ table, which contains the unique GUID, their email, a hashed password, and join_date – fairly minimal for now. If they decide later that they want to attach a social network, then they will get an entry in the ‘social_users’ table, which contains a foreign key for the user’s unique id, the name of the social network (facebook or google), and the social network token. This token is provided by the social network itself, and uniquely identifies the person’s social network account. I will go into more detail in the next section. So the API itself is fairly straightforward. It is mostly CRUD operations with a small amount of business logic. This includes verifying if a user already exists, as well as verifying login credentials.

React app – Server part

Note that this part does not contain any React code. All the React code is in the client part of the React App. The server part is similar to the backend API in that it uses Node and Express. The difference is that the backend API uses ES6 conventions and uses Babel to transpile it. I did not do the same on the Server part of the React app due to the extra complexity that would be involved in fiddling with some of the built-in functionality that comes with react-app. This was fine because, as a whole, there is much less code in this part than the backend API. In fact, the only major code in here relates to auth itself, and there are three parts. The handlers, the thin library layer, and the passporting. The handlers represent the auth endpoints that can be called right from the app, such as ‘login’, ‘signup’, ‘social login’, and ‘logout’. It also contains a few special handlers for dealing with the social logins, which we’ll get to. The thin library is essentially a wrapper for calling the backend API – For each auth-related API call mentioned above, there is a function in the library to just call the api and return the data – no mapping or filtering – just calls the API. The interesting section is the passporting. This file contains the main auth logic, for both local login and social login. The passport.js documentation is invaluable here, so I won’t repeat any of it. The basic idea is that once you add passporting to your project (done in the index.js file of this part of the project), you just define various middleware authentication functions that get called on the auth handlers.

For example, here is what the login function looks like:

Here is the handler:

app.post(
'/auth/login',
passport.authenticate('login', {
  failureRedirect: '/auth/loginfail',
  failureFlash: true,
}), (req, res) => {
  res.status(200).send(req.user);
});

So the passport ‘login’ function acts as a middleware for the handler. If the login is successful, it just returns the user, which gets added to the req object. If it fails, then it sends a message (using the connect-flash library) and redirects to a failure route. Here is the middleware function:

const passport = require('passport');
const LocalStrategy = require('passport-local').Strategy;
const keys = require('../config/keys');
const authLib = require('./authLib');

...

passport.use('login', new LocalStrategy(
{
  usernameField: 'email',
  passwordField: 'password',
  passReqToCallback: true,
},
(req, email, password, done) => {
  return authLib.getUserByEmail(email).then((user) => {
    authLib.login(email, password).then((login) => {
      if (login) {
        return done(null, user);
      }
      return done(null, false, { message: 'Incorrect password' });
    }).catch((err) => { throw err; })
  }).catch(err => done(null, false, { message: 'Email address not found' }));
}));

So this ‘use’ function acts as a middleware when the login handler is called. The LocalStrategy is another library that must be imported from passport.js. So as you might expect with a login call, the first step is to check if the user’s email exists. This is done by the backend API via the auth lib. If it does not, then return a message saying it does not exist. This will be displayed to the user. If the email does exist, then the login credentials are again sent to the backend API, where the password is hashed and compared with the stored hashed value (uses the bcrypt library). Again, if this fails, the appropriate message is sent to the user. If it succeeds, then the user gets returned in the ‘done’ function, which is a convention of passport.js.

The social logins are a little bit trickier. We’ll use Google as an example. In order to get an app to use Google’s authentication, you need to add your app on Google’s developer console. The first step is to define a handler that will redirect to Google’s login acceptance page. You also need to specify the scope of the data you want. All I care about for this application is the profile and the email address. The profile is required to get the unique profile id for a user so that it can be stored on our db to make sure that we know the user has used google’s social login on the app before. We also need to define a callback handler, which is what we tell Google to redirect to once the user has accepted on the Google login page. The handlers themselves are very straightforward:

app.get('/auth/google', passport.authenticate(GOOGLE, {
  scope: ['profile', 'email'],
}));

app.get('/auth/google/callback', passport.authenticate(GOOGLE), (req, res) => {
  res.redirect('/');
});

The authentication middleware for the Google login is a bit more involved:

passport.use(GOOGLE, new GoogleStrategy(
{
  clientID: keys.googleClientID,
  clientSecret: keys.googleClientSecret,
  callbackURL: '/auth/google/callback',
},
(accessToken, refreshToken, profile, done) => {
  const token = profile.id;
  const email = profile.emails[0].value;
  return socialStrategy(GOOGLE, token, email, done);
}));

The Google strategy from passport.js requires the app Id and secret, both of which are created when you add your app to the Google developer console. The callback URL is the one we mentioned above. So this function will redirect your browser to Google’s login screen, where the user confirms their login with a Google account. It returns some standard data, but we’re interested in the profile, which contains the unique Id and the email address, both of which we will store. The socialStrategy function is used so that we can reuse functionality for the Facebook login. Here it is:

function socialStrategy(network, token, email, done) {
  let userId;
  let message;
  return authLib.getUserBySocialNetworkAndToken(network, token)
    .then(existingUser => done(null, existingUser))
    .catch((err) => {
      if (err.response.status !==404) {
        return done(err);
      }

      // if email already exists, then associate; Otherwise add new
      return authLib.getUserByEmail(email).then((checkEmail) => {
        ({ userId } = checkEmail);
        message = 'associated existing';
        return authLib.linkSocial(userId, { network, token }).then(() => {
          const result = {
            userId, network, token, message,
          };
          return done(null, result);
        });
      }).catch((err) => {
      if (err.response.status !==404) {
        return done(err);
      }
      return authLib.createSocialUser({ network, token, email })
        .then((newUser) => {
          ({ userId } = newUser);
          message = 'created new';
          const result = { userId, network, token, message };
          return done(null, result);
        }).catch(err => done(err));
      });
   });
}
This checks if the user has previously logged in with Google by comparing the token and network with what is in the db. If it’s found, then we’re done. If not, then we check if the user has a local login with the app. If a user has previously signed up locally with the same email that is tied to their social account, then there will be a conflict. This ensures that the social login is just LINKED to their local login user id, rather than create a new one. So if they have an email, then the account is linked. If they do not, then it creates a brand new user.
Once all of this is done, passport adds the user to the req object, and redirects to the specified callback. In our case, this just redirects to the root path. So the user is now logged in!
Advertisements

Using React

I finally was able to make a significant change to the app that I’ve been wanting to do for a while. I’ve separated out the REST api and the web views into separate repositories. I’ve wanted to do this since the beginning, so I kept it in mind when writing the app, and it turned out to be fairly simple to do. I had to copy a few view-related folders over, create a separate heroku app, and make sure that everything could connect. This went fairly smoothly and in the process I was able to organize a lot of code on the REST api project.

When  started the app, my goal was just to get something working from front to back, so I made the decision to not waste time trying to learn a new technology, and just went with standard html and jQuery on the front end. Now that I’ve successfully put that out, I want to move to a more modern technology, namely React. I am currently taking a few Udemy Courses related to React to get a good feel for it. The first is a straightforward intro to React and Redux. I’m not too sure what Redux is at this point, but hopefully I’ll have some idea soon, and I might be able to make use of it. I find the most difficult part of development not to be the languages themselves, but rather the dev ops – i.e. putting everything together to make it all work. Because of this, I’ve also started a Udemy course that teaches full stack development using Node and React. Based on the first few sections of these, I think they will be more than enough information for me to get going.

My plan is to create a completely new front end repo (quizipedia-react), which will call the quizipedia REST api, just like the separate html project is now doing. Once it is ready to go, I can deploy it to Heroku and deprecate the html project, although I’d like to keep it up to date provided it doesn’t take too much work.

So for October, my plan is to complete the switch from the standard html/jquery project to the react project for front-end. I’m hoping to get through the bulk of the two courses – or at least learn as much as I need to accomplish the first task Once this is done, I have a backlog of a few features I’d like to add to the project as a whole. This includes adding user accounts, new sources for the quizzes,  and custom quizzes that can be shared. I’m hoping to add these features throughout November. Then in December I’d like to create an admin site and also start looking at React Native for a mobile version of the application.

Wikipedia integration

I’ve added integration with wikipedia into the web app. So it is now possible to run a wikipedia search, get a list of possible articles based on your search term, and then choose one of the those articles to build a quiz out of.

I used an npm module called wikijs– This package allows you to run keyword searches against wikipedia, and also get article content in plain-text format, which is exactly what I needed. As a bonus, it also allows getting a random article, which is something I will likely incorporate in the near future. Another future development with the wiki integration is to be able to search various mediaWikis, such as the Simpsons Wiki, Star Wars wiki, etc. There is a fairly straightforward api that already exists for it, so it shouldn’t be too difficult.

But now I want to focus on cleaning up the code behind the scenes instead of adding new features. I’ve decided that my next big task is going to be to split the project up into two separate projects – one will be the RESTful service and another will be the web application. This was part of my original design, and I knew I had to do it eventually, and I think now is the right time to do it. The code base is getting larger, but more importantly, I have a very good idea of which processing belongs on the client and which belongs on the server. For example, I didn’t know where it made more sense to build the word blank. I started it on the server, then had to move it to the client, but then found I could move most of it back to the server. It would’ve taken a lot of time to tinker with this if I started with two separate projects. But now that the contracts are fairly well-established, having separate projects should make things a lot smoother without much hassle.

I don’t have a lot of experience with front-end design outside of jQuery, so I’m going to be exploring some of the big names to see which makes most sense (ember, backbone, react, angular…). I’m hoping to have this part done by the end of September. I will also likely be doing a related Udemy course alongside it.

Non-Quizipedia update

Apart from Quizipedia, I’ve also had a few other things on the go related to programming. A big one is that I’ve started doing courses on Udemy. The first one I tried was ‘Javascript: The Weird Parts’. This course digs into the guts of javascript and gives a great explanation for what it really is, and compares it to other popular programming languages. I’ve been working with javascript professionally for about a year and a half, but I really felt that I needed a better understanding of the fundamentals, and this really helped. I’ve found a big list of courses that I’m interested in taking, so I will be making updates about those as well.

I had wanted to start looking at some open-source projects. I found a few that might be interesting, but I still haven’t got a chance to contribute. Most of my “dev” time has been going towards quizipedia and the udemy course, leaving no time for an open-source project. Once I complete a few of these courses I will have another look.

Quizipedia Update

It’s been a while since I’ve posted, but I’ve been working hard on a few different aspects of Quizipedia, as well as some other minor tech ventures.

Other the past couple months, I’ve reworked the code quite a bit, although the main ideas and technologies are still intact. I’ve improved the look and controls significantly – making it nicer to look at and easier to use. I’ve made the game itself a lot easier. The first letter of a word is always given, as well as the number of letters in the word. Other new functionality includes the option to ask for a hint if you have enough points, or even solve the clue.

I’ve made some minor trial-based tweaks to the relevant word algorithm, but most importantly, I’ve integrated a dictionary into the relevant word finder. There are quite a few options out there for npm packages, but I was actually surprised to learn that a lot of the big name dictionary apis are not totally free. I found an open-source one, but it seems to be lacking a little (jupiter isn’t considered a word!), so it’s an area that can still use some improvement. I’d like to find one that uses a greater variety of proper nouns. I may even try to integrate a wikipedia or wiktionary api for the word-checking. I just wanted to try one out, so I picked a simple one, and now I can test it, and improve as needed.

Some of my goals for September include the following:

  • Integrate links (wiki, tv tropes, etc.) that will read and parse html to build quizzes
  • Add a way to generate a random/sample test, possibly from wiki or tv tropes
  • I’d like to start logging info on the relevant word scores. This would include details on each of the scoring areas for a word. Once I start gathering this data, I can use it to try and improve the algorithm. This might end up going somewhere like Elasticsearch, which I want to start learning more about
  • I am interested in starting up a new project in the near future, so I want to get quizipedia in a state where it is very easy to jump into and out of to make adjustments. This was a goal from the start, and I’ve followed it quite closely, but there are a few improvements to be made

 

Quizipedia: Writing tests

I’ve started writing some tests for Quizipedia, and I’m seeing some advantages that a test-driven approach may have had. In order to write proper tests, I’m being forced to refactor a little bit of the code. It’s not too bad, since I WAS keeping the fact that I’d need to write tests at the back of my head while I was writing the code. My main goal with this was to get SOMETHING out and working, which I was able to do. Now I get to do all the clean up. Now that I’ve scratched the itch to actually get something working, any projects I work on in the future will be test-driven (famous last words?)

Anyways, I started with the basic tests for some of the main library functions in the game (creating/getting/deleting a game and also guessing answers). I am using the chai and mocha framework to write my tests, and so far it satisfies the basic functionality that I’m looking for. Those tests were fairly straightforward, so I’ve moved on to testing the actual game-building. This includes things like scoring a given word and deciding if a word is relevant. Writing the tests allowed me to make an important refactor in the code:

Previously I had a single function that would determine the score of a word. This was a class function, since the score depended on other properties of the gameBuilder object, for example the distance to the PREVIOUS relevant word. I wanted to test the score of a word without this dependency, so I added a STATIC relevance check, which is now part of the entire relevance check. This will give greater flexibility when I integrate some more powerful word scoring into the game, so I’m glad I caught this early. I was also able to do a tiny bit of tweaking/improvements on some of the word scoring while writing the tests.

So the two library test suites are the Basic Game functionality (done) and the Game Building functionality (in progress). Once these are done, I am going to move on to writing tests for the DAL (simply sql calls => should be pretty straightforward) and then tests for the handlers (slightly more complicated).

July goals

I’m back from my trip out East and ready to get back into some existing and new projects. The PostGres Vision conference was a good opportunity to learn about a bunch of different companies that are coming up with new software using postgres. Many of the talks I went to gave an overview of some technology, followed by the presenter’s software which makes use of the technology. It wasn’t a very big conference – maybe about 200 people- so it wasn’t too overwhelming, and in that respect, it was a good place for my first tech conference.

So now that I’m back, I have some goals for July:

  • Quizipedia tests: I didn’t get a chance to write the tests that I wanted to last month, so that is going to be my priority for July. I want to have some unit and integration tests around the entire codebase, and moving forward with the project, I want to get in the habit of doing test-driven development
  • Quizipedia improvements: The relevant word algorithm on quizipedia is very primitive, so I want to do a bit of research on more intelligent ways of improving it. I may look at some of the language processing libraries that I came across while studying Postgres. This is primarily about research and investigation, s I likely won’t make any windfall changes to the code in this area.
  • Open-source project: I’m going to shop around to see if I can find an open-source project to do some work on. I’m not locking in any theme of the project, but I am going to be looking especially hard for something related to AI. These days, saying that seems like a vague statement, so I may start with some areas that I have a bit of experience in, such as constraint satisfaction problems, writing Prolog, and motion planning.
  • Books: I’ve been reading quite a few software development books lately, and I’d like to write some reviews/summaries for them.

Postgres: Indexes

Today I got through the postgres chapter on indexes. I actually thought there would be a lot more to indexes, but it turns out I’m already fairly familiar with the important details. There are some index implementations of postgres that I was totally unfamiliar with (GiST, GIN, BRIN), but the B-tree is easily the most important implementation as far as I’m currently concerned.

There were a few enlightening lessons, such as the important of ordering your indexes properly, and the concept of partial indexes, but for the most part this ended up being a formalization and review.

Quizipedia – Client-Side code

Today I was able to add the client-side views and connections to the quizipedia app. There are two simple views:

  • The game creation form: Enter a game name and the text to be parsed into a quiz.
  • The game itself: This is where the blanked out game and interaction takes place

As with the server-side code, I had written quite a bit of this code previously in another repo, so it was matter of getting it organized and hooked up with the rest of the new system. So it only took me one session to get all that done. And at this point, I’ve pretty much used up all my previously-written code, so all the next stuff is going to be new for me, and will likely take longer.

The client-side functionality is written in JavaScript, so this gives me a chance to see JavaScript in action on both the client-side and the server-side. This gets rid of any mental context-switching, which can occasionally cause a bit of lag.

So the main connections are made, and it is working well both locally and on production! So quite a satisfying day, but still lots of work to do. Over the next day or two, I’m going to get as much of the following done as I can:

  • Add in logical error-handling and logging
  • Style the client-side so it looks nice
  • Providing more feedback on correct/incorrect answers to the user
  • Add tests for the handlers
  • General code cleanup: comments, proper conventions, consistency, etc.
  • DB maintenance – maybe automate some scripts to clean up the db every so often

Once I finish a few of those tasks, I’m going to be re-focusing my goals for the month of June, which I will discuss in a later post.

Quizipedia Server-side code

I finished up most of the server-side code for my quizipedia web application today. It was fairly easy to write, since I had a few repos of sandbox code that had at least 75% of the main functionality, so it was just a matter of cleaning it up and putting it all together.

Here’s what I’ve accomplished in the past two sessions of working on it:

  • Set up a Heroku project using the node-related tutorial
    • Resource
    • Heroku makes it very easy to actually deploy something and allows you to have an easy testing environment, using a local testing database
  • Set up a postgres database (local and prod)
    • Resource
    • To add a PG db I just had to “provision” the Heroku PostGres add-on, which is straightforward and included in the node tutorial
    • My main issue was that I was trying to connect my local environment to the production database that Heroku sets up for you. It took me too long to realize that I was supposed to be using a LOCAL db, which I quickly set up
    • I think it will be important to keep a change log of any db updates that I do. When making changes locally, I can just wipe the db and re-run the script I’ve saved. But this isn’t going to work on prod, where the existence of the data depends on me NOT destroying the tables. I have some experience using a migrations package, which allows you to create a new ‘migration’ file anytime you need to make a change, for example adding a table column. This will be essential for keeping track of all my changes. I’m not going to add it in right now, as my focus is to get something out and working.
  • Added the main handlers. This is a RESTful application, so the primary handlers right now include:
    • POST /game: This is where a user would post some text to be formatted into a quiz. Eventually I would like to have them just send a link to a webpage with the data, do the parsing, and create the quiz that way.
    • GET /game/:gameId: This is going to fetch the game information by the id generated in the saving functionality. It will return the blanked out game. Further parsing will be done on the client-side
    • POST /guess: This is is a call to allow the user to submit a guess for one of the blanks. Their guess will be verified against the true answer. If it is correct, it will be saved accordingly.
  • Connected the handlers to the DB via a function library and a data layer.
    • The data layer is essentially an active record setup, where the models reflect the columns in the database. I have three tables, so I have 3 dal classes.
    • The library is the business logic layer, where the relevant word algorithm gets used, and the objects get converted to the appropriate formats
    • Building the game is a large job, so I created a completely separate ‘gameBuilder’ helper class which is called by the main ‘game’ library class.

 

So I’m happy with this initial progress. Again, a lot of it was just organizing some existing code in some unorganized repos into something workable and organized.

My next big step is going to be to add the client-side handling. This will mainly involve submitting a new game text, presenting the constructed game, and filling out correct answers as needed.