MongoDB

MongoDB is the database provider we will use when implementing your Web API using the MERN Stack. It is what is known as a NoSQL database. This type of database is relatively new. It was first introduced about 20 years ago. Prior to the introduction of NoSQL databases, the standard way data was represented in databases was through relationships.

Relational Databases

A relational database, such as Microsoft's SQL Server, has tables, with records. Each table has a "schema" that defines the fields that will be included in each record, including their name and the type of data that will be stored in the field.

Below is an example of the database design diagram for some of the potential tables in Microsoft's SQL Server for our news site.

Each table has a field, typically named "id", that is the primary key for the table. The primary key field is typically auto-generated by the database. The primary key field uniquely identifies each record in the table.

The primary key also acts as a link, or foreign-key, when a record in one table is related to a record in another table.

For example, there is a relationship between a comment and an article. Each record in the Comment table, has a field, named authorId, whose value is the id of the related record in the Article table.

In addition to defining the field names, in a relational database you also define the type of data that will be stored in each field.

bigint - 64 bits
varchar(256) - a string up to 256 characters in length
varchar(MAX) - a pre-defined string length that is the max storage space for the string datatype.

SQL - Structured Query Language

A special query language, called Structured Query Language (SQL), was designed to be able to build queries to access the data within the related database tables.

You can select all the records/fields from a single table.

Or, just a subset of the fields

Or, you can join the rows/fields across to related tables. In this case we are joining the Article and Topic tables, to create a single record for each article that includes the fields from the related topic record.

Relational databases, with their associated schemas, have been the standard for database design up until about 20 years ago, when the first usage of the term NoSQL was introduced.

NoSQL Databases

When people use the term “NoSQL database”, they typically use it to refer to any non-relational database. Some say the term “NoSQL” stands for “non SQL” while others say it stands for “not only SQL.” Either way, most agree that NoSQL databases are databases that store data in a format other than relational tables. - MongoDB

A Document Database

In contrast to a structured relational database, MongoDB belongs to a new type of database, called a NoSQL database.

Both types of databases have the equivalent of a table that is used to stored data, but in MongoDB, the table is called a collection. And instead of records containing field values, a MongoDB record is called a document and is the smallest unit of data that the database manages.

In the image below, the database named "newsSiteCollab" has four collections: articles, comments, topics, and users.

In contract to the equivalent tables shown earlier in the SQL Server database, the MongoDB collections (tables), do not have a schema that describes the fields that are in each document and their data types.

Clicking on the "Insert Document" button brings up the following dialog:

All that is required for fields in the document, is an id field that is auto-generated by the database. You can add as many property/value pairs as you want, and they do not need to conform to a schema. This means that it is permissible to have documents that are widely different from each other stored in the same collection.

Comparison: SQL vs NOSQL Database

If you look at the collection with documents, you can see that there are no fields defined. It is just a collection of JSON objects.

Compared to a relational database.

NonSQL - Embedded Objects

One interesting aspect of NonSQL databases is that the JSON objects can contain embedded data. The data can be stored embedded within the document, or the related data can be embedded as part of a query.

{
  "_id": "60d24ffdb4f2084cb4ee9510",
  "title": "Article 1",
  "abstract": "Lorem ipsum dolor sit amet, consectetur adipisicing elit. Et ea impedit libero, beatae animi provident nesciunt molestias ipsam nemo ad.",
  "body": "Lorem ipsum dolor sit amet, consectetur adipisicing elit. Et ea impedit libero, beatae animi provident nesciunt molestias ipsam nemo ad. Lorem ipsum dolor sit amet, consectetur adipisicing elit. Et ea impedit libero, beatae animi provident nesciunt molestias ipsam nemo ad.",
  "imageURL": "https://images.pexels.com/photos/2599244/pexels-photo-2599244.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940",
  "author": {
    "_id": "60d24aeaeacbc4390430134c",
    "username": "sallysmith",
    "email": "ss@gmail.com",
    "firstName": "Sally",
    "lastName": "Smith",
    "imageURL": "https://images.unsplash.com/photo-1579591919791-0e3737ae3808?ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&ixlib=rb-1.2.1&auto=format&fit=crop&w=658&q=80",
    "createdAt": "2021-06-22T20:41:14.182Z",
    "updatedAt": "2021-06-22T20:41:14.182Z",
    "__v": 0
  },
  "createdAt": "2021-06-22T21:02:53.022Z",
  "updatedAt": "2021-06-22T21:02:53.022Z",
  "__v": 0
}

If the related data is going to be used independent of the containing object, such as comments being retrieved and displayed separately from the article, then it makes sense to store them in separate collections. On the other hand, if the embedded data is always only used in the context of the containing document, then it can be stored in the same document.

It is up to the database designers to determine the correct organization of the data that works for their application.

The use-case for NonSQL databases is best for cases where there is not a lot of related data.

Mapping Database Documents => Application Objects

In most applications, the MongoDB documents are mapped to JavaScript objects used within the application, and therefore there is a need to define a schema that can be used to translate the document objects within the database to JavaScript objects.

This application object that maps to the database object is represented by a "Model". For example, when the data from the Articles collection is retrieved, a JavaScript object is populated with the database document data.

Mongoose - Object Document Manager

This is where Mongoose comes in. Mongoose is a Object Document Manager for Node.js, which is a library that allows you to define schemas for your MongoDB data and provides utilities for mapping the MongoDB data to JavaScript objects.

In the same way that SQL Server defines the fields and data types within a table, Mongoose schema defines the properties and their types for the data within a document object. The schemas can also store information about validation, default values, and whether a particular property is required. Basically what I showed earlier in the database designer tool within SQL Server.

In addition to defining schemas, Mongoose provides utilities for querying the data within the database and instantiating JavaScript objects based on the schemas.

Mongoose Schema - defines the structure of a MongoDB document.
Mongoose Model - uses the schema to facilitate mapping the document data to and from JavaScript objects. It is also provides the queries to add/update/find/delete documents.

A Mongoose Document Schema and Model

Below is a sample of using Mongoose to define a schema and model for a MongoDB collection that stored data about users. Mongoose allows you to define the properties with the document, and their data types, and optional validators.

Mongoose also provides built-in helper methods to find, create, read, update, delete, search data from the MongoDB database.

// define the schema
const UserSchema = new mongoose.Schema({
  username: {
    type: String, 
    lowercase: true, 
    required: [true, "can't be blank"], 
  },
  email: {
    type: String, 
    lowercase: true, 
    required: [true, "can't be blank"],
  },
  firstName: {
    type: String,
    required: [true, "can't be blank"], 
  },
  lastName: {
    type: String,
    required: [true, "can't be blank"], 
  },
  imageURL: {
    type: String,
  }
}, {timestamps: true});


// define the mongoose model
const User = mongoose.model('User', userSchema);

// create a new user through the mongoose model
const user = new User({
    _id: new mongoose.Types.ObjectId(),
    username: 'sallysmith',
    email: 'ss@gmail.com',
    firstName: 'Sally',
    lastName: 'Smith'
});

// Save user in MongoDB using Mongoose save()
user.save((error) => {
    if (error) {
        return console.log(`Something went wrong: ${error}`);
    }
    console.log('Document saved successfully!');
})

Setting up a Database - Atlas

The first step is to create the database. In our case, we are going to create the database using a free, cloud-based service provided by MongoDB called Atlas.

There are a bunch of steps to setup the database. They involve the following tasks:

create an account on the Atlas website
create a database cluster (where/how the database is deployed/managed)
create a database within the cluster
create one or more document collections within the database (we'll start with one named Users, to store user profile information)
create a database account with full read/write access that we will use to access the database from JavaScript.
generate a password for the database user account.
configure network access to the database from our local computer (this is done by default, using IP address of 0.0.0.0/0, or localhost.)
copy the connection string and adjust it to have the correct information for our database/user account and insert it into our JavaScript code.

Here's a detailed walkthrough of the steps.

After signing up to use Atlas, the first step is to create a cluster. A cluster is a managed deployment of a MongoDB database. It manages the replication of databases for scaling and redundancy.

Click on the "Build a Cluster" button to proceed.

Next, choose the type of cluster. For our purposes, we're just signing up to create that most basic deployment model, which is free.

Next chose the cloud-provider and region. You can just stay with the defaults, which use Amazon's AWS and N. Virginia (where the servers are located).

Then, wait for your cluster to be created.

Next, we want to create a database within the cluster.

From within the Clusters view, click on the "Collections" button.

Then, click on the "Add My Own Data" button.

For the database name, I'll input "news-site-collab", since that's the name of our web site.

For the collection name, this is where we will be storing documents. We will have more than one collection. A collection is like a table in a relational database, so each object type that we're going to store in the database would have its own collection. We'll start out with a single collection for storing users.

Click the "Create" button, and the new database, with a single collection named "users" will be created.

Next, we need to set up a database user that we will use to interact with the database through its API (no change from default).

You can create multiple database user accounts and configure each with different access levels.

Click on the "Database Access" link on the left.

By default, Atlas creates a user named "test" that has read/write access to the database. It is considered to be in the "admin" role because it has full access to the database.

We will just set a new password for the account. We can just use the "Autogenerate Secure Password" button to create a new password and the later, when we need the password, we can just click the "Copy" button to get the password to paste into our connection string in the server.js file.

Next, we need to setup network access (no change from default).

This is configuring what IP addresses can access the database. The default address is "0.0.0.0/0". This works when you are testing from you local development computer. But, once you deploy a site, you would have to add an IP address for the web client computer.

Next, we need to setup a connection string to access the database.

Navigate back to the Clusters section and click on the "Connect" button.

Choose the "Connect your application" option.

Copy Your Database Connection String

The connection string will be used from your JavaScript code to access the database.

Here's the example from the database I am creating.

mongodb+srv://test:<password>@cluster0.qdsuf.mongodb.net/myFirstDatabase?retryWrites=true&w=majority

We need to replace two pieces of information in the connection string:

database username/password
database name

We'll do that when we move to setting up the JavaScript code.

Connecting to the database through Mongoose.

Within our server.js file, we are now going to setup the connection to the database by using the mongoose module.

First, we need to install the mongoose package through npm.

npm install mongoose

Next, we include the mongoose module with our server file, server.js. Here's our basic express web API foundation. To add mongoose, we'll add a few new items.

const express = require('express');
const app = express();

app.put('/api', (req, res)=> {
  res.send('Hello World!');
});

const port = 3000;
app.listen(port, ()=> {
  console.log(`Server started on port ${port}`);
});

line 2 - include the mongoose module
lines 9-17 - code to connect the database through Mongoose

const express = require('express');
const mongoose = require('mongoose');

const app = express();

// standard boiler-plate code to connect
// to the MongoDB database through Mongoose

const connectionOptions = {
  useNewUrlParser: true, useUnifiedTopology: true
}
const connectionString = 'mongodb+srv://test:etsrn3FOBY5UvaRN@newsdata.qdsuf.mongodb.net/myFirstDatabase?retryWrites=true&w=majority';
mongoose.connect(connectionString, connectionOptions);

const db = mongoose.connection;
db.on('error', (error)=>console.log(error));
db.once('open', ()=> console.log('connected to database'));


app.put('/api', (req, res)=> {
  res.send('Hello World!');
});

// start the server to listen on the port
const port = 3000;
app.listen(port, ()=> {
  console.log(`Server started on port ${port}`);
});

Configure the database connection string to access a specific database and use a specific database user account.

As you can see on line 12, the connection string still looks like what we copied from the Atlas configuration. We need to change it to contain the right connection information for our database.

account name: test (didn't change from the default)
password: go to the Database Access screen and copy the password.
database name: "news-site-collab'

Storing the connection string in an environment variable.

Right now the connection string is stored in a variable in the server.js file. This is a shared file that is stored in the project's GitHub repository. When you are working in your own development environment, you want to be able to configure your web application client to interact with your own database, instead of sharing the same database with all of your teammates.

The way to set this up is to use a feature within Node.js that reads environment variables stored in a file named .env at the root of your project. You store your db connection string in your .env file, and then you add the .env file to the .gitignore file so that it isn't included in the files stored in the git repository. That way, you have your own private copy of the .env file, and can configure what connection string you want to use.

Install DotENV VS Code extension

It just adds useful syntax highlighting for the .env variables in the file.

Install dotenv package

This package facilitate getting the values stored in the .env file from another JavaScript file.

npm install dotenv

Create .env File

# Variable
PORT=5000
DB_CONN_STRING="mongodb+srv://admin:uJjSaOuux52Olwwb@cluster0.qdsuf.mongodb.net/newsSiteCollab?retryWrites=true&w=majority"

Create a config.js File

This is a pattern you can use to simplify the importation of .env variables into other files. Once they are exported from this module, then you can use destructuring to access the specific export you want.

const dotenv = require('dotenv');
dotenv.config();
module.exports = {
  port: process.env.PORT
};

const { port } = require('./config');

Working Connection Code

const express = require('express');
const mongoose = require('mongoose');
const dotenv = require('dotenv');
const { port } = require('./config');

dotenv.config();
const app = express();

const connectionOptions = { useNewUrlParser: true, useUnifiedTopology: true}
mongoose.connect(process.env.DB_CONN_STRING, connectionOptions);

const db = mongoose.connection;
db.on('error', (error)=>console.log(error));
db.once('open', ()=> console.log('connected to database'));


app.put('/api', (req, res)=> {
  res.send('Hello World!');
});

app.listen(port, ()=> {
  console.log(`Server started on port ${port}`);
});