How I built a serverless BI dashboard at $2

I would like to share my experience of building a serverless dashboard. For example, any small startup or business would be doing some sort of digital marketing campaign to get traffic to their website, which in turn needs to be measured to know the ROI. Google Analytics comes in handy, also some sort of CRM , Support services would be used to support their customers, measuring the key metrics from these services is also inevitable to know how the enabling functions like Marketing, Sales, Support, Product teams are performing. Finally the product you have built should have some key metrics that needs to be measured for adoption rate.

I did build a first version of the dashboard to visualize the metrics from these services by leveraging AWS Elastic Beanstalk application considering the convenience it gives as a service for a developer, where in you can simply upload the code and it takes care of everything from provisioning to deployment. But after a while I did take a look at what I have built and decided to build it as a serverless application.

Cloudfront, S3, AWS Lambda, DynamoDB, API Gateway were used to build the serverless app. All these resources were provisioned using a SAM (Serverless Application Model) of Cloudformation (to define the resources like API Gateway, DynamoDB, Lambda functions) for serverless app in one click.

For demonstration purpose I used Google Analytics, Freshsales, Freshdesk (or any other products) and core product (core application) that are RESTFUL in nature from where I wanted to gain insights in a consolidated dashboard. In this example we would pull the data about websites’ traffic (Google Analytics), Sales leads, Support tickets and other key metrics from the core application in a single page. Here is what we need to build one.

We could host the front-end website in S3 and deliver through a CDN like Cloudfront. Use Route53 to redirect front-end website to the Cloudfront distribution URL and enable alternate CNAME like bi.yoursite.com in your distribution.

AngularJS framework was chosen to support dynamic views, while Python was used to build back-end micro-services that run as a Lambda functions.

The app has been split into Lambda functions like login, logout, createuser, changepassword, landing page, refresh. We can build these Lambda functions and test to see if we can get the JSON response from the API’s.

Lambda Functions:

You may write these functions in your preferred language. The ‘bilanding’ function used for refreshing the dashboard is given below for reference.

Declare any Environment variables in the “code” section of Lambda console.

Next we must create the API resources and use it as a “trigger” for the respective Lambda functions.

API Gateway Resources:

The resources in API Gateway console like below can be created by following the steps documented here.

Obtain the URL of the API to be used in the ‘app.js’ while calling the specific functions to be executed, it is hosted in S3 for serving website.

Deployment:

Package the application using the SAM template by issuing the following command. The documentation for deploying the serverless application is here.

aws cloudformation package — template-file dashboard.yaml — output-template-file dashboard-output.yaml — s3-bucket s3://bi.yoursite.com

Deploy the application using the packaged application using cloudformation deploy command.

aws cloudformation deploy — template-file /<appdir>/source/dashboard-output.yaml — stack-name dashboard

Appearance of the dashboard (truncated) looks like below once it is deployed.

Dashboard:

Summary:

Frontend: Website is hosted in S3, and served through Cloudfront(CDN). AngularJS framework to support dynamic views.Chart.js has been used for rendering the charts in the dashboard. Bootstrap was used as a library for building the website.

Backend: AWS Lambda to run the application code and API Gateway to build the API endpoints.

Database: DynamoDB for storing and rendering the analytics data in real time with sub-millisecond latency.

Deployment: AWS Cloudformation (SAM) has been leveraged to package and deploy the application.