Live Chat App Using Meteor and the DDP Protocol
For a moment, let's forget about all the available live chat solutions out there. Imagine that we want to write our live chat app, and we want to have a small chat widget on all our websites connected to our main app, so visitors of our websites could chat with us in real-time. This isn't a trivial task, but here we're going to go through the creation process using Meteor — an excellent real-time app platform and independent tool based on DDP (Distributed Data Protocol) — the stateful WebSockets protocol that Meteor uses to communicate between the client and the server. Our main goal here is to show how simple it is to create this kind of app with Meteor and introduce DDP as an approach to real-time data layers.
Be aware that you should have a basic understanding of Meteor to go through the code quickly. You should also know how to use Git because the whole article is based on three demo projects.
So let's dive right in.
If you don't want to read, you might want to check out SimpleChat.Support. Open Source and MIT licensed Live Chat App.
DDP Protocol and Meteor
Meteor is a real-time platform for building web and mobile apps. What does that mean? It means that it's built-in in a way that provides tools and API to start building real-time apps (like chats) very quickly. It is a platform that could be the best choice in our case. That's because it uses real-time communication protocol and data synchronization in its core architecture, and this is what we need here. We will be able to consume that without a complicated configuration phase. The protocol is called DDP (Distributed Data Protocol). Let's see what it is exactly.
What is DDP?
The DDP is something like REST, but for WebSockets. The protocol defines some standards in communication between the server and client. It establishes some JSON messages over WebSocket written down in the specification. WebSockets allow us to have two-way communication in real-time, and chat applications are the best example of this kind of use case.
Our app will be based on the primary purpose of the DDP protocol, which is when the client subscribes to a set of documents, and the server keeps the client informed about the contents of those documents as they change over time. So when our chat widget is connected to the host app's server, it will automatically update the messages list in the chat window. It will work the same for the operator logged in to the administrator panel of the host app. All the app should do is to write messages from both sides and synchronize them using DDP.
In Meteor core, all data transportation relies on DDP. For example, when we create a new Mongo collection called 'chat', it will register with DDP to keep data from that collection in sync. So every change in this collection will be automatically propagated to all clients reading from it. This is done without any additional work.
The Meteor team originally developed the DDP protocol, but nothing in it is Meteor specific. Moreover, the client and server parts of the Meteor app don't know about each other. They speak the same language - DDP. This leads us to the conclusion that it could be used in apps written in different technologies, and still, these apps would be able to talk freely to the Meteor app. They need to use DDP. So let's see what we need besides our Meteor host app.
Independent DDP Clients
We want to be able to use our chat widgets on all kinds of websites. It could also be a simple static website built only with HTML and CSS. So we need to have a browser-based DDP client and functionality written in a static JavaScript file which will consume the API from the DDP client and subscribe to the data with chat messages located in the Mongo database connected to the host app built with Meteor.
Fortunately, plenty of DDP clients are out there, so we don't need to write it by ourselves. Check the list linked above or just search for it on Google. We can find standalone JavaScript DDP clients and clients written in PHP, Ruby, or Objective-C.
Let's focus on our browser-based DDP client. We will be using DDP.js client in version 0.6.0 because it is bundled use in the browser. We could also use higher versions or something else like Asteroid, but then we need to bundle it for the browser used. Let's stick to the 0.6.0 version, which works quite well.
What it gives us is a clean API to use DDP in the browser. So we'll be able to connect to our host app built with Meteor. We'll start building it in the next chapter.
Chat Host App built with Meteor
What will be our host app? It will be an example application built with Meteor, which will be used to manage chat sessions from outside clients, connecting to it through chat widgets placed on many different websites. It is all possible because we'll use DDP. Meteor is integrated into the core, and our chat widgets will use browser-based DDP implementation. So basically Meteor app will provide Mongo collections where all messages will be stored, and the DDP connection will synchronize data changes in all connected clients.
We'll go through a basic example here. But at the end of the article, there will be a link to a ready and working product, which is also open-sourced and MIT licensed, so it is a source of additional knowledge on that topic.
Be aware that we will be building a straightforward app that will not support user accounts, and it will also not be secured. Anyway, there will be an opportunity to compare it with a production-ready app.
Ok, enough introduction, let's get into the code.
We'll start from the beginning. We need to have Node installed. Also we need to have Meteor and Git.
After that, we can create our Meteor host app. We can do this in a standard way by running meteor create live-chat-app, but the better way would be to clone the repository from GitHub, a simple, ready-to-use Meteor host app.
We can clone it by running:
$ git clone https://github.com/juliancwirko/live-chat-app-demo-host.git
(Or we can download the .zip file).
Then we can also run the app:
$ cd live-chat-app-demo-host
$ meteor
Let's explore the cloned repository. From Meteor in version 1.3, we can use ES2015 imports. There is a lovely trick in Meteor. All files we want to import in our Meteor app should be placed in the imports directory. They will be lazily evaluated. All files outside this directory will run as the app start. We will partially stick to the Meteor Guide Application Structure. So we have all functionality in the imports directory and three starting points outside this directory, which will import all files on the server-side, client-side, and sides. Here we're talking about the main.js files located in folders called, respectively, server, client, and both.
This is all about the app's structure. We could, of course, rearrange it as we want. We need to know how it works in Meteor. Read more in the Meteor Guide.
First of all, we need to look at our core functionality, MongoDB storage. We need to store our client apps and chat messages somewhere. This is done in the imports/both/collections.js. We need to declare two new Mongo collections, and we need to export them.
import { Mongo } from 'meteor/mongo';
export const Chat = new Mongo.Collection('chat');
export const Client = new Mongo.Collection('client');
That's all for now. We'll use them later, and we'll import them and use them in other files.
In this demo app, we have all client-side functionality in one imports/client/layout.js file. We could split it into many smaller files too. We need to remember about imports and order of imported files. For now, let's stick to the one file. It'll be more straightforward. We can ignore the index.js files in each folder, which are there for better readability when we import our files to the main.js files outside the imports folder.
In imports/client/layout.js the first, we import all necessary tools which come from the Meteor core.
import { Template } from 'meteor/templating';
import { ReactiveDict } from 'meteor/reactive-dict';
import { $ } from 'meteor/jquery';
import { Chat, Client } from '../both/collections.js';
import './layout.html';
We need a templating layer and a reactive dictionary responsible for managing the local app's state. We can also import jQuery, and of course, our two previously declared MongoDB collections. We also need to import our HTML templates here.
Each template has its helpers and events maps. We've implemented basic logic here to add and display client apps and chat messages. For example, let's take a look at the main template event to add a new client app:
Template.main.events({
'click .js-add-new-client-app'() {
const instance = Template.instance();
const name = instance.$('[name=client-app-name]').val();
if (name) {
Meteor.call('addClientApp', name);
instance.$('[name=client-app-name]').val('');
}
}
});
We need to know here that the app will provide a form through which we can add a new client app. Every newly created client app has a unique id. This id is essential because it will connect chat boxes on the website with this particular client app.
** Let's see how our data is managed here.**
We have two collections (declared before) - Client and Chat. In the first one, we store all created client apps. In the second one we store all chat messages which will be assigned to the client apps and chat sessions. On the client side we need to subscribe to the data. For example:
Template.main.onRendered(function () {
const instance = this;
instance.subscribe('Client.appsList');
instance.autorun(() => {
instance.subscribe('Chat.list', state.get('openedApp'));
});
});
We can do this because we have configured our publications in the [imports/server/publications.js](on the server-side) https://github.com/juliancwirko/live-chat-app-demo-host/blob/master/imports/server/publications.js). So we can subscribe to this particular data set on the client and listen for changes. This is how our publications on the server look:
import { Meteor } from 'meteor/meteor';
import { check } from 'meteor/check';
import {Chat, Client} from '../both/collections.js';
Meteor.publish('Client.appsList', () => Client.find());
Meteor.publish('Chat.list', (clientAppId) => {
check(clientAppId, String);
return Chat.find({clientAppId: clientAppId});
});
Meteor.publish('Chat.messagesList', (clientAppId, userSessionId) => {
check(clientAppId, String);
check(userSessionId, Match.Optional(String));
return Chat.find({clientAppId: clientAppId, userSessionId: userSessionId}, {sort: {date: 1}});
});
Read more about Meteor's pub/sub system in the Meteor Guide Publications and Data Loading.
So basically, everything is located in the client-side layout.js file. We subscribe for the data which is published on the server-side. Still, we also have a form with an event that fires a Meteor Method called addClientApp and the other one in the chatBox template, which will fire a Meteor Method called addChatMessage. We use Meteor Methods to add and edit data. They are declared in the imports/server/methods.js file. Client-side chat boxes will also use the addChatMessage method.
Methods declared on the server side:
import { Meteor } from 'meteor/meteor';
import { check } from 'meteor/check';
import {Chat, Client} from '../both/collections.js';
Meteor.methods({
addClientApp(name) {
check(name, String);
Client.insert({name: name});
},
addChatMessage(msg, clientAppId, userSessionId) {
check(msg, String);
check(clientAppId, String);
check(userSessionId, String);
Chat.insert({
msg: msg,
clientAppId: clientAppId,
userSessionId: userSessionId,
date: new Date()
});
}
});
Everything is ready here. Take a break, read the code in the repository and try to understand what is going on. Run the app and play with it. There will be no chat messages yet, but we can create our client apps.
If you are ready, we can look at the second part of the app, a client chatbox for our static HTML website, and another Meteor app.
Connect to the Host App from a Standard HTML Website.
We have our host app ready. We can go and run it now in the terminal. If it is not running, go to the live-chat-app-demo-host and run meteor command. Then go to the localhost:3000 in your browser and create a new client app.
Next, we can clone our second repository, a static HTML website with an implemented chatbox client. Go and clone it by:
$ git clone https://github.com/juliancwirko/live-chat-app-demo-html-client.git
(Or we can download the .zip file).
Then we can go to the live-chat-app-demo-html-client folder, and we can edit index.html file. We need to provide our unique id. Copy it from the host app and paste it in the liveChat.init('your_client_app_id_here'). Then open the index.html file in the browser. We should see a simple static HTML website with a chatbox opened. We should write in the chatbox and read the messages in the host app (running at localhost:3000) when we choose a proper client app and chat session. Remember that our host app should be running.
But let's see how it is possible.
In the live-chat-app-demo-html-client website, we have three files: index.html, live-chat-app-client.js, live-chat-app-client.css. We'll take a look at the two last files. The .js file because the .css file is just styling for our chatbox displayed at the bottom right corner.
In the live-chat-app-client.js file, we use DDP.js in version 0.6. It is hardcoded in the same file just for convenience, but you can also include it in the index.html file as a separated library.
We have here a whole chatbox implementation, so also a chatbox widget construction process and DOM manipulations. Our JavaScript init(...) method will create a chatbox from the ground and initialize the DDP connection. I will not describe all DOM operations which make chatbox. You can go through code - it shouldn't be hard to read.
The most important part here is the DDP.js library and its usage. We can see that we need to initialize DDP.
var ddp = new liveChatDDPClient({
endpoint: 'ws://localhost:3000/websocket',
SocketConstructor: WebSocket
});
We need to provide WebSocket endpoint our case is ws://localhost:3000/websocket because our host app runs at localhost:3000. We need to provide SocketConstructor. We use WebSocket here, but we could also use the SockJS library.
When we initialize our DDP connection, we can listen for the connected event. We do this by:
ddp.on('connected', function () {...});
After successful connection, we can run chat box creation and we can subscribe to our data served on the host app. We can subscribe to the same publications which are used on the host app. We can do something like:
ddp.sub('Chat.messagesList', [clientAppId, userSessionId]);
We take clientAppId from the provided value in the index.html file in the liveChat.init(...) method. and we also need to generate unique userSessionId, which we do by using the Random package from Meteor. Now we can listen for added event (new messages) and do something with this. In our case, we need to attach new message to the previously created chatbox.
This is all we need here in this client chatbox. All new messages for this userSessionId and clientAppId will be placed in one MongoDB collection, which the host app serves. If you are an operator and want to reply to the client writing to you from the static HTML website, you can do it by opening a proper client app (click on it) and then clicking on an appropriate chat session (which will appear below). All messages in both chat boxes should be synchronized so you'll be able to chat.
Now, let's play with it. We can create more client apps in the host app. We can connect more static websites just by copy and paste.
When you are ready, we can look at another use case, a Meteor client app. As we know, Meteor has DDP support at its core. Imagine that you want to use the chatbox, not on a static website, but another Meteor app. Do we need to use the same static JavaScript file? Of course not. Let's see why.
Connect to the Host App from Another Meteor App.
Again, we'll clone the third ready-to-use repository. Now it's another simple Meteor app. Clone it:
$ git clone https://github.com/juliancwirko/live-chat-app-demo-meteor-client.git
(Or we can download the .zip file).
We can run it as usual with a different port because our host app needs to be running, and it runs on the default Meteor port, which is 3000:
$ cd live-chat-app-demo-meteor-client
$ meteor -p 5050
When we run the app and go to localhost:5050, we should see a very similar view as the previous one from the static HTML website. The functionality is also the same. We need to connect to the host app and this is it. It should work.
Meteor.startup(function () {
liveChat.init('AQne8yYkemE923dqE');
});
So, what is different? We don't use DDP.js here because it is not needed. We have access to the core DDP library from Meteor, so we should use this one.
In our app, we just initialize liveChat. All functionality lays in the local package in /packages/live-chat-meteor-client. We initialize DDP in a different way. If we want to connect to our host app we should do something like:
ddp = DDP.connect('http://localhost:3000');
Remember that our host app should be running at localhost:3000.
Next, we need to create a new MongoDB collection that will synchronize data with the host app.
chatCollection = new Mongo.Collection('chat', {connection: ddp});
We need to provide a connection parameter, which is our newly initialized DDP connection.
Next, we can subscribe to the data published on the host app.
ddp.subscribe('Chat.messagesList', clientAppId, userSessionId);
We can also get the data from the collections by:
messages = chatCollection.find({userSessionId: userSessionId}, {sort: {date: 1}});
Please look at the local package in /packages/live-chat-meteor-client. Some files will build our chatbox, CSS files, and all Blaze events and helpers, which will populate our chatbox with messages from MongoDB served by the host app.
So as we can see, in another Meteor app, we don't need what we did in the previous example. We can use Meteor's core components to achieve the same effect. This is because Meteor apps are designed to be able to speak to each other with ease.
Ending thoughts
I've prepared three simple repositories which you can clone and play with. There is the Live Chat Meteor host app which provides functionality to create and store new client apps. There is functionality that makes it possible to reply to each connected client. There are also a simple static browser chatbox client app and the Meteor one, which is very similar but uses a different DDP client, the core one.
These are, of course, only demo projects. It would be best if you didn't use them in the production. They are just for learning and testing purposes. Although if you are interested, I've built a simple production-ready live chat app called SimpleChat.Support. It is also Open Source and MIT licensed. It is secured, has user account support, an awesome administrator panel, and the client chat boxes are much more usable. You can use it, learn from the code, or just built upon it. You'll find a link to the GitHub repo on the main website.
If you want to comment or improve the article, write to me. I'm open to all suggestions.