Ghost Themes
SimpleChat.Support - Open Source Live Chat App
0%
Live Chat App Using Meteor and the DDP Protocol
words - read.

Live Chat App Using Meteor and the DDP Protocol

For a moment let's forget about all the available live chat solutions out there. Imagine that we want to write our own live chat app and we want to have a small chat widget on all our websites connected to our main app, so visitors of our websites could chat with us in real-time. This isn't a trivial task, but here we're going to go through the creation process using Meteor — an awesome real-time app platform and independent tool based on DDP (Distributed Data Protocol) — the stateful WebSockets protocol that Meteor uses to communicate between the client and the server. Our main goal here is to show how simple it is to create this kind of app with Meteor and to introduce DDP as an approach to real-time data layer.

Be aware that you should have a basic understanding of Meteor to be able to go through the code easily. You should also know how to use Git because the whole article is based on three demo projects.

So let's dive right in.

If you don't want to read, you might want to check out SimpleChat.Support. Open Source and MIT licensed Live Chat App.

DDP Protocol and Meteor

Meteor is a real-time platform for building web and mobile apps. What does that mean? It means that it’s built in a way that provide tools and API to start building real-time apps (like chats) very quickly. It is a platform which could be the best choice in our case. That’s because it uses real-time communication protocol and data synchronization in its core architecture and this is what we need here. We will be able to consume that without a complicated configuration phase. The protocol is called DDP (Distributed Data Protocol). Let's see what it is exactly.

What is DDP?

The DDP is something like REST, but for WebSockets. The protocol defines some standards in communication between the server and client. It defines some JSON messages over WebSocket which are written down in the specification. WebSockets allows us to have two-way communication in real-time and chat applications are the best example of this kind of use case.

Our app will be based on the main purpose of the DDP protocol which is the process when the client subscribes to a set of documents, and the server keeps the client informed about the contents of those documents as they change over time. So when our chat widget is connected to the host app's server, it will automatically update messages list in the chat window. It will work the same for the operator logged in to the administrator panel of the host app. All the app should do is to write messages from the both sides and synchronize them using DDP.

In Meteor core, all data transportation relies on DDP. For example, when we create a new Mongo collection called ‘chat', it will register with DDP to be able to keep data from that collection in sync. So every change in this collection will be automatically propagated to all clients reading from it. This is done without any additional work.

The DDP protocol was originally developed by the Meteor team, but nothing in it is Meteor specific. Moreover, the client and server parts of the Meteor app doesn't know about each other. They just speak the same language - DDP. This leads us to the conclusion that it could be used in apps written in different technologies and still these apps would be able to talk freely to the Meteor app. They just need to use DDP. So let's see what we need besides our Meteor host app.

Independent DDP Clients

We want to be able to use our chat widgets on all kind of websites. It could be also a simple static website built only with HTML and CSS. So we need to have browser based DDP client and functionality written in static JavaScript file which will consume the API from DDP client and subscribe to the data with chat messages located in the Mongo database connected to the host app built with Meteor.

Fortunately there are plenty of DDP clients out there, so we don't need to write it by ourselves. Check the list linked above or just search for it on Google. We can find not only standalone JavaScript DDP clients, but also clients written in languages like PHP, Ruby or ObjectiveC.

Let's focus on our browser based DDP client. We will be using DDP.js client in version 0.6.0 because it is bundled to use in the browser. We could also use higher versions or something else like Asteroid but then we would need to bundle it for the browser usage. Let's stick to the 0.6.0 version, which works quite well.

What it gives us is a clean API to use DDP in the browser. So we'll be able to connect to our host app built with Meteor. We'll start building it in the next chapter.

Chat Host App built with Meteor

What will be our host app? It will be an example application built with Meteor which will be used to manage chat sessions from outside clients which will connect to it through chat widgets placed on many different websites. It is all possible because we'll use DDP. In Meteor, it is integrated in the core, and our chat widgets will use browser based DDP implementation. So basically Meteor app will provide Mongo collections where all messages will be stored and DDP connection will synchronize data changes in all connected clients.

We'll go through a basic example here. But at the end of the article, there will be a link to a ready and working product, which is also open sourced and MIT licensed, so it is a source of additional knowledge on that topic.

Be aware that here we will be building a very simple app that will not support user accounts and it will also not be secured. Anyway, there will be an opportunity to compare it with a production-ready app.

Ok, enough introduction, let's get into the code.

We'll start from the beginning. We need to have Node installed. Also we need to have Meteor and Git.

After that we can create our Meteor host app. We can do this in a standard way by running meteor create live-chat-app, but the better way would be to clone the repository from GitHub, which is a simple, ready-to-use Meteor host app.

We can clone it by running:

$ git clone https://github.com/juliancwirko/live-chat-app-demo-host.git

(Or we can download the .zip file).

Then we can also run the app:

$ cd live-chat-app-demo-host
$ meteor

Let's explore the cloned repository. From Meteor in version 1.3, we can use ES2015 imports. There is a nice trick in Meteor. All files which we want to import in our Meteor app should be placed in the imports directory. They will be lazy evaluated. All files outside this directory will run as the app start. We will partially stick to the Meteor Guide Application Structure. So we have all functionality in the imports directory and three starting points outside this directory which will import all files on the server side, client side and both sides. Here we're talking about the main.js files located in folders which are called, respectively, server, client and both.

This is all about the app's structure. We could of course rearrange it as we want. We just need to know how it works in Meteor. Read more in the Meteor Guide.

First of all, we need to take a look at our core functionality which is a MongoDB storage. We need to store our client apps and chat messages somewhere. This is done in the imports/both/collections.js. We need to declare two new Mongo collections and we need to export them.

import { Mongo } from 'meteor/mongo';

export const Chat = new Mongo.Collection('chat');
export const Client = new Mongo.Collection('client');

That’s basically all for now. We’ll use them later and we'll be able to import them and use in other files.

In this demo app, we have all client side functionality in one imports/client/layout.js file. We could split it into many smaller files too. We just need to remember about imports and order of imported files. For now, let's stick to the one file. It’ll be simpler. We can ignore the index.js files in each folder, which are there just for better readability when we import our files to the main.js files outside the imports folder.

In imports/client/layout.js the first, we import all necessary tools which comes from the Meteor core.

import { Template } from 'meteor/templating';
import { ReactiveDict } from 'meteor/reactive-dict';
import { $ } from 'meteor/jquery';
import { Chat, Client } from '../both/collections.js';

import './layout.html';

We need a templating layer and a reactive dictionary which will be responsible for managing the local app's state. We can also import jQuery and of course our two previously declared MongoDB collections. We also need to import our HTML templates here.

Each template has its helpers and events maps. We've implemented basic logic here to add and display client apps and chat messages. For example, let's take a look at the main template event to add a new client app:

Template.main.events({
    'click .js-add-new-client-app'() {
        const instance = Template.instance();
        const name = instance.$('[name=client-app-name]').val();
        if (name) {
            Meteor.call('addClientApp', name);
            instance.$('[name=client-app-name]').val('');
        }
    }
});

All we need to know here is that the app will provide a form through which we can add a new client app. Every newly created client app has a unique id. This id is important because it will connect chat boxes on the website with this particular client app.

Let's see how our data is managed here.

We have two collections (declared before) - Client and Chat. In the first one, we store all created client apps. In the second one we store all chat messages which will be assigned to the client apps and chat sessions. On the client side we need to subscribe to the data. For example:

Template.main.onRendered(function () {
    const instance = this;
    instance.subscribe('Client.appsList');
    instance.autorun(() => {
        instance.subscribe('Chat.list', state.get('openedApp'));
    });
});

We can do this because on the server side we have configured our publications in the imports/server/publications.js. So we are able to subscribe to this particular data set on the client and listen for changes. This is how our publications on the server looks:

import { Meteor } from 'meteor/meteor';
import { check } from 'meteor/check';
import {Chat, Client} from '../both/collections.js';

Meteor.publish('Client.appsList', () => Client.find());
Meteor.publish('Chat.list', (clientAppId) => {
    check(clientAppId, String);
    return Chat.find({clientAppId: clientAppId});
});
Meteor.publish('Chat.messagesList', (clientAppId, userSessionId) => {
    check(clientAppId, String);
    check(userSessionId, Match.Optional(String));
    return Chat.find({clientAppId: clientAppId, userSessionId: userSessionId}, {sort: {date: 1}});
});

Read more about Meteor's pub/sub system in the Meteor Guide Publications and Data Loading.

So basically, everything is located in the client side layout.js file. We subscribe for the data which is published on the server side, but we also have a form with an event which fires a Meteor Method called addClientApp and the other one in the chatBox template which will fire a Meteor Method called addChatMessage. We use Meteor Methods to add and edit data. They are declared in the imports/server/methods.js file. The addChatMessage method will be used also by client side chat boxes.
Methods declared on the server side:

import { Meteor } from 'meteor/meteor';
import { check } from 'meteor/check';
import {Chat, Client} from '../both/collections.js';

Meteor.methods({
    addClientApp(name) {
        check(name, String);
        Client.insert({name: name});
    },
    addChatMessage(msg, clientAppId, userSessionId) {
        check(msg, String);
        check(clientAppId, String);
        check(userSessionId, String);
        Chat.insert({
            msg: msg,
            clientAppId: clientAppId,
            userSessionId: userSessionId,
            date: new Date()
        });
    }
});

Everything is ready here. Take a break, read the code in the repository and try to understand what is going on. Run the app and play with it. There will be no chat messages yet, but we can create our client apps.

If you are ready we can take a look at the second part of the app which will be a client chat box for our static HTML website and another Meteor app.

Connect to the Host App from a Standard HTML Website.

We have our host app ready. We can go and run it now in the terminal. If it is not running, go to the live-chat-app-demo-host and run meteor command. Then go to the localhost:3000 in your browser and create a new client app.

Next we can clone our second repository, which is a static HTML website with implemented chat box client. Go and clone it by:

$ git clone https://github.com/juliancwirko/live-chat-app-demo-html-client.git

(Or we can download the .zip file).

Then we can go to the live-chat-app-demo-html-client folder and we can edit index.html file. We need to provide our unique id. Copy it from the host app and paste it in the liveChat.init('your_client_app_id_here'). Then open the index.html file in the browser. We should see a simple static HTML website with a chat box opened. We should be able to write in the chat box and read the messages in the host app (running at localhost:3000) when we choose a proper client app and chat session. Remember that our host app should be running.

But let's see how it is possible.

In the live-chat-app-demo-html-client website, we have three files: index.html, live-chat-app-client.js, live-chat-app-client.css. We'll take a look at the two last files. Basically the .js file because the .css file is just a styling for our chat box displayed at the bottom right corner.

In the live-chat-app-client.js file, we use DDP.js in version 0.6. It is hardcoded in the same file just for the convenience, but you can also include it in the index.html file as a separated library.

We have here a whole chat box implementation so also a chat box widget construction process and DOM manipulations. Our JavaScript init(...) method here will create chat box from the ground and it will also initialize DDP connection. I will not describe all DOM operations which creates chat box. You can go through code - it shouldn't be hard to read.

The most important part here is DDP.js library and its usage. We can see that we need to initialize DDP.

var ddp = new liveChatDDPClient({
    endpoint: 'ws://localhost:3000/websocket',
    SocketConstructor: WebSocket
});

We need to provide WebSocket endpoint which in our case is ws://localhost:3000/websocket because our host app runs at localhost:3000. We need to provide SocketConstructor. We use WebSocket here, but we could also use the SockJS library.

When we initialize our DDP connection, we can listen for the connected event. We do this by:

ddp.on('connected', function () {...});

After successful connection, we can run chat box creation and we can subscribe to our data served on the host app. We can subscribe to the same publications which are used on the host app. We can do something like:

ddp.sub('Chat.messagesList', [clientAppId, userSessionId]);

We take clientAppId from the provided value in the index.html file in the liveChat.init(...) method. and we also need to generate unique userSessionId, which we do by using the Random package from Meteor. Now we can listen for added event (new messages) and do something with this. In our case, we just need to attach new message to the previously created chat box.

This is basically all we need here in this client chat box. All new messages for this particular userSessionId and clientAppId will be placed in one MongoDB collection, which is served by host app. If you are an operator and you want to reply to the client who is writing to you from the static HTML website, you can do it by opening proper client app (click on it) and then by clicking on proper chat session (which will appear below). All messages in both chat boxes should be synchronized so you’ll be able to chat.

Now, let's play with it. We can create more client apps in the host app. We can connect more static websites just by copy and paste.

When you are ready, we can take a look at another use case which is a Meteor client app. As we know, Meteor has DDP support in its core. Imagine that you want to use chat box, not on static website, but on another Meteor app. Do we need to use the same static JavaScript file? Of course not. Let's see why.

Connect to the Host App from Another Meteor App.

Again, we'll clone the third ready-to-use repository. Now it’s another simple Meteor app. Clone it:

$ git clone https://github.com/juliancwirko/live-chat-app-demo-meteor-client.git

(Or we can download the .zip file).

We can run it as usual with different port because our host app needs to be running and it runs on the default Meteor port which is 3000:

$ cd live-chat-app-demo-meteor-client
$ meteor -p 5050

When we run the app and go to localhost:5050, we should see the very similar view as the previous one from the static HTML website. The functionality is also the same. We need to connect to the host app and this is it. It should work.

Meteor.startup(function () {
    liveChat.init('AQne8yYkemE923dqE');
});

So, what is different? We don't use DDP.js here because it is not needed. We have access to the core DDP library from Meteor so we should use this one.

In our app, we just initialize liveChat. All functionality lays in the local package in /packages/live-chat-meteor-client. We initialize DDP in a different way. If we want to connect to our host app we should do something like:

ddp = DDP.connect('http://localhost:3000');

Remember that our host app should be running at localhost:3000.

Next we need to create new MongoDB collection which will synchronize data with the host app.

chatCollection = new Mongo.Collection('chat', {connection: ddp});

We need to provide a connection parameter, which is our newly initialized DDP connection.

Next we can subscribe to the data published on the host app.

ddp.subscribe('Chat.messagesList', clientAppId, userSessionId);

We can also get the data from the collections by:

messages = chatCollection.find({userSessionId: userSessionId}, {sort: {date: 1}});

Take a look at the local package in /packages/live-chat-meteor-client there are also files which will build our chat box, CSS files and all Blaze events and helpers which will populate our chat box with messages from MongoDB served by the host app.

So as we can see, in the case of another Meteor app, we don't need what we did in the previous example. We can use Meteor's core components to achieve the same effect. This is because Meteor apps are designed to be able to speak to each other with ease.

Ending thoughts

I've prepared three simple repositories which you can clone and play with. There is the Live Chat Meteor host app which provides functionality to create and store new client apps. There is functionality which makes it possible to reply to each connected client. There are also simple static browser chat box client app and the Meteor one which is very similar but uses a different DDP client, the core one.

These are of course only demo projects. You shouldn't use them in the production. They is just for learning and testing purposes. Although if you are interested, I've built a simple production-ready live chat app which is called SimpleChat.Support. It is also Open Source and MIT licensed. It is secured, has user account support, an awesome administrator panel and the client chat boxes are much more usable. You can use it, learn from the code or just built upon it. You'll find a link to the GitHub repo on the main website.

If you want to comment or improve the article just write to me. I'm open to all suggestions.

Comments

comments powered by Disqus