By Raul Nohea Goodness
https://twitter.com/rngoodness
November 2021
Overview
I am in the middle of my about once or twice-a-decade process of reevaluating my entire web software development tools and approaches. I’m using a number of great new tools, but a new little JS lib i’m starting to use to tie them all together, hopefully in a resilient way: graphql-ws
The target audience for this post is a web developer using a modern GraphQL backend (Hasura in this case) and a modern reactive javascript/html front-end (in this example, Svelte).
This post is about the reasons for my tech stack choices, and how to use graphql-ws to elegantly tie them together.
Software Stack Curation
What is Hasura? Why use it?
Hasura, in my mind, is a revolutionary new backend data server. It sits in front of a database (PostgreSQL or others), and provides:
- Instant GraphQL APIs (which are typed)
- Configurable Authorization of resources, integrated at the SQL composition level
- Bring your own Authentication/Authorization provider (using JWTs or not), such as NHost, Auth0, Firebase, etc.
- Integrate with other GraphQL sources
- Integrate hooks with serverless functions/lambda
- Open Source (self-host or cloud service)
- Local CLI for developers
This can eliminate extensive hand-coding of backend REST APIs, with the custom cross-cutting concerns, like Auth. Also replaces the need for OR/M-style data access in backend code.
What is GraphQL? Why use it?
Now, this was my question a couple years ago, until i saw Hasura v1. Now i can answer it.
From graphql.org:
A query language for your API
GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.
In slightly more normal coder-speak, it is a defacto standard for querying and mutating (insert/update/delete) a data source over the web, sending a GQL statement, executing it, and receiving a response in JSON.
GraphQL consoles are “aware” of the underlying database types, which makes it easy to:
- Compose queries in a console like “graphiql” and test them
- Then copy/paste your GQL into your Javascript client code editor, for run-time execution
Arguably, this is less work than hand-coding the SQL or ORM code into your REST endpoint code. The JSON response comes for free. GraphQL also makes it easy to merge multiple data sources into a single JSON response.
What is Apollo Client? Why use it, and why would i not use it?
Apollo is a GraphQL client and server library. It is popular and there are many code examples for developers available. Since i am using Hasura i don’t need the Apollo Server, but i could use the Apollo Client to access my backend.
My initial tests worked with it. However the Apollo Client also has its own state-management system for caching gql queries and responses. It seemed like an overkill solution for my uses. I’m sure it works for other projects, but since the new concept count (to learn) in this case is already high, i opted to not use it.
Instead i started using a more lightweight library: https://graphql.org/graphql-js/graphql/
This worked good and was simple to understand, but only for queries and mutations, not subscriptions.
For gql subscriptions, there was a different library: apollographql / subscriptions-transport-ws . It is for graphql subscriptions over web sockets. We would want this in the case of a web UX which listens for changes in the underlying data, and reactively updates when it changes on the server.
What is graphql-ws? Why use it instead of subscriptions-transport-ws?
subscriptions-transport-ws does work, but there are 3 reasons not to use it:
- Bifurcated code – you have to use one lib for gql queries+mutations, and another for subscriptions
- graphql-ws implements a more standard GraphQL over WebSocket Protocol, using ping/pong messages, instead of subscriptions-transport-ws GCL_* messages.
- Apparently subscriptions-transport-ws is no longer actively maintained by the original developers, and they recommend using graphql-ws on their project page.
Note that Hasura added support for graphql-ws protocol as of v2.0.8.
What are graphql subscriptions?
From the Hasura docs:
Subscriptions – Realtime updates
The GraphQL specification allows for something called subscriptions that are like GraphQL queries but instead of returning data in one read, you get data pushed from the server.
This is useful for your app to subscribe to “events” or “live results” from the backend, while allowing you to control the “shape” of the event from your app.
GraphQL subscriptions are a critical component of adding realtime or reactive features to your apps easily. GraphQL clients and servers that support subscriptions allow you to build great experiences without having to deal with websocket code!
Put another way, our front-end UX can “listen” for changes on the backend, and the backend will send the changes to the frontend over the web socket in real time. The “reactive” frontend can instantly re-render the update to the user.
Not all graphql queries require using a subscription, but if we do use them, coding them will be much simpler to write and maintain.
What is Svelte? Why use it?
Svelte is a Javascript front-end reactive framework (not unlike React), but it is succinct and performant, and implemented as a compiler (to JS). Plus, it is fun to learn and code in. I’m talking 1999- era fun 😊
I recommend watching Rich Harris’ talk: Rethinking Reactivity
You can use a different frontend framework. But Svelte makes it easy due to svelte “stores” implementing the observable contract– the subscribe() method. Components will reactively update if the underlying object being observed changes.
What are Javascript Observables? RxJS?
RxJS is a library for reactive programming using Observables, to make it easier to compose asynchronous or callback-based code.
We don’t need RxJS to use observables, but it is a popular library. I used it with Angular in the past, and one of the simplest graphql-ws examples uses it, so i am too.
In short, in Javascript you call an observable’s subscribe() method to listen for/handle updates in the observable’s value.
The wire-up: two-way reactive front-end to backend using JS Observables + GraphQL Subscriptions over Web Sockets
The idea here is to render the rapidly-changing data in an HTML component for the user to just watch updates, without having to do anything.
Design – Focus-group input slider
This proof-of-concept will be a slider for use in a “focus group”. A group of participants get in a room and watch a video or debate, and “dial” in real-time their opinion (positive or negative) as things are being said. This example will just be a single person’s input being displayed or charted.
- The data captured will include: focus group id (text), username, rating (integer – 0 to 100), and datetime of event.
- UI will include:
- A start/stop button, to start recording rating records, in 1 second increments.
- A slider, which goes from 0 (negative/disagree) to 100 (positive/agree), default setting is 50 (neutral)
- A grid/table will display the last 10 records recorded in real-time (implemented as a graphql subscription).
- Optional: implement a chart component which updates in real-time from the data set.
Diagram
Code
Setup
npm init svelte@next fgslider
cd fgslider
code .PostgreSQL
I want the table to look like this:
create table ratingtick (
id serial,
focusgroup text not null,
username text not null,
rating integer not null,
tick_ts timestamp with time zone not null default current_timestamp
);
-- insert into ratingtick(focusgroup, username, rating) values ('pepsi ad', 'ekolu', 50);
-- insert into ratingtick(focusgroup, username, rating) values ('pepsi ad', 'ekolu', 65);
-- insert into ratingtick(focusgroup, username, rating) values ('pepsi ad', 'ekolu', 21);In this case, i’m going to do it on my local machine. I’m also going to create the Hasura instance locally using hasura-cli. Of course, you can do this on your local infrastructure or your own servers or cloud provider, or the specialized NHost.io .
Hasura
I’m going to create a Hasura container locally, which will also have a PostgreSQL v12 instance.
sudo apt install docker-compose docker
docker-compose up -d
If you get a problem, just tweak docker-compose.yml. I changed the port from 8080:8080 to 8087:8080
Connect to the Hasura web console:
http://localhost:8087
Connect the Hasura container instance to the Postgresql container instance:
Grab the database URL from docker-compose.yml and connect the database:
You will now see ‘pgcontainer’ in the databases list.
With Hasura, you can either create the Postgres schema first, then tell Hasura to scan the schema. Or create the schema in the Hasura console, which will execute the DDL on Postgres. Pick one or the other.
For this project, we’ll skip permissions, or more accurately, we’ll configure a ‘public’ role on the table, and allow select/insert/update/delete permissions.
Note: i had to add HASURA_GRAPHQL_UNAUTHORIZED_ROLE: public to the environment: section of docker-compose.yml and run “docker-compose up -d” to make it reload with the setting change, to treat anonymous requests as ‘public’ role (no need for x-hasura-role header).
Let’s now test GraphQL queries in “GraphIQL” tool. We should be able to set the x-hasura-role header to ‘public’ and still query/mutate. Setting the role header requires Hasura to evaluate the authorization according to that role. (note i did have problems getting the role header to work, so i instead made ‘public’ the default anonymous role).
We should also be able to insert via a mutation:
mutation MyMutation {
insert_ratingtick_one(object: {focusgroup: "pepsi ad", username: "ekolu", rating: 50}) {
id
}
}Response:
{
"data": {
"insert_ratingtick_one": {
"id": 1
}
}
}That means it inserted and returned the ‘id’ primary key of 1.
After inserting a few, we can query again:
{
"data": {
"ratingtick": [
{
"id": 1,
"focusgroup": "pepsi ad",
"username": "ekolu",
"rating": 50,
"tick_ts": "2021-11-16T22:56:07.094606+00:00"
},
{
"id": 2,
"focusgroup": "pepsi ad",
"username": "ekolu",
"rating": 45,
"tick_ts": "2021-11-16T22:57:56.323054+00:00"
},
{
"id": 3,
"focusgroup": "pepsi ad",
"username": "ekolu",
"rating": 98,
"tick_ts": "2021-11-16T22:58:01.047135+00:00"
},
{
"id": 4,
"focusgroup": "pepsi ad",
"username": "ekolu",
"rating": 96,
"tick_ts": "2021-11-16T22:58:09.495674+00:00"
},
{
"id": 5,
"focusgroup": "pepsi ad",
"username": "ekolu",
"rating": 43,
"tick_ts": "2021-11-16T22:58:17.550324+00:00"
},
{
"id": 6,
"focusgroup": "pepsi ad",
"username": "ekolu",
"rating": 23,
"tick_ts": "2021-11-16T22:58:25.547917+00:00"
}
]
}
}Finally, let’s test the subscription. We should be able to open the insert mutation in one window, and see the subscription update in real time in the second window.
Good. At this point, i’m confident all is working on the Hasura end. Time to work on the front-end code.
Svelte + graphql-ws
Please note that although i am using Svelte with graphql-ws , you can use any JS framework, or vanilla JS.
Remember, we created this directory as a sveltekit project, so now we’ll build on it. We do need to “npm install” to install the node dependencies. Then we can “npm run dev” which will run the dev http server on localhost:3000
- Create a new /slider route as src/routes/slider/index.svelte
- Add form inputs, and a slider widget
- Add a display grid which will display the last 10 tick records
SvelteKit uses Vite for modern ES6 module builds, which uses dotenv-style .env, but with the VITE_* prefix. So we create a .env file with entry like so:
VITE_TEST="this is a test env"
VITE_HASURA_GRAPHQL_URL=ws://localhost:8087/v1/graphqlNote: you must change the URI protocol from http://localhost:8087/v1/graphql to ws://localhost:8087/v1/graphql , in order to use graphql-ws. It is not normal http, it is web sockets (ws://) or ws secure (wss://). Otherwise, you get an error: [Uncaught (in promise) DOMException: An invalid or illegal string was specified client.mjs:140:12]
Then you can refer to them in your app via the import.meta.env.* namespace (src/routing/index.svelte):
Now let’s get into the “fish and poi” a/k/a “meat and potatoes” section, the src/routes/slider/index.svelte page.
First, the start/stop button, form elements and slider widget. Keeping it simple,
I will install a custom svelte slider component:
npm install svelte-range-slider-pips --save-dev
Also installing rxjs, for the timer() and later for wrapping the graphql-ws handle.
npm install rxjs
The first version here is basically a svelte app only, not using any backend yet:
<script>
import { text } from "svelte/internal";
import RangeSlider from "svelte-range-slider-pips";
import { timer } from 'rxjs';
let runningTicks = false;
let focusGroupName = "pepsi commercial";
let userName = "ekolu";
let sliderValues = [50]; // default
let tickLog = "";
let timerObservable;
let timerSub;
function timerStart() {
runningTicks = true;
timerObservable = timer(1000, 1000);
timerSub = timerObservable.subscribe(val => {
tickLog += `tick ${val}... `;
});
}
function timerStop() {
timerSub.unsubscribe();
runningTicks = false;
}
</script>
<h1>Slider</h1>
<p>
enter your focus group, name and click 'start'.
</p>
<p>
Once it starts, move the slider depending on how much you
agree/disagree with the video.
</p>
<form>
<label for="focusgroup">focus group: </label><input type="text" name="focusgroup" bind:value={focusGroupName} />
<label for="username">username: </label><input type="text" name="focusgroup" bind:value={userName} />
<label for="ratingslider">rating slider (0 to 100): </label>
<RangeSlider name="ratingslider" min={0} max={100} bind:values={sliderValues} pips all='label' />
<div>0 = bad/disagree, 50 = neutral, 100 = good/agree</div>
<div>slider Value: {sliderValues[0]}</div>
<button disabled={runningTicks} on:click|preventDefault={timerStart}>Start</button>
<button disabled={!runningTicks} on:click|preventDefault={timerStop}>Stop</button>
</form>
<div>
Tick output: {tickLog}
</div>
<div>
<a href="/">Home</a>
</div>I got a number of things working together here:
- Variables bound to UI components
- A slider component which will have values from 0 to 100, bound to variable
- An rxjs timer(), which executes a callback every second, bound to the start/stop buttons
Now i’m ready to hook up the graphql mutation and subscription.
npm install graphql-ws
I’m going to create src/lib/graphql-ws.js to manage the client setup and subscription creation.
import { createClient } from 'graphql-ws';
import { observable, Observable } from 'rxjs';
export function createGQLWSClient(url) {
// console.log(`createGQLWSClient(${url})`);
return createClient({
url: url,
});
}
export async function createQuery(client, gql, variables) {
// query
return await new Promise((resolve, reject) => {
let result;
client.subscribe(
{
query: gql,
variables: variables
},
{
next: (data) => (result = data),
error: reject,
complete: () => resolve(result)
}
);
});
}
export async function createMutation(client, gql, variables) {
// same as query
return createQuery(client, gql, variables);
}
export function createSubscription(client, gql, variables) {
// hasura subscription
// console.log("createSubscription()");
const operation = {
query: gql,
variables: variables,
};
const rxjsobservable = toObservable(client, operation);
// console.log("rxjsobservable: ", rxjsobservable);
return rxjsobservable;
}
// wrap up the graphql-ws subscription in an observable
function toObservable(client, operation) {
// console.log("toObservable()");
// the graphql-ws subscription may be cleaned up here,
// not sure about the RxJs Observable
// trying to make it more like the docs, w/custom unsubscribe() on subscription object
// https://rxjs.dev/guide/observable
return new Observable(function subscribe(subscriber) {
client.subscribe(operation, {
next: (data) => subscriber.next(data),
error: (err) => subscriber.error(err),
complete: () => subscriber.complete()
});
// Provide a way of canceling and disposing resources
return function unsubscribe() {
console.log("unsubscribe()");
};
});
}Now we are going to:
- setup the client in the index.svelte onMount() handler,
- execute createSubscription() in the onMount() handler and bind to a new grid/table component
- execute createMutation() on every tick with the current values
// browser-only code
onMount(async () => {
// setup the client in the index.svelte onMount() handler
gqlwsClient = createGQLWSClient(import.meta.env.VITE_HASURA_GRAPHQL_URL);
// execute createSubscription() in the onMount() handler
// and bind to a new grid/table component
// src/components/TopTicks.svelte
const gql = `subscription MySubscription($limit:Int) {
ratingtick(order_by: {id: desc}, limit: $limit) {
id
focusgroup
username
rating
tick_ts
}
}`;
const variables = { limit: 5 }; // how many to display
const rxjsobservable = createSubscription(
gqlwsClient,
gql,
variables
);
// const subscription = rxjsobservable.subscribe(subscriber => {
// console.log('subscriber: ', subscriber);
// });
// console.log('subscription: ', subscription);
// gqlwsSubscriptions.push(subscription);
gqlwsObservable = rxjsobservable;
});Timer start
function timerStart() {
runningTicks = true;
timerObservable = timer(1000, 1000);
timerSub = timerObservable.subscribe((val) => {
tickLog += `tick ${val}... `;
// execute createMutation() on every tick with the current values
submitLatestRatingTick(gqlwsClient);
});
}Functions to do the work:
function submitLatestRatingTick(client) {
const gql = `mutation MyMutation($focusgroup:String, $username:String, $rating:Int) {
insert_ratingtick_one(object: {focusgroup: $focusgroup, username: $username,
rating: $rating}) {
id
}
}
`;
const variables = buildRatingTick();
createMutation(client, gql, variables);
}
function buildRatingTick() {
return {
focusgroup: focusGroupName,
username: userName,
rating: sliderValues[0]
};
}Note, we can test the gql in GraphIQL and copy/paste into the JS template strings, also using the variable syntax.
Update: One more thing, i forgot to include my <TopTicks> component code:
<script>
export let observable;
</script>
{#if $observable}
<h3>Top Ticks</h3>
<table>
<thead>
<tr>
<th>id</th>
<th>focus group</th>
<th>user</th>
<th>rating</th>
<th>tick ts</th>
</tr>
</thead>
<tbody>
{#each $observable.data.ratingtick as item}
<tr>
<td>{item.id}</td>
<td>{item.focusgroup}</td>
<td>{item.username}</td>
<td>{item.rating}</td>
<td>{item.tick_ts}</td>
</tr>
{/each}
</tbody>
</table>
{/if}We pass the gqlwsObservable to the svelte component in a prop:
<TopTicks observable={gqlwsObservable} />If that all works, we will have a sweet reactive graphql-ws app.
Got it all working now! 😎
Animated GIF version:
Connecting it to a reactive chart is left as an exercise for the reader.
github
find the source for this app here:
https://github.com/nohea/enehana-fgslider
References
graphql-ws: GraphQL over WebSocket Protocol compliant server and client.
https://github.com/enisdenjo/graphql-ws
SpinSpire: live code: Svelte app showing realtime Postgres data changes (GraphQL subscriptions)