Sunday, November 17, 2024

Running a Ruby on Rails Application in Nix

I’m loving Nix. I recently wrote about how I use it to manage my new Mac, and I’m starting to wrap my head around how it can be used, not just for dotfile/package management, but individual application configurations as well.

Ruby on Rails is somewhat famously complicated to get running, both locally, and in production. Many Rails developers just install every dependency locally, directly onto their machine - Homebrew has certainly made this easier - and part of what made Heroku so popular was how easy it was to deploy a Rails app.1

I recently had to pull down one of my older Rails apps (my Shopify app Reporty) and try to get it running on my new Mac. I decided to try Nix to get it working. This config is good for me, but it’s not necessarily a fully-featured, batteries-included approach to Rails on Nix. Hopefully you can take it and expand it as needed for your own projects.

Note: this assumes you already have Nix installed. Check out my starter macOS Nix config for some links to get started.

First, we need to initialize a new flake. This is a Nix-specific way of organizing your project. Notably, it generates a lockfile much like Gemfile.lock for Ruby projects. Run this command:

$ nix flake init

This will create a flake.nix file in your project directory. This file is where you’ll define your Nix configuration. Inside of the file, we’ll set up a shell environment for our project. Here’s what mine looks like:

{
  description = "Rails app";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }: {
    devShells = {
      aarch64-darwin.default = import ./shell.nix {
        pkgs = import nixpkgs { 
          config.allowUnfree = true;
          system = "aarch64-darwin"; 
        };
      };
    };
  };
}

A few things worth noting here:

  • We’re using the aarch64-darwin system, which is the architecture of my new Mac. You can change this to whatever architecture you’re using.
  • We’re using the allowUnfree flag. This is a flag that allows you to install packages that aren’t in the NixOS package repository. This is useful for installing things like ngrok, which is a tool I use to test my app locally.

Now, we can create a shell.nix file. This is where we’ll define the environment for our project. Here’s what mine looks like:

{ pkgs ? import <nixpkgs> {} }:
with pkgs; 
let 
  postgresDataDir = "./pgdata";
in mkShell {
  buildInputs = [
    bundler
    ngrok
    nodejs
    redis
    ruby_3_3
    postgresql
  ];

  shellHook = ''
    echo "Setting up PostgreSQL..."
    export PGDATA=${postgresDataDir}
    export PGHOST=localhost
    export PGPORT=5432
  
    # Check if the data directory exists to avoid reinitialization
    if [ ! -d "$PGDATA" ]; then
      echo "Initializing PostgreSQL data directory at $PGDATA..."
      initdb --locale=en_US.UTF-8 -D $PGDATA

      echo "Starting PostgreSQL..."
      pg_ctl -D $PGDATA -l logfile start

      # Optionally create your database here
      createdb reportydb

      echo "Database setup is complete. PostgreSQL running."
    else
      echo "Starting existing PostgreSQL instance..."
      pg_ctl -D $PGDATA -l logfile start
    fi

    echo "PostgreSQL setup is ready!"
  '';
}

The shell.nix file is where most of the setup happens. Here’s what it does:

  1. Installs all needed dependencies, like ruby, postgresql, redis, and nodejs.
  2. Uses the shellHook function to set up a PostgreSQL database.
  3. Wraps all these steps in a mkShell function, which is a Nix-specific way of defining a shell environment.

The buildInputs are packages from Nix’s package repository. These are the packages that are installed when you run nix-env -i <package>. You can see the full list of packages here.

The good part about this is that these packages are already pre-compiled for your system architecture, so they are extremely quick to install. If you need to build a package manually - for instance, if you have a library that you need to compile against on your machine - you can use nativeBuildInputs instead.2

Now, we can run nix develop to enter the shell. This will take a few minutes, as it’s downloading all the dependencies. Once it’s done, you should be able to run bundle install and rails server to get your app running.

This was my minimal setup for running a Rails app locally in Nix. I haven’t deployed this config to a server yet - in production, at least - so I’ll cover that in a future post. But I enjoyed this approach because it declaratively sets up all the dependencies for my project, and due to the characteristics of Nix, the build is isolated from the other dependencies on my machine. Nice!

Footnotes

  1. To be fair - this is something the Rails devs care about a lot. DHH has worked on Omakub, a wrapper around Ubuntu that installs all the needed dependencies for a fully-functional dev environment. And Kamal is a new tool that makes it easy to deploy Docker containers on remote servers. There’s a lot of work going on right now to improve the experience!

  2. Man in the arena moment! I began by using nativeBuildInputs for all of the dependencies, and wondered why it was so slow, particularly to compile Ruby. It seemed strange. I went and looked at the Nix docs, and realized that instead of compiling it by hand, I can rely on Nix having a pre-compiled version for my aarch64-darwin architecture by using buildInputs instead.

Sunday, November 10, 2024

Beverage Hydration Index

What is hydration?

Being hydrated is having enough water and electrolytes in your body. When you sweat, breath, and urinate, you lose water in your body.

Hydration is the process of re-adding water into your body so that it can continue to use and circulate water throughout your body.

Electrolytes help regulate the body’s usage and retaining of fluids. If the body has sufficient levels of electrolytes, it helps combat dehydration. Elecrolytes help make this happen by regulating the amount of water that is expelled from your body. This is why sport drinks like Gatorade are so popular for athletes: they have high levels of electrolytes, which help them retain fluids in their body.

We drink water to hydrate. We know that water is of crucial importance to our bodies - obviously - but what if water isn’t the best way to rehydrate your body? Specifically, is “pure” water all we need?

Beverage Hydration Index

The Beverage Hydration Index (BHI) was developed by researchers from the University of Stirling in Scotland. The concept was introduced in a study published in The American Journal of Clinical Nutrition in 2016. The BHI evaluates the hydrating potential of different beverages compared to water by measuring how much and how quickly a liquid is absorbed and retained in the body over time.

The BHI and studies covering it look at how different types of beverages contribute to your level of hydration. This happens by tracking urination. If you drink a drink X versus drink Y, you can compare the urine content and see how a human body stays hydrated by measuring the mass of urine being expelled.

Coca-Cola versus water

This leads to a funny fact that I saw mentioned on 𝕏 over the weekend: Coca-Cola is better than water for rehydration:

I spent some time researching it and found the source:

Here’s the full abstract of the study:

Background: The identification of beverages that promote longer-term fluid retention and maintenance of fluid balance is of real clinical and practical benefit in situations in which free access to fluids is limited or when frequent breaks for urination are not desirable. The postingestion diuretic response is likely to be influenced by several beverage characteristics, including the volume ingested, energy density, electrolyte content, and the presence of diuretic agents.

Objective: This study investigated the effects of 13 different commonly consumed drinks on urine output and fluid balance when ingested in a euhydrated state, with a view to establishing a beverage hydration index (BHI), i.e., the volume of urine produced after drinking expressed relative to a standard treatment (still water) for each beverage.

Design: Each subject (n = 72, euhydrated and fasted male subjects) ingested 1 L still water or 1 of 3 other commercially available beverages over a period of 30 min. Urine output was then collected for the subsequent 4 h. The BHI was corrected for the water content of drinks and was calculated as the amount of water retained at 2 h after ingestion relative to that observed after the ingestion of still water.

Results: Total urine masses (mean ± SD) over 4 h were smaller than the still-water control (1337 ± 330 g) after an oral rehydration solution (ORS) (1038 ± 333 g, P < 0.001), full-fat milk (1052 ± 267 g, P < 0.001), and skimmed milk (1049 ± 334 g, P < 0.001). Cumulative urine output at 4 h after ingestion of cola, diet cola, hot tea, iced tea, coffee, lager, orange juice, sparkling water, and a sports drink were not different from the response to water ingestion. The mean BHI at 2 h was 1.54 ± 0.74 for the ORS, 1.50 ± 0.58 for full-fat milk, and 1.58 ± 0.60 for skimmed milk.

Conclusions: BHI may be a useful measure to identify the short-term hydration potential of different beverages when ingested in a euhydrated state. This trial was registered at www.isrctn.com as ISRCTN13014105.

It seems counter-intuitive to say Coca-Cola is objectively better than water at rehydration - although very amusing, taken at face value1). However, we can see from the chart in the study (which I screenshotted in my post) that Coca-Cola does lead to more fluid retention than plain water. Drinks like orange juice and milk can lead to even more fluid retention! But how does it work? Why do some drinks, like Coca-Cola or milk, more effectively rehydrate the body?

Glucose and electrolytes

The answer is two-fold: glucose, and electrolytes. To recap biology class, glucose is a sugar that your body uses for energy. When you eat food, particularly carbohydrates, your body converts them into glucose and either uses it immediately, or stores it for use later. In your intestines, glucose helps water transport more effectively by triggering sodium absorption. This is called sodium-glucose co-transport.

In the case of Coca-Cola, the presence of glucose through sugar - particularly in the excellent Mexican coke variety, which uses cane sugar2 instead of high-fructose corn syrup - effectively provides sugar that can help the body take water and transport it through your body. Amazing!

Milk scores even better on the Beverage Hydration Index. The reason is the high electrolyte, protein, and fat contents. Milk has high level of sodium and potassium, therefore helping the body stay hydrated after drinking fluids. The protein and fat slow the absorption of fluids, also helping with hydration.

This is a fun study that challenges some of the assumptions people have about rehydration. It is worth noting that caffeine is a known dieuretic, so I would still argue that if you’re on the hunt for rehydration specifically, drinking fruit juices or milk may be more effective. Coca-Cola in particular does have its uses (Ray Peat has written a lot about this), but I’ll classify this whole thing as a fun factoid that I dug into this weekend as I enjoy a nice Mexican coke. Biology is fun!

Footnotes

  1. This falls under the “funny factoids that I enjoy sharing in conversation” category - it seems crazy, has science to back it up, and is a fun thought exercise. Cool!

  2. Cane sugar is an excellent form of sugar. It contains sucrose, which is made up of roughly 50% glucose and 50% fructose.

Friday, November 8, 2024

My starter macOS Nix Config

I understand what Nix is. But I barely know how to use it. Earlier this week, I shared that I was interested in trying it on my new macOS machine, after upgrading to the new M4 MacBook Pro.

It arrived earlier today, and I got started setting up Nix. I still don’t 100% feel comfortable in it, but I learned enough that I was able to get it up and running in about an hour. Here’s what I did to get it running.

Installation

This guide assumes a brand new macOS install. Skip steps as needed if you’re using an existing machine.

First, you need to install the built-in Command Line Tools:

$ xcode-select --install

This is macOS’ build-essential-equivalent. A bunch of tools for compiling and building software. If you are a dev on an existing Mac setup, you already have this installed.

There’s a number of ways to install Nix. I chose to use Determinate System’s installer, which does a bunch of nifty stuff with separating the Nix config into a separate drive, and a bunch of sane defaults that I probably don’t even know I need:

$ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install --determinate

Setup

With Nix installed, we can install nix-darwin, which has a bunch of Mac-specific defaults.

I used the Nixcademy blog post “Setting up Nix on macOS” here to get an initial file to work off ot. The important part is to run these commands:

$ mkdir nix-darwin-config
$ cd nix-darwin-config
$ nix flake init -t nix-darwin

Where you put that directory doesn’t seem particularly important. I stuck mine in ~/Developer/nix.

You’ll get flake.nix, which is your Nix config. You can read the Nixcademy blog post above to learn more about how to customize it.

With everything set up, you can run the main command to re-init your Nix config:

$ nix run nix-darwin -- switch --flake .

The fun stuff

Now, it’s time to configure. Here’s my full Nix config that got me from start to finish with this repo, the one for the website you’re reading. That means installing Neovim, Zellij, and Node.js, as well as Homebrew. My username on my machine is kristian and the hostname for my machine is bilbo, so change as needed:

{
  description = "Darwin system flake";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    nix-darwin.url = "github:LnL7/nix-darwin";
    nix-darwin.inputs.nixpkgs.follows = "nixpkgs";
    nix-homebrew.url = "github:zhaofengli-wip/nix-homebrew";
    homebrew-core = {
      url = "github:homebrew/homebrew-core";
      flake = false;
    };
    homebrew-cask = {
      url = "github:homebrew/homebrew-cask";
      flake = false;
    };
  };

  outputs = inputs@{ self, nix-darwin, nixpkgs, nix-homebrew, homebrew-core, homebrew-cask }:

  let
    configuration = { pkgs, ... }: {
      # List packages installed in system profile. To search by name, run:
      # $ nix-env -qaP | grep wget
      environment.systemPackages = [
        pkgs.neofetch 
        pkgs.neovim 
        pkgs.nodejs_22
        pkgs.zellij
      ];

      nix-homebrew = {
        enable = true;
        enableRosetta = true;
        user = "kristian";

        taps = {
          "homebrew/homebrew-core" = homebrew-core;
          "homebrew/homebrew-cask" = homebrew-cask;
        };
      };

      homebrew = {
        enable = true;
        onActivation.cleanup = "uninstall";
        taps = [];
        brews = [];
        casks = [ "1password" "kitty" "plexamp" ];
      };

      # Auto upgrade nix package and the daemon service.
      services.nix-daemon.enable = true;
      # nix.package = pkgs.nix;

      # Necessary for using flakes on this system.
      nix.settings.experimental-features = "nix-command flakes";

      # Enable rosetta binaries
      nix.extraOptions = ''
        extra-platforms = x86_64-darwin aarch64-darwin
      '';

      # Enable alternative shell support in nix-darwin.
      # programs.fish.enable = true;

      # Set Git commit hash for darwin-version.
      system.configurationRevision = self.rev or self.dirtyRev or null;

      # Used for backwards compatibility, please read the changelog before changing.
      # $ darwin-rebuild changelog
      system.stateVersion = 5;

      # The platform the configuration will be used on.
      nixpkgs.hostPlatform = "aarch64-darwin";

      # Enable Touch ID support
      security.pam.enableSudoTouchIdAuth = true;

      # System settings
      system.defaults = {
        finder.AppleShowAllExtensions = true;
        finder.FXPreferredViewStyle = "clmv";
        loginwindow.LoginwindowText = "REWARD IF LOST: kristian@kristianfreeman.com";
        screencapture.location = "~/Pictures/Screenshots";
        screensaver.askForPasswordDelay = 10;

        # Dock
        dock = {
          autohide = true;
          mru-spaces = false;
          persistent-apps = [
            "/Applications/Safari.app"
          ];
        };
      };
    };
  in
  {
    # Build darwin flake using:
    # $ darwin-rebuild build --flake .#bilbo
    darwinConfigurations."bilbo" = nix-darwin.lib.darwinSystem {
      modules = [ 
        nix-homebrew.darwinModules.nix-homebrew
        configuration 
      ];
    };

    # Expose the package set, including overlays, for convenience.
    darwinPackages = self.darwinConfigurations."bilbo".pkgs;
  };
}

There’s some really interesting and impressive stuff here:

  • system.defaults.dock allows full Dock customization! You can programatically set which items are in your Dock, which is quite cool.
  • You can install Homebrew packages from within Nix. And with the onActivation.cleanup = "uninstall" config option, any packages you remove will get installed. It’s declarative Homebrew, which is super useful.

There’s sure to be a lot more I’ll do here. I’ve heard home-manager is cool, and now that I’m home for the winter, I’m dual-working on my Mac Studio and this laptop, so syncing config between them is sure to be useful. Excited to dig into this more!

Tuesday, November 5, 2024

Effective web clipping with Obsidian

I’m experimenting with Obsidian’s new-ish Web Clipper to capture web pages into my Obsidian vault.

It has a great feature where you can create a template for a “type” of web page, and use it to clip that web page in a specific format.

Here’s an example. If you clip a PubMed article, you can create a template for it like this:

PubMed Clipping Template

One neat trick: by putting the PMID field as an note alias, it will automatically recognize other documents and notes that reference that PMID, and put them as “Unlinked mentions”. This is great if an 𝕏 post mentions “PMID: $ID”, and I clip it into Obsidian with Readwise.

In this example, I’m grabbing the title, authors, and PMID from the page, using meta tags1. By default, grabbing the actual content of the page happens automatically, but you can also customize how the content looks as well:

PubMed Content Example

The Web Clipper uses the triggers field to determine which web pages to capture. If I run the extension on a URL that starts with https://pubmed.ncbi.nlm.nih.gov, it will use my “PubMed” template to clip the page.

You can export any of these templates to JSON, meaning you can import/export them to share with others. If you want my PubMed template, you can grab it below:

{
	"schemaVersion": "0.1.0",
	"name": "PubMed",
	"behavior": "create",
	"noteContentFormat": "{{content}}\n\nPubMed ID: {{meta:name:citation_pmid}}",
	"properties": [
		{
			"name": "pubmed_id",
			"value": "{{meta:name:citation_pmid}}",
			"type": "text"
		},
		{
			"name": "pubmed_title",
			"value": "{{meta:name:citation_title}}",
			"type": "text"
		},
		{
			"name": "pubmed_authors",
			"value": "{{meta:name:citation_authors|split:\\\",\\\"}}",
			"type": "multitext"
		},
		{
			"name": "url",
			"value": "{{url}}",
			"type": "text"
		},
		{
			"name": "aliases",
			"value": "{{meta:name:citation_pmid}}",
			"type": "multitext"
		}
	],
	"triggers": [
		"https://pubmed.ncbi.nlm.nih.gov"
	],
	"noteNameFormat": "{{title|split:\" - \"|first}}",
	"path": "Sources/PubMed"
}

Footnotes

  1. Meta tags are usually embedded in the <head> of a web page, and are formatted like <meta name="citation_authors" content="Smith, John, Doe, Jane">.

Tuesday, November 5, 2024

Install fonts with Homebrew

One of the best parts about Homebrew is Homebrew Cask. Cask allows you to install GUI apps for macOS using the standard brew commands. It used to be a separate installation, but now it is included in Homebrew itself.

Most people are familiar with Cask for installing GUI apps, but you may not know that it also allows you to install fonts. Installing fonts is usually an annoying process:

  1. Download the font family
  2. Unzip the archive
  3. Open every font and click “Install” in Font Book

With Homebrew Cask, you can do this in a single command:

$ brew install font-$fontname

Let’s install Atkinson Hyperlegible, the font I use on this site.

You can also search for any font by using brew search. This will show the exact cask name that you need to install the font:

$ brew search atkinson
==> Casks
font-atkinson-hyperlegible

I can run brew install to automatically install the font, including all variants:

$ brew install font-atkinson-hyperlegible

Monday, November 4, 2024

Cloudflare Stream

Cloudflare Stream is a service that allows you to host videos for your website. It can be purchased as part of a $10/month plan that includes Cloudflare Images, with up to 1k minutes of video hosting and 5k minutes of video delivery included.

In my Workers GraphQL Server post, I included a video walkthrough of how to use the project. It’s hosted with Stream.

Integration

Here’s how I integrated it into my site:

  1. Upload the video to Stream
  2. Get the video ID
  3. Install the Cloudflare Stream React component:
$ npm install @cloudflare/stream-react
  1. Import the component into the blog post and use it (MDX supports JSX components):
---
title: Workers GraphQL Server v2
description: Showing off some updates to my workers-graphql-server project.
pubDate: 'Nov 01 2024'
tags:
  - cloudflare
  - webdev
---
import StreamVideo from "@/components/StreamVideo";

I've been working on updating the [`workers-graphql-server`](https://github.com/kristianfreeman/workers-graphql-server) project.

Five years ago, it was one of the first things I built after joining [Cloudflare](https://cloudflare.com). Since then, the way that we write Workers has changed a lot.

The [new v2 release](https://github.com/cloudflare/workers-graphql-server/releases/tag/2.0.0-alpha) is a complete rewrite. Some big changes:

1. Supports Wrangler v2 (better local dev, deployment, bindings, etc.)
2. Written as a module Worker (modern syntax for writing Workers)
3. Built on top of [Hono](https://honojs.dev/)
4. Examples for integrating external data sources, and service bindings
5. Updated implementation of the [Workers KV](https://developers.cloudflare.com/kv/) cache

I made a video showing how the new version works. Check it out!

[See the video on Cloudflare Stream](https://customer-rg94wz3mmkgblc3e.cloudflarestream.com/8e7113ab230ea2e352270dba1101b5f3/iframe)

Note the usage of the client:only attribute. This is a feature of Astro that allows you to only hydrate the component on the client-side. This is important because the Stream component will try to create a new <video> tag on the client, so if it tries to render on the server, the video content won’t show up.

Captions

A recent feature that Stream added is the ability to add captions. The cool part is that it can use Workers AI to generate the captions automatically.

The Stream dashboard provides a UI for generating captions.

Selecting this option will generate captions for the video in just a few minutes. Once the captions are generated, you can use them by providing a defaultTextTrack prop to the <Stream> component that matches the language of the captions:

import { Stream } from "@cloudflare/stream-react";

<Stream 
  client:only="react" 
  controls 
  defaultTextTrack="en"
  src="$videoId" 
/>

If you’re like me, you may be a little disappointed with the caption quality. Since Stream uses AI, it’s not perfect. Instead, generating your own caption locally, so you have the ability to easily tweak it, might be a good option.

To do this, I used MacWhisper to use a higher-quality Whisper AI model to generate a first pass of the captions. Then I opened the .vtt file in Vim and fixed some of the references to technical terms. The captions still aren’t perfect, but they’re decent and legible. After creating those captions, I re-uploaded them and replaced the generated captions inside of Stream.

My component

I created a component that I can use in my blog posts to make it easier to include a video. It allows me to style and space the video component to fit the design of my site. Here’s the code for components/StreamVideo.tsx:

import { Stream } from "@cloudflare/stream-react";

export default function StreamVideo({ src }: { src: string }) {
  return (
    <div className="mt-4">
      <Stream
        controls
        defaultTextTrack="en"
        src={src}
      />
    </div>
  );
}

Monday, November 4, 2024

Get started with 𝕏 AI and Grok

Grok is 𝕏‘s new foundational AI model. And as of today, you can get $25 in credits every month to build with it.

In this post, I’ll show you how to build a simple app with Grok and Vercel AI SDK. We’ll build a full-stack application using Astro, and deploy it to Cloudflare Pages. If you’re interested in seeing the full source code for what we build, check it out on GitHub.

Create a project

First, we’ll create a new project using the npm create cloudflare CLI - selecting the “Astro” framework:

$ npm create cloudflare@latest -- grok-starter --framework=astro
$ cd grok-starter

This will create a new project in the grok-starter directory.

Add Vercel AI SDK

Next, we’ll add the Vercel AI SDK and the 𝕏 AI Provider to our project.

$ npm install ai @ai-sdk/openai

Let’s define an API endpoint that we can use to call the AI model:

import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

export async function POST({ locals, request }: any) {
  const apiKey = locals.runtime.env.XAI_API_KEY;

  if (!apiKey) return new Response('No API key provided', { status: 400 });

  const xai = createOpenAI({
    name: 'xai',
    baseURL: 'https://api.x.ai/v1',
    apiKey
  });

  try {
    const body = await request.json();

    if (!body || !body.prompt) {
      return new Response('No prompt provided', { status: 400 });
    }

    const prompt = body.prompt;

    const { text } = await generateText({
      model: xai('grok-beta'),
      prompt,
    });

    return new Response(text);
  } catch (error) {
    console.error(error);
    return new Response('Error generating text', { status: 500 });
  }
}

Note: this function is typed as any to simplify the tutorial. In the full code, the function is properly typed.

Get an API key

To use the Grok model, you’ll need an API key. You can get one by signing up for an account at console.x.ai. After you’ve confirmed your email address, you can generate an API key:

Get an API key

Add the API key to .dev.vars in the root of your project:

XAI_API_KEY="xai-$key"

Run locally

Now we can run the app locally:

$ npm run dev

To generate our first text from Grok, we can send a POST request to the /api/generate endpoint1.

$ xh POST http://localhost:4321/api/generate prompt="What is the capital of France?"
"The capital of France is Paris."

Create a frontend interface

Now we can create a simple frontend interface to interact with our API. In src/pages/index.astro, we’ll replace the content generated by Astro with basic styling, and add a form with an input and a button:

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
    <meta name="viewport" content="width=device-width" />
    <meta name="generator" content={Astro.generator} />
    <title>Grok Starter</title>

    <style>
      body {
        font-family: system-ui, sans-serif;
        padding: 2rem;
      }

      form {
        display: flex;
        column-gap: 1rem;
      }

      input {
        width: 100%;
        padding: 0.5rem;
        font-size: 1rem;
        border-radius: 0.5rem;
        border: 1px solid #ccc;
      }

      button {
        padding: 0.5rem;
        font-size: 1rem;
        border-radius: 0.5rem;
        border: 1px solid #ccc;
        background-color: #ccc;
        cursor: pointer;
      }

      #grok-response {
        margin-top: 1rem;
        white-space: pre-wrap;
        word-wrap: break-word;
      }
    </style>
  </head>

  <body>
    <h1>Grok Starter</h1>
    <form id="form">
      <input type="text" id="input" placeholder="Enter your text here..." />
      <button type="submit">Generate</button>
    </form>
    <div id="grok-response"></div>
  </body>
</html>

This form will render, but submitting it won’t do anything. Let’s write a basic function that will take the text from the input and send it to our API endpoint:

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
    <meta name="viewport" content="width=device-width" />
    <meta name="generator" content={Astro.generator} />
    <title>Grok Starter</title>

    <style>
      body {
        font-family: system-ui, sans-serif;
        padding: 2rem;
      }

      form {
        display: flex;
        column-gap: 1rem;
      }

      input {
        width: 100%;
        padding: 0.5rem;
        font-size: 1rem;
        border-radius: 0.5rem;
        border: 1px solid #ccc;
      }

      button {
        padding: 0.5rem;
        font-size: 1rem;
        border-radius: 0.5rem;
        border: 1px solid #ccc;
        background-color: #ccc;
        cursor: pointer;
      }

      #grok-response {
        margin-top: 1rem;
        white-space: pre-wrap;
        word-wrap: break-word;
      }
    </style>
  </head>

  <body>
    <h1>Grok Starter</h1>
    <form id="form">
      <input type="text" id="input" placeholder="Enter your text here..." />
      <button type="submit">Generate</button>
    </form>
    <div id="grok-response"></div>
  </body>

  <script>
    const button = document
      .getElementById("form")!
      .querySelector("button[type='submit']") as HTMLButtonElement;
    const input = document.getElementById("input")! as HTMLInputElement;
    const grokResponse = document.getElementById(
      "grok-response",
    )! as HTMLDivElement;

    const generateText = async (text: string) => {
      input.disabled = true;
      button.disabled = true;
      button.innerHTML = "Generating...";

      const response = await fetch("/api/generate", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          prompt: text,
        }),
      });

      const output = await response.text();
      grokResponse.innerHTML = output;

      input.disabled = false;
      input.value = "";

      button.disabled = false;
      button.innerHTML = "Generate";
    };

    const form = document.getElementById("form")!;
    form.addEventListener("submit", async (event) => {
      event.preventDefault();
      await generateText(input.value);
    });
  </script>
</html>

The complete implementation looks like this:

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
    <meta name="viewport" content="width=device-width" />
    <meta name="generator" content={Astro.generator} />
    <title>Grok Starter</title>

    <style>
      body {
        font-family: system-ui, sans-serif;
        padding: 2rem;
      }

      form {
        display: flex;
        column-gap: 1rem;
      }

      input {
        width: 100%;
        padding: 0.5rem;
        font-size: 1rem;
        border-radius: 0.5rem;
        border: 1px solid #ccc;
      }

      button {
        padding: 0.5rem;
        font-size: 1rem;
        border-radius: 0.5rem;
        border: 1px solid #ccc;
        background-color: #ccc;
        cursor: pointer;
      }

      #grok-response {
        margin-top: 1rem;
        white-space: pre-wrap;
        word-wrap: break-word;
      }
    </style>
  </head>

  <body>
    <h1>Grok Starter</h1>
    <form id="form">
      <input type="text" id="input" placeholder="Enter your text here..." />
      <button type="submit">Generate</button>
    </form>
    <div id="grok-response"></div>
  </body>

  <script>
    const button = document
      .getElementById("form")!
      .querySelector("button[type='submit']") as HTMLButtonElement;
    const input = document.getElementById("input")! as HTMLInputElement;
    const grokResponse = document.getElementById(
      "grok-response",
    )! as HTMLDivElement;

    const generateText = async (text: string) => {
      input.disabled = true;
      button.disabled = true;
      button.innerHTML = "Generating...";

      const response = await fetch("/api/generate", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          prompt: text,
        }),
      });

      const output = await response.text();
      grokResponse.innerHTML = output;

      input.disabled = false;
      input.value = "";

      button.disabled = false;
      button.innerHTML = "Generate";
    };

    const form = document.getElementById("form")!;
    form.addEventListener("submit", async (event) => {
      event.preventDefault();
      await generateText(input.value);
    });
  </script>
</html>

The final UI can accept prompts and generate responses:

Example of AI endpoint

Deployment

Now that the UI and endpoint work correctly, we can deploy the application. To do this, we’ll use Cloudflare Pages. This is a great way to deploy a static site, and it’s free for small projects.

If you haven’t created an account already, you can do so here. Once you’ve created your account, you’ll be able to create a new project.

Once you’ve created an account, you can authenticate with the CLI by running:

$ npx wrangler login

Finally, we can deploy our application by running:

$ npm run pages:deploy

This will build the site, and deploy it to Cloudflare Pages.

Before we can use it in production, we need to set up the 𝕏 API key that we generated earlier. To do this, run the following command:

$ npx wrangler pages secret put XAI_API_KEY

After setting the secret, you should deploy one more time, to ensure that the secret is available to your application:

$ npm run pages:deploy

After doing this, you can visit the site and test it out!

Securing with AI Gateway

We have one more trick up our sleeve. We can use Cloudflare AI Gateway to help protect our application from abuse. Since we’re essentially deploying an unprotected API endpoint that would allow anyone to use 𝕏 AI/Grok via our application, it would be easy for a malicious actor to use up all our credits!

AI Gateway allows you to protect your API endpoints from abuse. It works by rate-limiting requests, caching responses, and adding useful logging/analytics to your AI endpoints.

Here’s how to integrate it into your application:

  1. Enable AI Gateway by visiting the Dashboard and enabling it.
  2. Create a new AI Gateway by clicking the “Create Gateway” button.
  3. Select the “API” button, and find the “Grok” endpoint option
  4. Copy this value.

This AI Gateway endpoint is what we will proxy our AI requests through.

In the src/pages/api/openai.ts file, we’ll update the URL to use this endpoint:

import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

export async function POST({ locals, request }: { locals: App.Locals; request: Request }) {
  const apiKey = locals.runtime.env.XAI_API_KEY;

  if (!apiKey) {
    return new Response('No API key provided', { status: 400 });
  }

  const baseURL = 'https://gateway.ai.cloudflare.com/v1/$accountId/grok-starter/grok';

  const xai = createOpenAI({
    name: 'xai',
    baseURL: 'https://api.x.ai/v1',
    baseURL
    apiKey
  });

  try {
    const body = await request.json();

    if (!body || !body.prompt) {
      return new Response('No prompt provided', { status: 400 });
    }

    const prompt = body.prompt;

    const { text } = await generateText({
      model: xai('grok-beta'),
      prompt,
    });

    return new Response(text);
  } catch (error) {
    console.error(error);
    return new Response('Error generating text', { status: 500 });
  }
}

Redeploy your application, and try issuing a request from the UI to your AI endpoint. You should begin seeing logs generated in the AI Gateway console.

AI Gateway log

Finally, we can turn on a few settings to make AI Gateway securely protect our API endpoint:

  1. “Cache Responses”. This will cache any response from Grok that matches a previous prompt.
  2. “Rate Limit Requests”. This will limit the number of requests that can be made to the API endpoint from any given IP address. You can tweak this to be, for instance, 10 requests per minute.
AI Gateway settings

Conclusion

I’m impressed with 𝕏 AI and the Grok model! It’s pretty smart, and it’s easy to integrate into applications. The $25 free monthly credit they’ve announced is awesome, and I’m excited to keep building with it. In the blog post, they mention support for tool calling and system prompts. This will probably get integrated into Vercel SDK AI soon, so it will be another great model to have in the toolbelt while building AI apps.

If you’re interested in seeing the full source code for this project, check it out on GitHub!

Footnotes

  1. This example uses xh, a great cURL alternative, for simplicity. It’s a basic POST request, so adapt to cURL as needed.

Tuesday, October 29, 2024

Workers Entrypoints

Cloudflare Workers traditionally has allowed definition of a single entrypoint for a Workers application. The initial version of Workers did this via an event listener:

const handleRequest = request => {
  return new Response('Hello world!')
}

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

In the last few years, most developers have migrated to the “module” syntax for Workers, as we’ve standardized on it in our tooling:

export default {
  async fetch(request, env, ctx) {
    return new Response('Hello world!')
  }
}

This means that your Workers code, for instance, defined in index.js, has a default export. That export is a module with a collection of functions representing different events - fetch, to handle HTTP requests, or scheduled, to trigger on a cron trigger.

This module is the primary entrypoint for a Workers application. But it turns out that you can define other entrypoints to serve other purposes.

Workflows

Yesterday, I wrote about Cloudflare Workflows:

Workflows are a new feature in Cloudflare’s developer platform. You can use workflows to safely execute a series of steps as defined by code. Inside the workflows-starter directory, src/index.ts defines two top-level exports: the workflow entrypoint, as well as a default module that handles HTTP requests.

To define a workflow, you can export a class that extends WorkflowEntrypoint. The module that is exported by default still allows fetch, scheduled, and other events to access the Worker:

export class MyWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    await step.sleep('wait on something', '1 minute');

    await step.do('test step', async () => {
      return 'OK, done!';
    });
  }
}

// Module entrypoint
export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    return new Response('Hello world!');
  }
};

This is an interesting model! It allows you to define multiple entry points for a single Workers application.

Workers RPC

In an upcoming release of workers-graphql-server, I’ve added preliminary support for Workers RPC. Workers RPC allows you to access Workers applications from one another. Let’s look at how that works.

You can arbitrarily export any number of classes from your Workers application, as long as they extend the WorkersEntrypoint class. For example, a MoviesService that connects to a D1 database1. That’s right - more entrypoints!

import { WorkerEntrypoint } from "cloudflare:workers";

export class MoviesService extends WorkerEntrypoint {
  async getMovies() {
    const query = "select * from movies";
    const { results: movies } = await this.env.DB
      .prepare(query)
      .all();
    return movies;
  };
}

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    if (url.pathname === "/movies") {
      const { results: movies } = await this.env.DB
        .prepare(query)
        .all();
      return new Response(JSON.stringify(movies));
    }
  }
}

In any project, you can then import this class via service bindings:

name = "workers-graphql-server"

[[services]]
binding = "MOVIES"
entrypoint = "MoviesService"
service = "d1-movie-example"

That binding MOVIES connects to the MoviesService class inside of d1-movie-example. You can use it like any other function, but it is a genuine different application, imported somewhat magically:

function getBestMovies(env) {
  const movies = await env.MOVIES.getMovies()
  return movies.sort((a, b) => b.rating - a.rating)
}

This is super legit. It’s microservice architecture without needing to care about deployments, scaling, or any of that other stuff. Really impressive!

Conclusion

As a long-time user2, I’m quite happy to see how entrypoints have evolved into a way to access the same application and core logic through different strategies - not just HTTP requests, but workflows, RPC calls, and (I imagine) even more ways in the future.

Footnotes

  1. This example is derived from my d1-movie-example repo.

  2. Sure, I work at Cloudflare - but I don’t have any particular insight into how entrypoints will evolve on the Workers platform, or how they’ll be used in the future.

Monday, October 28, 2024

An introduction to Cloudflare Workflows

Workflows are a new feature in Cloudflare’s developer platform. You can use workflows to safely execute a series of steps as defined by code.

Let’s start by creating a new workflow.

$ npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter"

Inside the workflows-starter directory, src/index.ts defines two top-level exports: the workflow entrypoint, as well as a default module that handles HTTP requests:

import { WorkflowEntrypoint, WorkflowStep, WorkflowEvent } from 'cloudflare:workers';

type Env = {
  // Add your bindings here, e.g. Workers KV, D1, Workers AI, etc.
  MY_WORKFLOW: Workflow;
};

// User-defined params passed to your workflow
type Params = {
  email: string;
  metadata: Record<string, string>;
};

// Workflow entrypoint
export class MyWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    // Can access bindings on `this.env`
    // Can access params on `event.payload`

    const files = await step.do('my first step', async () => {
      // Fetch a list of files from $SOME_SERVICE
      return {
        inputParams: event,
        files: [
          'doc_7392_rev3.pdf',
          'report_x29_final.pdf',
          'memo_2024_05_12.pdf',
          'file_089_update.pdf',
          'proj_alpha_v2.pdf',
          'data_analysis_q2.pdf',
          'notes_meeting_52.pdf',
          'summary_fy24_draft.pdf',
        ],
      };
    });

    const apiResponse = await step.do('some other step', async () => {
      let resp = await fetch('https://api.cloudflare.com/client/v4/ips');
      return await resp.json<any>();
    });

    await step.sleep('wait on something', '1 minute');

    await step.do(
      'make a call to write that could maybe, just might, fail',
      // Define a retry strategy
      {
        retries: {
          limit: 5,
          delay: '5 second',
          backoff: 'exponential',
        },
        timeout: '15 minutes',
      },
      async () => {
        // Do stuff here, with access to the state from our previous steps
        if (Math.random() > 0.5) {
          throw new Error('API call to $STORAGE_SYSTEM failed');
        }
      },
    );
  }
}

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    let url = new URL(req.url);

    if (url.pathname.startsWith('/favicon')) {
      return Response.json({}, { status: 404 });
    }

    // Get the status of an existing instance, if provided
    let id = url.searchParams.get('instanceId');
    if (id) {
      let instance = await env.MY_WORKFLOW.get(id);
      return Response.json({
        status: await instance.status(),
      });
    }

    // Spawn a new instance and return the ID and status
    let instance = await env.MY_WORKFLOW.create();
    return Response.json({
      id: instance.id,
      details: await instance.status(),
    });
  },
};

What is a workflow?

A workflow is a class that extends WorkflowEntrypoint. It has access to env, which contains the bindings for the Workers application. It can also accept (typed) parameters used to instantiate the workflow.

A workflow is comprised of steps. You can call step.do to execute a step:

await step.do('do something', async () => {
  return 'OK, done!';
});

Steps should be awaited, as they are asynchronous. You can return a value from the step, and capture it as a variable:

const result = await step.do('do something', async () => {
  return 'OK, done!';
});

You can use step.sleep to pause the workflow for a period of time:

await step.sleep('wait on something', '1 minute');

The second parameter is a duration-style string, such as '1 minute', '5 seconds', or '1 year'.

All steps are retried by default - see the retry steps section in the docs for more details. You can override the default behavior by passing a retries option to step.do:

await step.do(
  'make a call to write that could maybe, just might, fail',
  // Define a retry strategy
  {
    retries: {
      limit: 5,
      delay: '5 second',
      backoff: 'exponential',
    },
    timeout: '15 minutes',
  },
  async () => {
    // Do stuff here, with access to the state from our previous steps
    if (Math.random() > 0.5) {
      throw new Error('API call to $STORAGE_SYSTEM failed');
    }
  },
);

Building a custom workflow

So far, we’ve looked at the default code used in Cloudflare’s workflow template. Now, we’ll build our own workflow.

Imagine that we want to build a workflow that fetches analytics for a SaaS product we’re building. It will then take those analytics and report them to a private Slack channel. We can break this down into three steps:

  1. Fetch analytics from our SaaS product
  2. Transform/format those analytics
  3. Report those analytics to a private Slack channel

That workflow can be defined like this:

type Env = {
  REPORT_ANALYTICS_WORKFLOW: Workflow;
  ANALYTICS_ENDPOINT: string;
};

type Params = {
  channelId: string;
};

export class ReportAnalyticsWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const analytics = await step.do('fetch analytics', async () => {
      const resp = await fetch(env.ANALYTICS_ENDPOINT);
      return await resp.json<any>();
    });

    const formatted = await step.do('format analytics', async () => {
      return formatAnalytics(analytics);
    });

    await step.do('report analytics', async () => {
      const resp = await fetch('https://slack.com/api/chat.postMessage', {
        method: 'POST',
        body: JSON.stringify({
          channel: event.payload.channelId,
          text: JSON.stringify(formatted),
        }),
      });

      if (!resp.ok) {
        throw new Error('Failed to report analytics');
      }
    });
  }
}

Although it’s deceptively simple, this workflow as defined is quite powerful. If any step fails, it will retry the workflow using exponential backoff, starting from that step. There’s minimal try/catch style code in this example, yet it still handles errors gracefully.

Calling workflows

Now that we’ve built a workflow, let’s explore how to execute it. In src/index.ts, we can define and export a default module that handles HTTP requests. Inside of that handler function, we can call the workflow by using the create function defined on our workflow:

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const channelId = req.url.searchParams.get('channelId') || 'C123456789';
    await env.REPORT_ANALYTICS_WORKFLOW.create({ channelId });
  }
}

Where does env.REPORT_ANALYTICS_WORKFLOW come from? It’s defined in wrangler.toml:

#:schema node_modules/wrangler/config-schema.json
name = "workflows-starter"
main = "src/index.ts"
compatibility_date = "2024-10-22"

[observability]
enabled = true
head_sampling_rate = 1 # optional. default = 1.

[[workflows]]
name = "report-analytics"
binding = "REPORT_ANALYTICS_WORKFLOW"
class_name = "ReportAnalyticsWorkflow"

By defining the workflow in wrangler.toml, we can access it from our code.

It’s useful to call workflows manually, but we can also call them using a scheduled trigger. For an analytics workflow, we may want to run it every day at a certain time. We can do this by defining a triggers block in wrangler.toml:

#:schema node_modules/wrangler/config-schema.json
name = "workflows-starter"
main = "src/index.ts"
compatibility_date = "2024-10-22"

[observability]
enabled = true
head_sampling_rate = 1 # optional. default = 1.

[[workflows]]
name = "report-analytics"
binding = "REPORT_ANALYTICS_WORKFLOW"
class_name = "ReportAnalyticsWorkflow"

[triggers]
# Run the workflow every day at 12:00 AM
crons = ["0 * * * *"]

With the cron trigger enabled, we can add a new function scheduled to our src/index.ts file:

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const channelId = req.url.searchParams.get('channelId') || 'C123456789';
    await env.REPORT_ANALYTICS_WORKFLOW.create({ channelId });
  }

  async scheduled(event, env, ctx) {
    ctx.waitUntil(
      env.REPORT_ANALYTICS_WORKFLOW.create({
        channelId: 'C123456789',
      }),
    );
  },
};

With the scheduled function defined, the analytics workflow will run every day at 12:00 AM.

Conclusion

I’m quite enjoying playing with Workflows. They’re a great way to reliably execute multi-step code without having to worry about retries, timeouts, or error handling. You can check out the documentation for Workflows to learn more. I also wrote about my indexer project a few days ago, which is built on top of Cloudflare Workflows and is fully open-source.

Sunday, October 27, 2024

Announcing the Astro Cloudflare Redirects integration

Cloudflare, as part of the Cloudflare Pages platform, has a file specification for redirects, usually defined via a _redirects file. This feature allows you to specify a list of URLs and their destinations, and Cloudflare will automatically redirect traffic to those URLs to the specified destinations.

It looks like this:

/home301 / 301
/home302 / 302
/querystrings /?query=string 301
/twitch https://twitch.tv
/trailing /trailing/ 301
/notrailing/ /nottrailing 301
/page/ /page2/#fragment 301
/blog/* https://blog.my.domain/:splat
/products/:code/:name /products?code=:code&name=:name

This is a helpful way to keep broken URLs from piling up on your site, and I wanted to use it inside of my Astro-based site.

Luckily, the vite-plugin-cloudflare-redirect was created by a member of Astro’s community. I used that Vite plugin and wrapped in an Astro integration package, meaning it’s incredibly easy to install:

$ npx astro add astro-cloudflare-redirects

This will do a guided install to add the integration to your Astro project.

npx astro add astro-cloudflare-redirects
⚠ astro-cloudflare-redirects is not an official Astro package.
✔ Continue? … yes
✔ Resolving with third party packages...

  Astro will run the following command:
  If you skip this step, you can always run it yourself later

 ╭────────────────────────────────────────────────╮
 │ npm install astro-cloudflare-redirects@^0.0.1  │
 ╰────────────────────────────────────────────────╯

✔ Continue? … yes
✔ Installing dependencies...

  Astro will make the following changes to your config file:

 ╭ astro.config.mjs ─────╮
 │ // ...                │
 ╰───────────────────────╯

✔ Continue? … yes
  
   success  Added the following integration to your project:
  - astro-cloudflare-redirects

You can also install it manually:

$ npm install astro-cloudflare-redirects

Once installed, you can add the integration to your Astro config:

import { defineConfig } from 'astro/config';
import cloudflareRedirects from 'astro-cloudflare-redirects';

export default defineConfig({
  integrations: [cloudflareRedirects()],
});

Usage

You can use it by generating a file in public/_redirects with the same format as the Cloudflare redirects file. For instance, if you wanted to redirect /old-url to /new-url, you could create a _redirects file like this:

/old-url /new-url

You can also specify a custom file location, if you’d like:

import { defineConfig } from 'astro/config';
import cloudflareRedirects from 'astro-cloudflare-redirects';

export default defineConfig({
  integrations: [cloudflareRedirects({
    redirectsFile: './src/_redirects',
  })],
});

Why build this?

I like Cloudflare’s _redirects file format, but by pulling it into Astro via a Vite plugin, you can use it in any environment. Regardless of how you’re deploying your Astro application, you can use this plugin for easy redirect handling.