I’m writing this because what follows should be obvious to any seasoned developer:

1. Should time fields be modeled as a dedicated Timestamp type or as a primitive long?
2. Should API time values be exposed as raw epoch integers or as RFC3339‑formatted strings?

The Answers

Timestamps should be modeled as a dedicated type, not a long.

Using a primitive like long or int64 to represent a timestamp might seem easy and performant—but in most cases, it’s a bad idea. It’s error-prone, opaque, and worst of all, that performance “gain” is rarely something your application actually needs.

A raw long gives no context. Is it seconds? Milliseconds? Nanoseconds? Since epoch? From some custom epoch? Why should you– or anyone else–have to keep asking these tedious questions? It’s a timestamp—use a type that makes that explicit!

There's also no type safety. It’s easy for developers to accidentally assign unrelated numeric values, or confuse durations with points in time.

And then there’s functionality: you can’t easily perform date/time operations on a long. A proper time type usually comes with everything you need—comparison, formatting, arithmetic, etc.

However, some still might argue, “I don’t need all that—I just want to get the difference in seconds between two time points.” Fair enough. But doesn’t a timestamp type give you that too, plus a lot more?

Sure, there are exceptions. For instance, if you're building a database or a low-level storage engine, modeling timestamps with long might make sense for efficiency. But outside of those rare cases, reach for a real timestamp type. Your future self—and your teammates—will thank you.

RFC3339 for time values instead of epoch in APIs.

I’m not talking about binary protocols or compact serialization formats. Don’t argue on that. I’m talking about APIs—typically text-based formats like JSON, XML, etc.

Timestamps in APIs should be human-readable and unambiguous. That’s what RFC3339 gives us.

Epochs are not readable—1681724411 means nothing at a glance. They're also ambiguous: is that seconds? Milliseconds? Something else?

RFC3339 is a standard. It's a widely adopted format (2006-01-02T15:04:05Z), easily parsed by both humans and machines, and supported almost in every language and API tookit.

If you're concerned about performance or payload size, optimize elsewhere. Don’t sacrifice clarity and interoperability.

Some might argue, “We don’t need to look at API responses directly. Who cares about readability? Epoch is more performant. Why optimize for clarity?”

Frankly, I don’t even want to engage with that line of reasoning. It misses the point entirely. APIs are not just for machines—they’re for developers too. Readability, debuggability, and long-term maintainability do matter.

Best Practices for Handling Timestamps in Your Program – The input → process → output Pattern

When dealing with a timestamp, it should be divided into 3 phases.

• Input phase: Parse raw timestamp values (e.g. from API requests or file inputs) into proper time objects—like java.time.Instant in Java or time.Time in Go. Additionally, time objects should be timezone-aware. Time types without timezone information should be forbidden—they're a breeding ground for bugs and confusion.
• Processing phase: Use these time objects throughout your program logic. This gives you type safety, clarity, and access to useful date/time operations.
• Output phase: When producing output (e.g. an API response), format the timestamp as needed. For text-based APIs, RFC3339 is the go-to choice for readability and interoperability.

Actually, this practice also applies to any other data types that involve conversions—for instance, URLs, string encodings, and so on.

I’ve lost count of how many times I’ve seen developers mess up URLs by treating them like plain strings. Here’s what I mean:

base_url = "https://example.com/search"
query = "apple"
url = base_url + "?q=" + query + "&sort=asc"

I’m not saying you should never build a URL by concatenating strings. In some simple, controlled cases, it’s perfectly fine. But it should be done with caution—and only when the structure is dead simple and the parameters are guaranteed to be safe. For anything dynamic or user-controlled, use proper URL utilities.

Following the pattern above, you should parse the raw URL string into a proper URL object as part of the input phase. From there on, treat it like what it is: a structured object. Use the provided methods to inspect or modify it—don’t fall back to string hacks.

Why this "input → process → output" methodology matters?

This approach works for any kind of data that needs conversion, like timestamps (Instant), encoded strings, file paths, and more.

By centralizing conversions into the input and output phases, you get these advantages:

• Cleaner code: your main logic deals with proper types.
• Fewer bugs: invalid data gets caught early, and you avoid bugs by using the proper functions (typically provided with the types).
• No duplicate work: you parse once, use many times in the entire program.
• Better separation: input/output is handled at the edge, business logic stays focused.

Closing Thoughts

These debates aren't about preference—they're about clarity, safety, and long-term maintainability. We should aim for conventions that prevent bugs and help other developers (and future you) understand what’s going on at a glance.

Let’s stop reinventing the wheel and adopt what the industry has already learned through decades of hard lessons.

This post is introducing a way to run Go snippets on websites. Which leveraged the official Go Playground service at https://go.dev/play/.

Demo

Let’s take a quick look at the two demos below. You can click the Run button to see the output of the corresponding program. Click Share button to open the code in the official Go Playground to further edit/test the code on your hand.

Demo 1. Hello World

package main

func main() {
	println("hello world")
}

Demo 2. Import third-party package from GitHub

package main

import (
	"fmt"
	"net/http"
	"net/http/httptest"

	"github.com/ggicci/httpin" // 3rd-party packages: will be auto-downloaded
	"github.com/justinas/alice"
)

type ListUsersInput struct {
	Page    int64  `in:"query=page;default=1"`
	PerPage int64  `in:"query=per_page;default=20"`
	Token   string `in:"header=x-access-token;required"`
}

func ListUsers(rw http.ResponseWriter, r *http.Request) {
	input := r.Context().Value(httpin.Input).(*ListUsersInput)
	fmt.Printf("input: %#v\n", input)
	rw.WriteHeader(204)
}

func main() {
	r, _ := http.NewRequest("GET", "/?page=3", nil)
	r.Header.Set("X-Access-Token", "secret...")
	handler := alice.New(
		httpin.NewInput(ListUsersInput{}),
	).ThenFunc(ListUsers)
	rw := httptest.NewRecorder()
	handler.ServeHTTP(rw, r)
}

How it works?

Since we were not able to make AJAX requests directly to go.dev because of CORS issue. I setted up a reverse proxy goplay.ggicci.me, which works as a downstream of go.dev. It is responsible to make requests on behalf of the clients to go.dev and return the response back to the clients.

I also implemented a simple JS SDK @ggicci/goplay to easily interact with the go playground service. Use it on your page to connect to your own reverse proxy and run go snippets.

Set up a reverse proxy to go.dev

with Caddy:

goplay.ggicci.me {
    header {
      # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers#cors
      access-control-allow-origin      "https://ggicci.me"
      access-control-allow-headers     "content-type"
      access-control-allow-methods     "OPTIONS,HEAD,GET,POST"
    }

    # only proxy the playground API of go.dev/play
    route /_/* {
      reverse_proxy https://go.dev {
        header_up Host go.dev
      }
    }

    respond 404
}

Use @ggicci/goplay

The dead simple SDK just implemented a class GoPlayProxy with 3 primary APIs:

class GoPlayProxy {
  // CTOR: specify a proxy url to connect
  constructor(public proxyUrl: string = '/goplay') {}
  // compile source code and get JSON response
  public async compile(sourceCode: string, goVersion?: CompilerVersion): Promise<CompilerResponse>
  // same as `compile`, but renders the JSON response as an `HTMLElement` into the specified container
  public async renderCompile(container: HTMLElement, sourceCode: string, goVersion?: CompilerVersion): Promise<void>
  // get the share URL of the source code
  public async share(sourceCode: string, goVersion?: CompilerVersion): Promise<string>
}

Visit https://github.com/ggicci/goplay for more details.

Hugo Shortcode goplay

If your are using hugo to generate your website. This section should be helpful to you.

By using the goplay shortcode, you can easily turn your Go code block into a Go playground. Take the following example, in your markdown file, just wrap the code block between {{% goplay %}} and {{% /goplay %}}.

## Sample Code

{{% goplay %}}

```go
package main

func main() {
  println("hello world")
}
```

{{% /goplay %}}

The latest code of goplay - hugo shortcode used by this site (former version) can be found here. Save it under your hugo directory correspondingly. Typically it is ${YourHugoSiteRoot}/layouts/shortcodes/goplay.html.

Use goplay in your Ghost Blog

My site is now using Ghost. Thanks to its Code Injection function (go to your blog settings and find it there), we can add our custom JavaScript code to drive goplay on our website.

The script I'm using on this website can be found here:

https://gist.github.com/ggicci/c85b4665e959a10fdbe0c97b33f44eb0

Who is using @ggicci/goplay?

• httpin Documentation
• add yours here by commenting below :)

Before reading this post, I recommend you to read A Better ID Generator For PostgreSQL written by Rob Conery. To recap briefly, their post had discussed:

1. The problem you will face in the rapid grow of your systems if you used GUID as the primary key.
2. How Twitter generating auto-incrementing keys with Snowflake.
3. Create a functional Snowflake equivalent for PostgresSQL.

And the third point above will be the focus of discussion in this post.

The Algorithm

create schema shard_1;
create sequence shard_1.global_id_sequence;

CREATE OR REPLACE FUNCTION shard_1.id_generator(OUT result bigint) AS $$
DECLARE
    our_epoch bigint := 1314220021721;
    seq_id bigint;
    now_millis bigint;
    -- the id of this DB shard, must be set for each
    -- schema shard you have - you could pass this as a parameter too
    shard_id int := 1;
BEGIN
    SELECT nextval('shard_1.global_id_sequence') % 1024 INTO seq_id;

    SELECT FLOOR(EXTRACT(EPOCH FROM clock_timestamp()) * 1000) INTO now_millis;
    result := (now_millis - our_epoch) << 23;
    result := result | (shard_id << 10);
    result := result | (seq_id);
END;
$$ LANGUAGE PLPGSQL;

select shard_1.id_generator();

This algorithm will generate an integer in 64 bits consisting of:

   +----------------+---------------+-------------+
   | Timestamp (41) | Shard ID (13) | Seq ID (10) |
   +----------------+---------------+-------------+
   |<-------------- Unique ID (64) -------------->|

Here we can expand Seq ID part to gain the ablility of supporting more operations per ms. But as a trade off, the Shard ID will be shrunk.

For example, if we used 10 bits for Shard ID and 13 bits for Seq ID, we will allow 8192 write operations per ms. Which should be 8M ops/s. However, as a reminder, this 8M ops/s is evenly distributed over 1s. Which means when your system had a writing spike with over 8192 writes in the same millisecond, it can fail.

Safe Number Problem in JSON

The above algorithm generates a 64-bit integer. In practice, it's not "JSON comaptible". Because in JSON world, number is a "double-like" type. And there's in fact a limitation at JavaScript/ECMAScript level of precision to 53-bit for integers. The maximum safe integer in JavaScript is 2^53 - 1. You can check it by running the following JS code in your browser:

Number.MAX_SAFE_INTEGER; // 9007199254740991
Number.isSafeInteger(9007199254740992); // false

So, How we work with 64-bit integers in our API and JavaScript?

I have two solutions here:

1. Encode a 64-bit integer to text by using binary-to-text encoding methods, e.g. Base58, etc.
2. Generate "safe" integers, which only occupies the lower order 53-bits.

Let's talk about the second solution by tweaking the algorithm above to generate "safe" unique IDs.

Generate "JSON-Safe" 53-bit Unique IDs for PostgresSQL

Here are some points you need to consider:

1. Use s or ms for the timestamp?
   • Quick Answer: ms for tighter distributions of write operations, and s is more flexible to handle write spikes.
2. Set epoch offset or not?
   • Quick Answer: It's a must if using ms for timestamp. Otherwise optional. Better not.
3. How many shards should I have?
   • Quick Answer: 16 is sufficient for small and medium applications.
4. TPS considerations?
   • Quick Answer: think of how many write operations your application needs, and the performance (TPS) of your database/shard.

Here's the code for the 7th option:

CREATE SEQUENCE public.global_id_sequence;
CREATE OR REPLACE FUNCTION public.id_generator(OUT result bigint) AS $$
DECLARE
    now_seconds bigint;
    shard_id int := 1;
    seq_id bigint;
BEGIN
    SELECT nextval('public.global_id_sequence') % 65536 INTO seq_id;

    SELECT FLOOR(EXTRACT(EPOCH FROM CLOCK_TIMESTAMP())) INTO now_seconds;
    result := now_seconds << 21; -- Shard(5) + Seq(16)
    result := result | (shard_id << 16); -- Seq(16)
    result := result | (seq_id);
END;
$$ LANGUAGE PLPGSQL;

SELECT public.id_generator();

Many people use net/http package directly in Go to deal with HTTP requests, including reading URL parameters, HTTP headers, and the request body. It's straightforward and efficient, though. We can still get bored writing so much tedious code for just reading and parsing the URL params. Especially when we were maintaining a service with hundres of APIs.

Let's see a piece of code for dealing with HTTP requests by using net/http package:

// GET /v1/users?page=1&per_page=20&is_member=true
func ListUsers(rw http.ResponseWriter, r *http.Request) {
	page, err := strconv.ParseInt(r.FormValue("page"), 10, 64)
	if err != nil {
		// Invalid parameter: page.
		return
	}
	perPage, err := strconv.ParseInt(r.FormValue("per_page"), 10, 64)
	if err != nil {
		// Invalid parameter: per_page.
		return
	}
	isMember, err := strconv.ParseBool(r.FormValue("is_member"))
	if err != nil {
		// Invalid parameter: is_member.
		return
	}

	// Read database and return the user list.
}

It's okay if there are only a few (usually less than 3) parameters to be parsed. But what if more? Both the number of lots of local variables and the writing of statements of parsing string URL params to target types can kill us. Someone even spreads these statements all over in an API handler. 🤒

For such cases, what can we do to save a clean and tidy code base?

Using ggicci/httpin

httpin - 🍡HTTP Input for Go - Decode an HTTP request into a custom struct

ggicci/httpin is an awesome package which helps you easily decoding HTTP reqeusts from:

• Query string (URL parameters), e.g. ?name=john&is_member=true
• Headers, e.g. Authorization: xxx
• Form data, e.g. login=john&password=*****
• JSON/XML body, e.g. POST {"name": "john", "is_member": true}
• Path variables, e.g. /users/{username}
• File uploads

You don't need to write parsing code for any parameter by yourself. With httpin, you only care about the definition of the input struct and in which handler it will be used.

Let's see an example (working with net/http):

// 1. Define you input struct.
type ListUsersInput struct {
	Page     int  `in:"form=page"`
	PerPage  int  `in:"form=per_page"`
	IsMember bool `in:"form=is_member"`
}

// 2. Bind this input struct with an HTTP handler.
func init() {
	http.Handle("/users", alice.New(
		httpin.NewInput(ListUsersInput{}),
	).ThenFunc(ListUsers))
}

// 3. Get your input data in one line of code in your handler, all the parsing stuff are handled by httpin.
func ListUsers(rw http.ResponseWriter, r *http.Request) {
	input := r.Context().Value(httpin.Input).(*ListUsersInput)
}

httpin is:

• well documentated: at https://ggicci.github.io/httpin/.
• well tested: over 98% in coverage
• open integrated: with net/http, go-chi/chi, gorilla/mux, gin-gonic/gin, etc.
• extensible (advanced feature): by adding your custom directives. Read httpin - custom directives for more details.

You will find that with the help of httpin, you can get the following benefits:

• ⌛️ Saving developer time
• ♻️ Lower code repetition rate
• 📖 Higher readability
• 🔨 Higher maintainability

❤️ Have a try with it :)

And please don't forget to give it a big ⭐️, thanks in advance!

HTTP Live Streaming (HLS) is an HTTP-based adaptive bitrate streaming communications protocol developed by Apple Inc. and released in 2009.

Some key points of HLS:

1. Created by Apple.
2. Consists of a playlist/manifest file (e.g. index.m3u8) and segment video files (e.g. index01.ts).
3. H264 codec of video + AAC of audio.
4. Use HTTP, easily leveraging CDN to reach the widest audience without worring about the bandwidth and firewalls.
5. Adaptive streaming, enables changing the quality of the video mid-stream.
6. Widely supported across devicies and platforms. PC, mobile, Web, IOS, Andorid, etc.

Preface

Let's start from an sample video, which can be downloaded from vimeo. In this post, we are going to:

1. Convert the sample video from MP4 to HLS (a .m3u8 playlist file and .ts segment files) with AES-128 encryption method;
2. Serve the key files on a key server;
3. Serve the HLS files on a content server;
4. Test the deployment with VLC player and video.js.

🔄 Transcode (MP4 -> HLS)

The original sample video was in .mp4 format. While we need HLS files to serve with. We can use ffmpeg to do the transcoding.

Without AES-128, we can simply run the following command:

ffmpeg -i sample-video.mp4 -codec: copy -start_number 0 -hls_time 10 -hls_list_size 0 -f hls sample-noaes.m3u8

Open sample-noaes.m3u8 with VLC player. You should see it playing well.

With AES-128, we need to firstly generate an encryption key and an optional IV (initialization vector) for the AES algorithm.

openssl rand 16 > enc.key  # Key to encrypt the video
openssl rand -hex 16       # IV# de0efc88a53c730aa764648e545e3874

And then make a key info file having the following content:

KEY URI
Path to the key file
IV (optional)

Which is used by ffmpeg. e.g. enc.keyinfo:

https://ksm.ggicci.me/e9672408-b38b-4465-ab47-519c554ae402/enc.key
enc.key
de0efc88a53c730aa764648e545e3874

• The first line is the URI of the key. Which will be written to m3u8 file. And the player will consult this URI for the key to decrypt the segments files while playing.
• The second line is the path to the file containg the encryption/decription key.
• The third line is an optional initialization vector.

Now then, we can feed it to ffmpeg to do the transcoding again. But this time, we should get an encrypted version of the output:

ffmpeg \
  -i sample-video.mp4 \  # input file
  -hls_time 9 \ # 9s for each chunk
  -hls_key_info_file enc.keyinfo \ # encryption key
  -hls_playlist_type vod \ # video on demand mode
  -hls_segment_filename "sample-%d.ts" \ # name the segment files in pattern
  sample.m3u8 # HLS playlist (aka. HLS manifest)

Open sample.m3u8 with VLC. It won't play. Because VLC tried to retrieve the key file from this URI https://ksm.ggicci.me/e9672408-b38b-4465-ab47-519c554ae402/enc.key as noted in sample.m3u8. But it failed. Since we don't have this key server ready, yet.

🔑 Start a Key Server

Let's serve our key file enc.key on a web server and make it accessible from the URI above. Then go back to check if VLC can play this sample.m3u8.

I recommend using Caddy to start up a web server. Which should be easy to learn. Sample site configuration in the Caddyfile:

ksm.ggicci.me {
  log {
    level INFO
  }

  tls {
    alpn http/1.1
  }

  file_server {
    root /var/www/ksm.ggicci.me
  }
}

Copy file enc.key to our web server in the path specified by the URI. And open sample.m3u8 again. This time it should work, since the key URI became accessible.

🎞️ Start a Content Server

So far, our encryption key file has been served well on our key server. As long as we move our video content to a web server, too. We can consume our videos online with assistance of video players. We call it a Content Server.

Again, we use Caddy to achieve our goal:

v.ggicci.me {
  log {
    level INFO
  }

  tls {
    alpn http/1.1
  }

  file_server {
    root /var/www/v.ggicci.me
    index index.html
  }
}

Copy the playlist file and segment files to the web server. Under /sample folder in my case. And it's now accessible at https://v.ggicci.me/sample/index.m3u8. Try open it with VLC player (Media > Open Network Stream...). It should work like a charm.

📺 Use of video.js

Now that we have set up servers to host our stream. Why not host a video player, too? Thus we can consume our videos through web browsers!

Here we will try video.js - the world's most popular open source HTML5 player framework.

With a little research on its official documentation. We can compose a test HTML like this:

<!DOCTYPE html>
<head>
  <link href="https://vjs.zencdn.net/7.11.4/video-js.css" rel="stylesheet" />
</head>

<body>
  <video
    id="my-video"
    class="video-js"
    controls
    preload="auto"
    height="420"
    poster="/sample/cover.png"
    data-setup="{}"
  >
    <source src="/sample/index.m3u8" type="application/x-mpegURL" />
    <p class="vjs-no-js">
      To view this video please enable JavaScript, and consider upgrading to a
      web browser that
      <a href="https://videojs.com/html5-video-support/" target="_blank"
        >supports HTML5 video</a
      >
    </p>
  </video>

  <script src="https://vjs.zencdn.net/7.11.4/video.min.js"></script>
</body>

Since we need to read encryption key files from ksm.ggicci.me on v.ggicci.me. We will meet CORS problem. Solving it by adding corresponding headers to ksm.ggicci.me's site configurations:

ksm.ggicci.me {
  log {
    level INFO
  }

  tls {
    alpn http/1.1
  }

  file_server {
    root /var/www/ksm.ggicci.me
  }

  header {
    Vary "Origin"
    Access-Control-Allow-Origin "https://v.ggicci.me"
    Access-Control-Allow-Methods "OPTIONS, HEAD, GET"
  }
}

Visit https://v.ggicci.me/sample to see the result. Also try it with your mobile devices :)

Architecture

References

• Apple Developer - HTTP Live Streaming
• HLS Encryption: How to Encrypt Videos in AES-128 For HTTP Live Streaming [2021 Update]
• Apple FPS - FairPlay Streaming

Generally speaking, as we were using some open-source libraries we will inevitably find that there are some bugs or points that can be improved. And sometimes we even have to develop a brand new feature based on these libraries to satisfy our own needs. Out of respect for sprit of dedication, contributing our code to these opensource libraries is a good way to give back to the community.

My Practice

When I decided to contribute to a repository, I will first try looking for if there is a contribution guide for it. Such guides usually lied as a section in the README or as a single document named CONTRIBUTING. Only when no guides were found, I will try to open a pull request to this repository on Github.

My Best Workflow

I'm using github.com/gorilla/mux as a sample repository to which I'm going to contribute. Here's my workflow:

1. On GitHub

Fork the repo to my account, e.g. github.com/ggicci/mux

2. On local machine

# clone the repo
git clone git@github.com:gorilla/mux.git

# rename origin to upstream
git remote rename origin upstream

# add my forked repo as the origin remote
git remote add origin git@github.com:ggicci/mux.git

# make some changes to the source code and commit to origin
git add ...
git commit ...
git push origin ...

3. Back to GitHub

Open a pull request from ggicci to gorilla.

If you don’t take any optimization measures, docker images can easily get large. And in most cases we just wrapped too many inessential things into the images. So, we should take actions to get rid of it.
Let’s take a look at a common example Dockerfile for building an application written in Go.

FROM golang:1.10.2-stretch

WORKDIR /go/src/github.com/ggicci/docker-example-server
COPY . .

RUN go install

LABEL \
 me.ggicci.appdemo.image.created="2006-01-02T15:04:05+08:00" \
 me.ggicci.appdemo.image.version="1.0.0"

# (and more)

WORKDIR /app
ENTRYPOINT [ "docker-example-server" ]
EXPOSE 8080

After building the image by running command docker build -t example:1.0.0 ., we can see the size of the image we get is 800+ MB.

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
example             1.0.0               9daf905d097d        12 seconds ago      801MB

However as we know, go programs are statically compiled and linked. We don’t need many things in the image to run the final binary:

1. Source code.
2. Go compiling tools (the go command).
3. Big linux image.

Then, let’s cut them off.

Use Multi-stage Builds

# Stage: builder
FROM golang:1.10.2-stretch as builder

WORKDIR /go/src/github.com/ggicci/docker-example-server
COPY . .

RUN go install

# Stage: runner
FROM alpine:3.7

WORKDIR /app
COPY --from=builder /go/bin/docker-example-server /app

LABEL \
  me.ggicci.appdemo.image.created="2006-01-02T15:04:05+08:00" \
  me.ggicci.appdemo.image.version="1.0.0"
  # (and more)

ENTRYPOINT [ "docker-example-server" ]
EXPOSE 8080

There are two stages defined in the Dockerfile above:

• builder: based on a linux image with go tools installed, we build our application;
• runner: based on a tiny linux image alpine, we copy the application binary from the builder image and discard the builder image.

Now, the image we get is small enough. Which is only 10+ MB.

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
example-2           1.0.0               77488a7264a1        9 seconds ago       11MB

Cheers 🎉