Abusing Generators for Typesafety
Abusing Generators for Typesafety
The Typescript Effect library (in)famously promotes using its Typescript generator syntax to make its code style look more familiar to devs. Ironically, most Typescript devs are unfamiliar with generators too so it’s often not a useful comparison anyways. While teaching myself Effect I found it useful to rederive how exactly Effect generator-style functions work and justify why this syntax would even be desirable. Here’s what I found.
Contents
The Result type
I think the most valuable lesson most devs can take away from Effect is the mere existence of the Result type pattern. The Result<T, E> type encodes the idea that a function can either succeed, returning a value of type T, or fail, producing an error of type E. In Typescript it’s actually not possible to express in a function’s signature that it may throw an error at all; Typescript only gives tools for expressing the successful return type of a function. So then how could the Result<T, E> type be implemented?
The most obvious implementation is most of the way there: let’s just write functions so that they actually never throw and just return their error instead.
function parseJson(json: string) {
try {
return JSON.parse(json);
} catch (error) {
// return the parsing error instead of throwing it
return error;
}
}
From here callers can just check if the returned value is the parsed data or an instance of an Error. We can clean this up a bit by introducing a discriminator that lets us just check an attribute on an object to determine whether we’re dealing with the success value or the error value.
type Result<T, E> =
| {
type: "Ok";
value: T;
}
| {
type: "Err";
error: E;
};
function parseJson(json: string): Result<object, string> {
try {
return { type: "Ok", value: JSON.parse(json) };
} catch (error) {
return {
type: "Err",
error: error instanceof Error ? error.message : String(error),
};
}
}
const result = parseJson("{ nope }");
if (result.type === "Ok") {
console.log(result.value);
} else {
console.error(result.error); // will print the parsing error
}
Actually using the Result type
The Result type is a pretty useful pattern but it’s not very ergonomic to use. If we want to chain together a series of functions that might fail we have to write a lot of boilerplate to handle the error cases.
declare function mightFail1(): Result<number, Error>;
declare function mightFail2(input: number): Result<string, Error>;
const result1 = mightFail1();
if (result1.type === "Ok") {
const result2 = mightFail2(result1.value);
if (result2.type === "Ok") {
console.log(result2.value);
} else {
console.error(result2.error);
}
} else {
console.error(result1.error);
}
Functional programmer bros might recommend common FP tools like map, flatMap, and fold to give us handles for manipulating the success and error values without completely unwrapping them with .value or .error. But even then it ends up looking like callback hell from Promise.then chains. Take this example using the Result.andThen method from the great neverthrow library.
declare function f1(): Result<number, Error>;
declare function f2(input: number): Result<string, Error>;
declare function f3(input: string): Result<boolean, Error>;
const result = f1()
.andThen((num) => {
if (num < 0) {
return new Err<boolean, Error>(new Error("number must be positive"));
}
return f2(num);
})
.andThen((str) => {
if (str.length === 0) {
return new Err<boolean, Error>(new Error("string must not be empty"));
}
return f3(str);
});
if (result.isOk()) {
console.log(result.value);
} else {
console.error(result.error);
}
With promises we escaped callback hell by introducing the await keyword. Can we do something similar for Results?
Introducing Generators
I’ve been writing Javascript for a long time and didn’t even realize the language had generators until I started investigating Effect. They have an API almost identical to Python generators, but are much less popular. The basic idea is that a generator function has two ways to “return” a value to its caller: either it can return a value directly like a normal function or it can yield. When a generator function yields a value it pauses itself and gives a value back to the caller. Note that during the “pause” the generator retains all of its internal state like local variables and the call stack. The next time we call the generator, it resumes execution from its last yield point and runs until it either yields another value or returns like normal.
# Python generators
def generator():
yield 1
yield 2
yield 3
gen = generator()
print(next(gen)) # 1
print(next(gen)) # 2
print(next(gen)) # 3
// Typescript generators
function* generator() {
yield 1;
yield 2;
yield 3;
}
const gen = generator();
console.log(gen.next()); // 1
console.log(gen.next()); // 2
console.log(gen.next()); // 3
Importantly, generators can themselves run other generators. Often one generator wants to completely delegate to another generator, treating the sub-generator as a subsequence. In Python we use the yield from keyword and in Typescript we use the yield* keyword to express this delegation.
# Python generators
def subgenerator():
yield 4
yield 5
yield 6
def generator():
yield 1
yield 2
yield 3
def outer_generator():
yield from generator()
yield from subgenerator()
yield 7
yield 8
yield 9
gen = outer_generator()
print(list(gen)) # [1, 2, 3, 4, 5, 6, 7, 8, 9]
// Typescript generators
function* subgenerator() {
yield 4;
yield 5;
yield 6;
return "done";
}
function* generator() {
yield 1;
yield 2;
yield 3;
return "done";
}
function* outerGenerator() {
yield* generator();
yield* subgenerator();
yield 7;
yield 8;
yield 9;
return "done";
}
// The `return` just indicates the generator is done - it doesn't affect the values that are yielded
console.log([...outerGenerator()]); // [1, 2, 3, 4, 5, 6, 7, 8, 9]
Multiple Return Channels
The trick to using generators for processing Results is to realize that what we actually want is some system to express two return channels: one for the success value and one for the error value. Generators can provide this with the yield channel and the normal return channel. Imagine if we could achieve syntax like:
function* parseJson(json: string) {
try {
// put success values in the normal return channel
return JSON.parse(json);
} catch (error) {
// put errors in the yield channel
yield new Error("Failed to parse JSON");
}
}
function* checkIsObject(o: unknown) {
if (typeof o !== "object" || o === null) {
yield new Error("Value must be an object");
}
return o;
}
function* chainResults(json: string) {
// yield* delegates to the inner generator's yield channel
const parsed = yield* parseJson(json);
const object = yield* checkIsObject(parsed);
return object;
}
This syntax is not far off from what we’re used to, but now that we’ve specified errors in their own yield channel we’d hope that Typescript can infer the actual error types for us.
function* chainResults(json: string) {
const parsed = yield* parseJson(json);
const object = yield* checkIsObject(parsed);
return object;
}
const final = chainResults('{ "name": "John", "age": 30 }');
// ^ Return channel: number; Yield channel: Error("Number must be positive") | Error("Failed to parse JSON")
Full Implementation
Now here is a minimal implementation of the Result type with generator semantics. The whole thing fits in fewer than 50 lines. I strongly recommend checking out the code on Typescript Playground so you can hover the type inferences and actually execute it to see how the data flows. This implementation is heavily inspired by neverthrow so you should check out their implementation too.
class Ok<T, E> {
value: T;
constructor(given: T) {
this.value = given;
}
// an Ok is an iterator that immediately stops - never yields
*[Symbol.iterator](): Generator<Err<never, E>, T> {
return this.value;
}
}
class Err<T, E> {
error: E;
constructor(given: E) {
this.error = given;
}
// an Err is an iterator that yields itself
*[Symbol.iterator](): Generator<Err<never, E>, T> {
// @ts-expect-error - the T generic is fake so Err<T, E> is functionally assignable to Err<never, E>
yield this;
// @ts-expect-error - the T generic is fake and we need it to match the generator return of Ok
return this;
}
}
type Result<T, E> = Ok<T, E> | Err<T, E>;
type InferOkTypes<R> = R extends Result<infer T, unknown> ? T : never;
type InferErrTypes<R> = R extends Result<unknown, infer E> ? E : never;
// this extra function signature overload allows for type inference to merge the Err variants of each yield* in the generator
function runGenerator<
YieldErr extends Err<never, unknown>,
GeneratorReturnResult extends Result<unknown, unknown>,
>(
gen: () => Generator<YieldErr, GeneratorReturnResult>,
): Result<
InferOkTypes<GeneratorReturnResult>,
InferErrTypes<YieldErr> | InferErrTypes<GeneratorReturnResult>
>;
function runGenerator<T, E>(
gen: () => Generator<Err<never, E>, Result<T, E>>,
): Result<T, E> {
// just take the first thing that is yielded (or returned if nothing yields)
const n = gen().next();
return n.value;
}
// ========== example =============
function couldFail(): Result<string, boolean> {
if (Math.random() < 0.5) {
return new Err(false);
}
return new Ok("yes");
}
function mightFail(): Result<number, Date> {
if (Math.random() < 0.1) {
return new Err(new Date());
}
return new Ok(100);
}
function* genResults() {
// yield* does nothing for the Ok iterator as it does not yield so you are left with the return value T
const a = yield* couldFail();
// yield* delegates to the Err iterator which will just yield itself up to the caller of `genResults`
const b = yield* mightFail();
if (b < 0) {
// technically the yield* is unnecessary but I think it's good practice to yield* anything that _might_ be an Err
return yield* new Err<never, Error>(new Error("must be positive"));
}
return new Ok<string, never>(`String ${a} - Number ${b}`);
}
const final = runGenerator(genResults);
// ^ Result<string, boolean | Date | Error>
console.log(final); // Ok: { "value": "String yes - Number 100" } or Err: { "error": false }
Many people like to compare the const a = yield* couldFail() syntax to promises like await couldFail(). Here yield* is telling you “bubble up the error value or give me the success value” while await is telling you “code will pause here until the promise is resolved”. In both cases the value assigned to a is always the success value and in both cases the caller has an error bubbled up to them. The difference is that yield* gets to tell the type system exactly what that error type is.
// with promises
let result;
promiseResults()
.then((res) => {
result = res;
})
.catch((err) => {
result = err; // type is unknown
});
// with results
const result = runGenerator(genResults);
if (result.isOk()) {
console.log(result.value);
} else {
console.error(result.error); // type is the Yield channel of genResults
}
Conclusion
The Effect library’s generator syntax is scary at first but it’s ultimately just a mechanism for expressing multiple return channels. If you were previously intimidated by it, now you can just be intimidated by the giant Effect ecosystem and runtime instead. I’d highly HIGHLY recommend at least giving neverthrow a shot in your projects. It’s much simpler than Effect but gives you all the same typesafety for the Result type. Though, once you’re comfortable with neverthrow you should definitely give Effect another shot and see if the extra bells and whistles aren’t too scary anymore.