exist

When Imperfect Systems are Good, Actually: Bluesky's Lossy Timelines Often when designing systems, we aim for perfection in things like consistency of data,...

2 weeks ago

5

When Imperfect Systems are Good, Actually: Bluesky's Lossy Timelines

from exist [alt+shift+b] in AI

2 weeks ago

Often when designing systems, we aim for perfection in things like consistency of data, availability, latency, and more. The hardest part of system design is that it’s difficult (if not impossible) to design systems that have perfect consistency, perfect availability, incredibly...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

How HLS Works Over the past few weeks, I’ve been building out server-side short video support for Bluesky. The...

8 months ago

4

How HLS Works

from exist [alt+shift+b] in AI

8 months ago

Over the past few weeks, I’ve been building out server-side short video support for Bluesky. The major aim of this feature is to support short (90 second max) video streaming at a quality that doesn’t cost an arm and a leg for us to provide for free. In order to stay within these...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Your Data Fits in Memory (GraphD Part 1) I recently shipped a new revision of Bluesky’s global AppView at the start of February and things...

10 months ago

3

Your Data Fits in Memory (GraphD Part 1)

from exist [alt+shift+b] in AI

10 months ago

I recently shipped a new revision of Bluesky’s global AppView at the start of February and things have been going very well. The system scales and handles millions of users without breaking a sweat, the ScyllaDB-backed Data Plane service sits at under 5% DB load in the most...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

An entire Social Network in 1.6GB (GraphD Part 2) In Part 1 of this series, we tried to answer the question “who do you follow who also follows user...

10 months ago

3

An entire Social Network in 1.6GB (GraphD Part 2)

from exist [alt+shift+b] in AI

10 months ago

In Part 1 of this series, we tried to answer the question “who do you follow who also follows user B” in Bluesky, a social network with millions of users and hundreds of millions of follow relationships. At the conclusion of the post, we’d developed an in-memory graph store for...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Jetstream: Shrinking the AT Proto Firehose by >99% Bluesky recently saw a massive spike in activity in response to Brazil’s ban of Twitter. As a...

5 months ago

2

Jetstream: Shrinking the AT Proto Firehose by >99%

from exist [alt+shift+b] in AI

5 months ago

Bluesky recently saw a massive spike in activity in response to Brazil’s ban of Twitter. As a result, the AT Proto event firehose provided by Bluesky’s Relay at bsky.network has increased in volume by a huge amount. The average event rate during this surge increased by...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Workload Agnosticism in Large Language Models: The Foundation for the Next Generation of Computing As discussed in my previous post, LLMs such as OpenAI’s ChatGPT and GPT-4, Google’s Bard, and Meta’s...

a year ago

2

Workload Agnosticism in Large Language Models: The Foundation for the Next Generation of Computing

from exist [alt+shift+b] in AI

a year ago

As discussed in my previous post, LLMs such as OpenAI’s ChatGPT and GPT-4, Google’s Bard, and Meta’s LLaMA have risen seemingly out of nowhere, poised to disrupt the future of computing. Cloud Compute changed the landscape drastically when it was introduced by Amazon in the mid...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Scaling Go to 192 Cores with Heavy I/O For the past few months I’ve been working alongside Why, Jacob, Dan, and Divy on a new revision of...

a year ago

2

Scaling Go to 192 Cores with Heavy I/O

from exist [alt+shift+b] in AI

a year ago

For the past few months I’ve been working alongside Why, Jacob, Dan, and Divy on a new revision of Bluesky’s global AppView. The AppView is a piece of infrastructure that aggregates posts, likes, follows, etc. from all across ATProto and merges them into a consistent view of the...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Solving Thundering Herds with Request Coalescing in Go Caches are a wonderful way to make your most frequent operations cheaper. If you’ve got a resource...

a year ago

2

Solving Thundering Herds with Request Coalescing in Go

from exist [alt+shift+b] in AI

a year ago

Caches are a wonderful way to make your most frequent operations cheaper. If you’ve got a resource somewhere on disk (or a network hop away) that is accessed often, changes infrequently, and fits in memory, you’ve got an excellent candidate for a cache! Caching Celebrity...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Speeding up Postgres Queries by 200x with Analyze I’ve been working on a growing series of services that archive, analyze, and represent data from a...

a year ago

2

Speeding up Postgres Queries by 200x with Analyze

from exist [alt+shift+b] in AI

a year ago

I’ve been working on a growing series of services that archive, analyze, and represent data from a social network. Part of this process involves archiving every post on a social network, running Computer Vision models on every image posted to the network, and running sentiment...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

How to use ChatGPT to Write Good Code Faster There’s been a lot of hype about Large Language Models (LLMs) lately with lots of cool examples of...

a year ago

2

How to use ChatGPT to Write Good Code Faster

from exist [alt+shift+b] in AI

a year ago

There’s been a lot of hype about Large Language Models (LLMs) lately with lots of cool examples of people using tools like ChatGPT to draft Python scripts, Go code, and all sorts of other useful things. Okay, so how do I use ChatGPT effectively? How do I Use ChatGPT...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

Speeding Up Massive PostgreSQL Joins with Common Table Expressions I’ve been continuing to work on a growing series of services that archive, analyze, and represent...

a year ago

2

Speeding Up Massive PostgreSQL Joins with Common Table Expressions

from exist [alt+shift+b] in AI

a year ago

I’ve been continuing to work on a growing series of services that archive, analyze, and represent data from a social network. This network creates text-based posts at a rate of around 400,000 posts per day, and I’ve been feeding the posts through different ML models to try and...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

exist

A Tale of Two Technologies: Why Large Language Models are the Future and the Metaverse Isn't In the digital landscape of recent years, two major technologies have vied for the spotlight: the...

a year ago

2

A Tale of Two Technologies: Why Large Language Models are the Future and the Metaverse Isn't

from exist [alt+shift+b] in AI

a year ago

In the digital landscape of recent years, two major technologies have vied for the spotlight: the Metaverse and Large Language Models (LLMs). Though the Metaverse, a virtual reality-based universe, initially garnered significant attention and expectations, it ultimately failed to...

Direct Link [→] Remove from reading list Add to reading list [alt+a]

upvote [alt+ctrl+↑] downvote [alt+ctrl+↓] prev [↑] next [↓]

New here?

bored reading