More from Cognitive Computations
With the recent update to OpenAI's Terms of Use on October 23, 2024, there’s been a flurry of online discussions around what these terms mean for developers, businesses, and everyday users of AI tools like ChatGPT. Much of the conversation, especiall...
Gratitude to https://tensorwave.com/ for giving me access to their excellent servers! Few have tried this and fewer have succeeded. I've been marginally successful after a significant amount of effort, so it deserves a blog post. Know that you are in for rough waters. And even when you arrive - There are lots of optimizations tailored for nVidia GPUs so, even though the hardware may be just as strong spec-wise, in my experience so far, it still may take 2-3 times as long to train on equivalient AMD hardware. (though if you are a super hacker maybe you can fix it!) Right now I'm using Axolotl. Though I am probably going to give LlamaFactory a solid try in the near future. There's also LitGpt and TRL. But I kind of rely on the dataset features and especially the sample packing of Axolotl. But more and more LlamaFactory is interesting me, it supports new features really fast. (like GaLore is the new hotness at the moment). This blog post will be about getting Axolotl up and running in AMD, and I may do one about LlamaFactory if there is demand. I am using Ubuntu 22.04 LTS, and you should too. (unless this blog post is really old by the time you read it). Otherwise you can use this post as a general guide. Here are all the environment variables I ended up setting in my .bashrc and I'm not exactly sure which ones are needed. You better set them all just in case. export GPU_ARCHS="gfx90a" # mi210 - use the right code for your GPUexport ROCM_TARGET="gfx90a"export HIP_PATH="/opt/rocm-6.0.0"export ROCM_PATH="/opt/rocm-6.0.0"export ROCM_HOME="/opt/rocm-6.0.0"export HIP_PLATFORM=amdexport DS_BUILD_CPU_ADAM=1 export TORCH_HIP_ARCH_LIST="gfx90a" Part 1: Driver, ROCm, HIP Clean everything out. There shouldn't be any trace of nvidia, cuda, amd, hip, rocm, anything like that. This is not necessarily a simple task, and of course it totally depends on the current state of your system. and I had to use like 4 of my daily Claude Opus questions to accomplish this. (sad face) By the way Anthropic Claude Opus is the new king of interactive troubleshooting. By far. Bravo. Don't nerf it pretty please! Here are some things I had to do, that might help you: sudo apt autoremove rocm-core sudo apt remove amdgpu-dkms sudo dpkg --remove --force-all amdgpu-dkms sudo apt purge amdgpu-dkms sudo apt remove --purge nvidia* sudo apt remove --purge cuda* sudo apt remove --purge rocm-* hip-* sudo apt remove --purge amdgpu-* xserver-xorg-video-amdgpu sudo apt clean sudo reboot sudo dpkg --remove amdgpu-install sudo apt remove --purge amdgpu-* xserver-xorg-video-amdgpu sudo apt autoremove sudo apt clean rm ~/amdgpu-install_*.deb sudo reboot sudo rm /etc/apt/sources.list.d/amdgpu.list sudo rm /etc/apt/sources.list.d/rocm.list sudo rm /etc/apt/sources.list.d/cuda.list sudo apt-key del A4B469963BF863CC sudo apt update sudo apt remove --purge nvidia-* cuda-* rocm-* hip-* amdgpu-* sudo apt autoremove sudo apt clean sudo rm -rf /etc/OpenCL /etc/OpenCL.conf /etc/amd /etc/rocm.d /usr/lib/x86_64-linux-gnu/amdgpu /usr/lib/x86_64-linux-gnu/rocm /opt/rocm-* /opt/amdgpu-pro-* /usr/lib/x86_64-linux-gnu/amdvlk sudo reboot I love Linux (smile with tear) Now finally do like sudo apt-get updatesudo apt-get upgrade and sudo apt-get dist-upgrade and make sure there's no errors or warnings! You should be good to begin your journey. Install AMD drivers, ROCm, HIP wgethttps://repo.radeon.com/amdgpu-install/23.40.2/ubuntu/jammy/amdgpu-install_6.0.60002-1_all.deb (at time of this writing). But you should double check here. And the install instructions here. sudo apt-get install ./amdgpu-install_6.0.60002-1_all.deb sudo apt-get update sudo amdgpu-install -y --accept-eula --opencl=rocr --vulkan=amdvlk --usecase=workstation,rocm,rocmdev,rocmdevtools,lrt,opencl,openclsdk,hip,hiplibsdk,mllib,mlsdk If you get error messages (I did) try to fix them. I had to do this: sudo dpkg --remove --force-all libvdpau1 sudo apt clean sudo apt update sudo apt --fix-broken install sudo apt upgrade and then, again, I had to run sudo amdgpu-install -y --accept-eula --opencl=rocr --vulkan=amdvlk --usecase=workstation,rocm,rocmdev,rocmdevtools,lrt,opencl,openclsdk,hip,hiplibsdk,mllib,mlsdk Check Installation rocm-smirocminfo/opt/rocm/bin/hipconfig --full I hope that worked for you - if not, I suggest asking Claude Opus about the error messages to help you figure it out. If that doesn't work, reach out to the community. Part 2: Pytorch, BitsAndBytes, Flash Attention, DeepSpeed, Axolotl Conda mkdir -p ~/miniconda3wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.shbash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3rm -rf ~/miniconda3/miniconda.sh~/miniconda3/bin/conda init bash Exit your shell and enter it again. conda create -n axolotl python=3.12conda activate axolotl Pytorch I tried the official install command from pytorch's website, and it didn't work for me. Here is what did work: pip install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.0python -c "import torch; print(torch.version.hip)" This tests both Torch, and Torch's ability to interface with HIP. If it worked, it will print HIP version. Otherwise, it will print None. BitsAndBytes BitsAndBytes is by Tim Dettmers, an absolute hero among men. It lets us finetune in 4-bits. It gives us qLoRA. It brings AI to the masses. There is a fork of BitsAndBytes that supports ROCm. This is provided not by Tim Dettmers, and not by AMD, but by a vigilante superhero, Arlo-Phoenix. In appreciation, here is a portrait ChatGPT made for Arlo-Phoenix, vigilante superhero. I hope you like it, if you see this Arlo-Phoenix. <3 git clone https://github.com/arlo-phoenix/bitsandbytes-rocm-5.6cd bitsandbytes-rocm-5.6git checkout rocmROCM_TARGET=gfx90a make hip # use the ROCM_TARGET for your GPUpip install . Flash Attention This fork is maintained by AMD git clone --recursive https://github.com/ROCmSoftwarePlatform/flash-attention.gitcd flash-attentionexport GPU_ARCHS="gfx90a" # use the GPU_ARCHS for your GPUpip install . DeepSpeed Microsoft included AMD support in DeepSpeed proper, but there's still some undocumented fussiness to get it working, and there is a bug I found with DeepSpeed, I had to modify it to get it to work. git clone https://github.com/microsoft/DeepSpeedcd DeepSpeedgit checkout v0.14.0 # but check the tags for newer version Now, you gotta modify this file: vim op_builder/builder.py Replace the function assert_no_cuda_mismatch with this: (unless they fixed it yet) def assert_no_cuda_mismatch(name=""): cuda_available = torch.cuda.is_available() if not cuda_available and not torch.version.hip: # Print a warning message indicating no CUDA or ROCm support print(f"Warning: {name} requires CUDA or ROCm support, but neither is available.") return False else: # Check CUDA version if available if cuda_available: cuda_major, cuda_minor = installed_cuda_version(name) sys_cuda_version = f'{cuda_major}.{cuda_minor}' torch_cuda_version = torch.version.cuda if torch_cuda_version is not None: torch_cuda_version = ".".join(torch_cuda_version.split('.')[:2]) if sys_cuda_version != torch_cuda_version: if (cuda_major in cuda_minor_mismatch_ok and sys_cuda_version in cuda_minor_mismatch_ok[cuda_major] and torch_cuda_version in cuda_minor_mismatch_ok[cuda_major]): print(f"Installed CUDA version {sys_cuda_version} does not match the " f"version torch was compiled with {torch.version.cuda} " "but since the APIs are compatible, accepting this combination") return True elif os.getenv("DS_SKIP_CUDA_CHECK", "0") == "1": print( f"{WARNING} DeepSpeed Op Builder: Installed CUDA version {sys_cuda_version} does not match the " f"version torch was compiled with {torch.version.cuda}." "Detected `DS_SKIP_CUDA_CHECK=1`: Allowing this combination of CUDA, but it may result in unexpected behavior." ) return True raise CUDAMismatchException( f">- DeepSpeed Op Builder: Installed CUDA version {sys_cuda_version} does not match the " f"version torch was compiled with {torch.version.cuda}, unable to compile " "cuda/cpp extensions without a matching cuda version.") else: print(f"Warning: {name} requires CUDA support, but torch.version.cuda is None.") return False return True pip install -r requirements/requirements.txtHIP_PLATFORM="amd" DS_BUILD_CPU_ADAM=1 TORCH_HIP_ARCH_LIST="gfx90a" python setup.py install Axolotl Installing Axolotl might overwrite BitsAndBytes, DeepSpeed, and PyTorch. Be prepared for things to break, they do often. Your choice is either modify the setup.py and requirements.txt (if you are confident to change those things) or pay attention to what libraries get deleted and reinstalled, and just delete them again and reinstall the correct ROCm version that you installed earlier. If Axolotl complains about incorrect versions - just ignore it, you know better than Axolotl. Right now, Axolotl's Flash Attention implementation has a hard dependency on Xformers for its SwiGLU implementation, and Xformers doesn't work with ROCm, you can't even install it. So, we are gonna have to hack axolotl to remove that dependency. https://github.com/OpenAccess-AI-Collective/axolotl.gitcd axolotl from requirements.txt remove xformers==0.0.22 from setup.py make this change (remove any mention of xformers) $ git diff setup.pydiff --git a/setup.py b/setup.pyindex 40dd0a6..235f1d0 100644--- a/setup.py+++ b/setup.py@@ -30,7 +30,7 @@ def parse_requirements(): try: if "Darwin" in platform.system():- _install_requires.pop(_install_requires.index("xformers==0.0.22"))+ print("hi") else: torch_version = version("torch") _install_requires.append(f"torch=={torch_version}")@@ -45,9 +45,6 @@ def parse_requirements(): else: raise ValueError("Invalid version format")- if (major, minor) >= (2, 1):- _install_requires.pop(_install_requires.index("xformers==0.0.22"))- _install_requires.append("xformers>=0.0.23") except PackageNotFoundError: pass And then in src/axolotl/monkeypatch/llama_attn_hijack_flash.py make this change: --- a/src/axolotl/monkeypatch/llama_attn_hijack_flash.py+++ b/src/axolotl/monkeypatch/llama_attn_hijack_flash.py@@ -22,7 +22,9 @@ from transformers.models.llama.modeling_llama import ( apply_rotary_pos_emb, repeat_kv, )-from xformers.ops import SwiGLU+class SwiGLU:+ def __init__():+ print("hi") from axolotl.monkeypatch.utils import get_cu_seqlens_from_pos_ids, set_module_name@@ -45,15 +47,7 @@ LOG = logging.getLogger("axolotl") def is_xformers_swiglu_available() -> bool:- from xformers.ops.common import get_xformers_operator-- try:- get_xformers_operator("swiglu_packedw")()- return True- except RuntimeError as exc:- if "No such operator xformers::swiglu_packedw " in str(exc):- return False- return True+ return False Now you can install axolotl pip install -e .accelerate launch -m axolotl.cli.train examples/openllama-3b/lora.yml Welcome to finetuning on ROCm!
I started to understand that a lot of people are using and enjoying Dolphin - so I decided to put a list here of products or projects that use Dolphin. If you would like to be listed here please reach out to me and I'll add you! HopeBot https://disboard.org/server/696448387964469339 I am part of a staff team that runs a Discord server for those struggling with addiction. We have a few docments that we've created over the years, which compile healthy strategies and coping mechanisms for addicts. But, these documents have grown unwieldy over the years, and sometimes its easier just say what your issue is and get some advice on what you can do better. So, we created HopeBotnamed after Hope, one of our staff members. HopeBot was taught about addiction in general, and even about our particular server, so that members can ask a question to HopeBot and get a relevant, thoughtful response. We've only had HopeBot around for about a week, and we've already gotten so much positive feedback .... I am truly grateful to Eric for his work on Dolphin, and so are our members. Thank you!! Recovery Staff Team
Wanna chat with Dolphin locally? (no internet connection needed) Here is the easy way - Ollama. install ollama. after you finsh you should be able to run ollama from the command line. Also you will see the ollama icon up top like this: Iff you are curious - anytime you see that icon, that means ollama is running in the background and it also has a port open (11434) that can accept api calls. Sadly, as of this writing there is no swagger. Back to business. download your dolphin. You can choose which size you want from here. I am choosing size Q5_K_M which is just over 5gb. mkdir ~/models cd ~/models wget https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF/resolve/main/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf Now you need a Modelfile. vim Modelfile.dolphin.mistral Add the following text to Modelfile.dolphin.mistral FROM ./dolphin-2.2.1-mistral-7b.Q5_K_M.gguf TEMPLATE """{{ if and .First .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}<|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant """ SYSTEM """You are Dolphin, an uncensored, unbiased AI assistant""" PARAMETER stop <|im_end|> Now you can use Ollama to install this model. ollama create dolphin.mistral -f Modelfile.dolphin.mistral Now look, you can run it from the command line. Which is cool enough. But we are just getting started. If you want, you can install samantha too so you have two models to play with. wget https://huggingface.co/TheBloke/samantha-1.2-mistral-7B-GGUF/resolve/main/sama ntha-1.2-mistral-7b.Q5_K_M.gguf vim Modelfile.samantha.mistral And enter the following into Modelfile.samantha.mistral FROM ./samantha-1.2-mistral-7b.Q5_K_M.gguf TEMPLATE """{{ if and .First .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}<|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant """ SYSTEM """You are Samantha, an AI companion""" PARAMETER stop <|im_end|> Then install the model ollama create samantha -f Modelfile.samantha.mistral And now you can also chat with Samantha from the command line. Cool yeah? We are just getting started. Let's get Ollama Web UI installed. cd ~ git clone https://github.com/ollama-webui/ollama-webui.git cd ollama-webui npm i npm run dev Now you can open that link http://localhost:5173 in your web browser. now you can choose dolphin or samantha from the dropdown (I have installed a few others too) Well talking to these models from the command line and the web ui is just the beginning. Also, frameworks such as langchain, llamaindex, litellm, autogen, memgpt all can integrate with ollama. Now you can really play with these models. Here is a fun idea that I will leave as an exercise - given some query, ask dolphin to decide whether a question about coding, a request for companionship, or something else. If it is a request for companionship then send it to Samantha. If it is a coding question, send it to deepseek-coder. Otherwise, send it to Dolphin. And just like that, you have your own MoE.
More in programming
If your executive calendar is packed back to back, you have no room for fires, customers, or serendipities. You've traded all your availability for efficiency. That's a bad deal. Executives of old used to know this! That's what the long lunches, early escapes to the golf course, and reading the paper at work were all about. A great fictional example of this is Bert Cooper from Mad Men. He knew his value was largely in his network. He didn't have to grind every minute of every day to prove otherwise. His function was to leap into action when the critical occasion arose or decision needed to be made. But modern executives are so insecure about seeming busy 24/7 that they'll wreck their business trying to prove it. Trying to outwork everyone. Sacrificing themselves thin so they can run a squirrel-brain operation that's constantly chasing every nutty idea. Now someone is inevitably going to say "but what about Elon!!". He's actually a perfect illustration of doing this right, actually. Even if he works 100-hour weeks, he's the CEO of 3 companies, has a Diablo VI addiction, and keeps a busy tweeting schedule too. In all of that, I'd be surprised if there was more than 20-30h per company per week on average. And your boss is not Elon. Wide open calendars should not be seen as lazy, but as intentional availability. It's time we brought them back into vogue.
A secret master plan, the official launch of Automerge 3, and an update on Sketchy Calendars
Create a small example app and send payloads from the server to the client using RSC's
I'm way too discombobulated from getting next month's release of Logic for Programmers ready, so I'm pulling a idea from the slush pile. Basically I wanted to come up with a mental model of arrays as a concept that explained APL-style multidimensional arrays and tables but also why there weren't multitables. So, arrays. In all languages they are basically the same: they map a sequence of numbers (I'll use 1..N)1 to homogeneous values (values of a single type). This is in contrast to the other two foundational types, associative arrays (which map an arbitrary type to homogeneous values) and structs (which map a fixed set of keys to heterogeneous values). Arrays appear in PLs earlier than the other two, possibly because they have the simplest implementation and the most obvious application to scientific computing. The OG FORTRAN had arrays. I'm interested in two structural extensions to arrays. The first, found in languages like nushell and frameworks like Pandas, is the table. Tables have string keys like a struct and indexes like an array. Each row is a struct, so you can get "all values in this column" or "all values for this row". They're heavily used in databases and data science. The other extension is the N-dimensional array, mostly seen in APLs like Dyalog and J. Think of this like arrays-of-arrays(-of-arrays), except all arrays at the same depth have the same length. So [[1,2,3],[4]] is not a 2D array, but [[1,2,3],[4,5,6]] is. This means that N-arrays can be queried on any axis. ]x =: i. 3 3 0 1 2 3 4 5 6 7 8 0 { x NB. first row 0 1 2 0 {"1 x NB. first column 0 3 6 So, I've had some ideas on a conceptual model of arrays that explains all of these variations and possibly predicts new variations. I wrote up my notes and did the bare minimum of editing and polishing. Somehow it ended up being 2000 words. 1-dimensional arrays A one-dimensional array is a function over 1..N for some N. To be clear this is math functions, not programming functions. Programming functions take values of a type and perform computations on them. Math functions take values of a fixed set and return values of another set. So the array [a, b, c, d] can be represented by the function (1 -> a ++ 2 -> b ++ 3 -> c ++ 4 -> d). Let's write the set of all four element character arrays as 1..4 -> char. 1..4 is the function's domain. The set of all character arrays is the empty array + the functions with domain 1..1 + the functions with domain 1..2 + ... Let's call this set Array[Char]. Our compilers can enforce that a type belongs to Array[Char], but some operations care about the more specific type, like matrix multiplication. This is either checked with the runtime type or, in exotic enough languages, with static dependent types. (This is actually how TLA+ does things: the basic collection types are functions and sets, and a function with domain 1..N is a sequence.) 2-dimensional arrays Now take the 3x4 matrix i. 3 4 0 1 2 3 4 5 6 7 8 9 10 11 There are two equally valid ways to represent the array function: A function that takes a row and a column and returns the value at that index, so it would look like f(r: 1..3, c: 1..4) -> Int. A function that takes a row and returns that column as an array, aka another function: f(r: 1..3) -> g(c: 1..4) -> Int.2 Man, (2) looks a lot like currying! In Haskell, functions can only have one parameter. If you write (+) 6 10, (+) 6 first returns a new function f y = y + 6, and then applies f 10 to get 16. So (+) has the type signature Int -> Int -> Int: it's a function that takes an Int and returns a function of type Int -> Int.3 Similarly, our 2D array can be represented as an array function that returns array functions: it has type 1..3 -> 1..4 -> Int, meaning it takes a row index and returns 1..4 -> Int, aka a single array. (This differs from conventional array-of-arrays because it forces all of the subarrays to have the same domain, aka the same length. If we wanted to permit ragged arrays, we would instead have the type 1..3 -> Array[Int].) Why is this useful? A couple of reasons. First of all, we can apply function transformations to arrays, like "combinators". For example, we can flip any function of type a -> b -> c into a function of type b -> a -> c. So given a function that takes rows and returns columns, we can produce one that takes columns and returns rows. That's just a matrix transposition! Second, we can extend this to any number of dimensions: a three-dimensional array is one with type 1..M -> 1..N -> 1..O -> V. We can still use function transformations to rearrange the array along any ordering of axes. Speaking of dimensions: What are dimensions, anyway Okay, so now imagine we have a Row × Col grid of pixels, where each pixel is a struct of type Pixel(R: int, G: int, B: int). So the array is Row -> Col -> Pixel But we can also represent the Pixel struct with a function: Pixel(R: 0, G: 0, B: 255) is the function where f(R) = 0, f(G) = 0, f(B) = 255, making it a function of type {R, G, B} -> Int. So the array is actually the function Row -> Col -> {R, G, B} -> Int And then we can rearrange the parameters of the function like this: {R, G, B} -> Row -> Col -> Int Even though the set {R, G, B} is not of form 1..N, this clearly has a real meaning: f[R] is the function mapping each coordinate to that coordinate's red value. What about Row -> {R, G, B} -> Col -> Int? That's for each row, the 3 × Col array mapping each color to that row's intensities. Really any finite set can be a "dimension". Recording the monitor over a span of time? Frame -> Row -> Col -> Color -> Int. Recording a bunch of computers over some time? Computer -> Frame -> Row …. This is pretty common in constraint satisfaction! Like if you're conference trying to assign talks to talk slots, your array might be type (Day, Time, Room) -> Talk, where Day/Time/Room are enumerations. An implementation constraint is that most programming languages only allow integer indexes, so we have to replace Rooms and Colors with numerical enumerations over the set. As long as the set is finite, this is always possible, and for struct-functions, we can always choose the indexing on the lexicographic ordering of the keys. But we lose type safety. Why tables are different One more example: Day -> Hour -> Airport(name: str, flights: int, revenue: USD). Can we turn the struct into a dimension like before? In this case, no. We were able to make Color an axis because we could turn Pixel into a Color -> Int function, and we could only do that because all of the fields of the struct had the same type. This time, the fields are different types. So we can't convert {name, flights, revenue} into an axis. 4 One thing we can do is convert it to three separate functions: airport: Day -> Hour -> Str flights: Day -> Hour -> Int revenue: Day -> Hour -> USD But we want to keep all of the data in one place. That's where tables come in: an array-of-structs is isomorphic to a struct-of-arrays: AirportColumns( airport: Day -> Hour -> Str, flights: Day -> Hour -> Int, revenue: Day -> Hour -> USD, ) The table is a sort of both representations simultaneously. If this was a pandas dataframe, df["airport"] would get the airport column, while df.loc[day1] would get the first day's data. I don't think many table implementations support more than one axis dimension but there's no reason they couldn't. These are also possible transforms: Hour -> NamesAreHard( airport: Day -> Str, flights: Day -> Int, revenue: Day -> USD, ) Day -> Whatever( airport: Hour -> Str, flights: Hour -> Int, revenue: Hour -> USD, ) In my mental model, the heterogeneous struct acts as a "block" in the array. We can't remove it, we can only push an index into the fields or pull a shared column out. But there's no way to convert a heterogeneous table into an array. Actually there is a terrible way Most languages have unions or product types that let us say "this is a string OR integer". So we can make our airport data Day -> Hour -> AirportKey -> Int | Str | USD. Heck, might as well just say it's Day -> Hour -> AirportKey -> Any. But would anybody really be mad enough to use that in practice? Oh wait J does exactly that. J has an opaque datatype called a "box". A "table" is a function Dim1 -> Dim2 -> Box. You can see some examples of what that looks like here Misc Thoughts and Questions The heterogeneity barrier seems like it explains why we don't see multiple axes of table columns, while we do see multiple axes of array dimensions. But is that actually why? Is there a system out there that does have multiple columnar axes? The array x = [[a, b, a], [b, b, b]] has type 1..2 -> 1..3 -> {a, b}. Can we rearrange it to 1..2 -> {a, b} -> 1..3? No. But we can rearrange it to 1..2 -> {a, b} -> PowerSet(1..3), which maps rows and characters to columns with that character. [(a -> {1, 3} ++ b -> {2}), (a -> {} ++ b -> {1, 2, 3}]. We can also transform Row -> PowerSet(Col) into Row -> Col -> Bool, aka a boolean matrix. This makes sense to me as both forms are means of representing directed graphs. Are other function combinators useful for thinking about arrays? Does this model cover pivot tables? Can we extend it to relational data with multiple tables? Systems Distributed Talk (will be) Online The premier will be August 6 at 12 CST, here! I'll be there to answer questions / mock my own performance / generally make a fool of myself. Sacrilege! But it turns out in this context, it's easier to use 1-indexing than 0-indexing. In the years since I wrote that article I've settled on "each indexing choice matches different kinds of mathematical work", so mathematicians and computer scientists are best served by being able to choose their index. But software engineers need consistency, and 0-indexing is overall a net better consistency pick. ↩ This is right-associative: a -> b -> c means a -> (b -> c), not (a -> b) -> c. (1..3 -> 1..4) -> Int would be the associative array that maps length-3 arrays to integers. ↩ Technically it has type Num a => a -> a -> a, since (+) works on floats too. ↩ Notice that if each Airport had a unique name, we could pull it out into AirportName -> Airport(flights, revenue), but we still are stuck with two different values. ↩
We don’t want to bury the lede: we have raised a $100M Series B, led by a new strategic partner in USIT with participation from all existing Oxide investors. To put that number in perspective: over the nearly six year lifetime of the company, we have raised $89M; our $100M Series B more than doubles our total capital raised to date — and positions us to make Oxide the generational company that we have always aspired it to be. If this aspiration seems heady now, it seemed absolutely outlandish when we were first raising venture capital in 2019. Our thesis was that cloud computing was the future of all computing; that running on-premises would remain (or become!) strategically important for many; that the entire stack — hardware and software — needed to be rethought from first principles to serve this market; and that a large, durable, public company could be built by whomever pulled it off. This scope wasn’t immediately clear to all potential investors, some of whom seemed to latch on to one aspect or another without understanding the whole. Their objections were revealing: "We know you can build this," began more than one venture capitalist (at which we bit our tongue; were we not properly explaining what we intended to build?!), "but we don’t think that there is a market." Entrepreneurs must become accustomed to rejection, but this flavor was particularly frustrating because it was exactly backwards: we felt that there was in fact substantial technical risk in the enormity of the task we put before ourselves — but we also knew that if we could build it (a huge if!) there was a huge market, desperate for cloud computing on-premises. Fortunately, in Eclipse Ventures we found investors who saw what we saw: that the most important products come when we co-design hardware and software together, and that the on-premises market was sick of being told that they either don’t exist or that they don’t deserve modernity. These bold investors — like the customers we sought to serve — had been waiting for this company to come along; we raised seed capital, and started building. And build it we did, making good on our initial technical vision: We did our own board designs, allowing for essential system foundation like a true hardware root-of-trust and end-to-end power observability. We did our own microcontroller operating system, and used it to replace the traditional BMC. We did our own platform enablement software, eliminating the traditional UEFI BIOS and its accompanying flotilla of vulnerabilities. We did our own host hypervisor, assuring an integrated and seamless user experience — and eliminating the need for a third-party hypervisor and its concomitant rapacious software licensing. We did our own switch — and our own switch runtime — eliminating entire universes of integration complexity and operational nightmares. We did our own integrated storage service, allowing the rack-scale system to have reliable, available, durable, elastic instance storage without necessitating a dependency on a third party. We did our own control plane, a sophisticated distributed system building on the foundation of our hardware and software components to deliver the API-driven services that modernity demands: elastic compute, virtual networking, and virtual storage. While these technological components are each very important (and each is in service to specific customer problems when deploying infrastructure on-premises), the objective is the product, not its parts. The journey to a product was long, but we ticked off the milestones. We got the boards brought up. We got the switch transiting packets. We got the control plane working. We got the rack manufactured. We passed FCC compliance. And finally, two years ago, we shipped our first system! Shortly thereafter, more milestones of the variety you can only get after shipping: our first update of the software in the field; our first update-delivered performance improvements; our first customer-requested features added as part of an update. Later that year, we hit general commercial availability, and things started accelerating. We had more customers — and our first multi-rack customer. We had customers go on the record about why they had selected Oxide — and customers describing the wins that they had seen deploying Oxide. Customers starting landing faster now: enterprise sales cycles are infamously long, but we were finding that we were going from first conversations to a delivered product surprisingly quickly. The quickening pace always seemed to be due in some way to our transparency: new customers were listeners to our podcast, or they had read our RFDs, or they had perused our documentation, or they had looked at the source code itself. With growing customer enthusiasm, we were increasingly getting questions about what it would look like to buy a large number of Oxide racks. Could we manufacture them? Could we support them? Could we make them easy to operate together? Into this excitement, a new potential investor, USIT, got to know us. They asked terrific questions, and we found a shared disposition towards building lasting value and doing it the right way. We learned more about them, too, and especially USIT’s founder, Thomas Tull. The more we each learned about the other, the more there was to like. And importantly, USIT had the vision for us that we had for ourselves: that there was a big, important market here — and that it was uniquely served by Oxide. We are elated to announce this new, exciting phase of the company. It’s not necessarily in our nature to celebrate fundraising, but this is a big milestone, because it will allow us to address our customers' most pressing questions around scale (manufacturing scale, system scale, operations scale) and roadmap scope. We have always believed in our mission, but this raise gives us a new sense of confidence when we say it: we’re going to kick butt, have fun, not cheat (of course!), love our customers — and change computing forever.