More from Louwrentius - Solar
Skip to the bottom two paragraph for instructions on how to replace the battery. I bought my Bose SoundLink on-ear Bluetooth headphones for 250 Euros around 2017 and I really like them. They are small, light, comfortable and can easily fit in a coat pocket when folded. Up until now (about 7 years later) I have replaced the ear cushions in 2019 (€25) and 2024 (€18). Early 2025, battery capacity had deteriorated to a point where it became noticeable. The battery was clearly dying. Unfortunately these headphones aren't designed for easy battery replacement: Bose hasn't published instructions on how to replace the battery, doesn't offer a replacement battery and hasn't documented which battery type/model is used. The left 'head phone' has two Torx security screws and most people won't have the appropriate screwdriver for this size There is soldering involved I wanted to try a battery replacement anyway as I hate to throw away a perfectly good, working product just because the battery has worn out. Maybe at some point the headband needs replacing, but with a fresh battery, these headphones can last another 7 years. Let's prevent a bit of e-waste with a little bit of cost and effort. Most of all, the cost of this battery replacement is much lower than a new pair of headphones as the battery was €18 including taxes and shipping. Right to repair should include easy battery replacement Although my repair seemed to have worked out fine, it requires enough effort that most people won't even try. For this reason, I feel that it should be mandatory by law that: Batteries in any product must be user-replaceable (no special equipment or soldering required) Batteries must be provided by the vendor until 10 years after the last day the product was sold (unless it's a standard format like AA(A) or 18650). Batteries must be provided at max 10% of the cost of the original product The penalty for non-compliance should be high enough such that it won't be regarded as the cost of doing business For that matter, all components that may wear down over time should be user-replaceable. What you need to replace the battery Buy the exact battery type: ahb571935pct-01 (350mAh) (notice the three wires!) A Philips #0 screwdriver / bit A Torx T6H security screwdriver / bit (iFixit kits have them) A soldering iron Solder Heat shrink for 'very thin wire' Multimeter (optional) a bit of tape to 'cap off' bare battery leads Please note that I found another battery ahb571935pct-03 with similar specifications (capacity and voltage) but I don't know if it will fit. Putting the headphone ear cushion back on can actually be the hardest part of the process, you need to be firm and this process is documented by Bose. Battery replacement steps I took Make sure you don't short the wires on the old or new battery during replacement The battery is located in the left 'head phone'. Use a multimeter to check if your new battery isn't dead (should be 3+ volt) Remove the ear cushion from the left 'head phone' very gently as not to tear the rim Remove the two philips screws that keep the driver (speaker) in place Remove the two Torx screws (you may have to press a bit harder) Remove the speaker and be carefull not to snap the wire Gently remove the battery from the 'head phone' Cut the wires close to the old battery (one by one!) and cover the wires on the battery to prevent a short Strip the three wires from the headphones a tiny bit (just a few mm) Put a short piece of heat shrink on each of the three wires of the battery Solder each wire to the correct wire in the ear cup Adjust the location of the heat shrink over the freshly soldered joint. Use the soldering iron close to the heat shrink to shrink it (don't touch anything), this can take some time, be patient Check that the heat shrink is fixed in place and can't move Put the battery into it's specific location in the back of the 'head phone' Test the headphones briefly before reassembling the headphones Reassemble the 'head phone' (consider leaving out the two Torx screws) Dispose of the old battery in a responsible manner
My 4U 71 TiB ZFS NAS built with twenty-four 4 TB drives is over 10 years old and still going strong. Although now on its second motherboard and power supply, the system has yet to experience a single drive failure (knock on wood). Zero drive failures in ten years, how is that possible? Let's talk about the drives first The 4 TB HGST drives have roughly 6000 hours on them after ten years. You might think something's off and you'd be right. That's only about 250 days worth of runtime. And therein lies the secret of drive longevity (I think): Turn the server off when you're not using it. According to people on Hacker News I have my bearings wrong. The chance of having zero drive failures over 10 years for 24 drives is much higher than I thought it was. So this good result may not be related to turning my NAS off and keeping it off most off the time. My NAS is turned off by default. I only turn it on (remotely) when I need to use it. I use a script to turn the IoT power bar on and once the BMC (Baseboard Management Controller) is done booting, I use IPMI to turn on the NAS itself. But I could have used Wake-on-Lan too as an alternative. Once I'm done using the server, I run a small script that turns the server off, wait a few seconds and then turn the wall socket off. It wasn't enough for me to just turn off the server, but leave the motherboard, and thus the BMC powered, because that's just a constant 7 watts (about two Raspberry Pis at idle) being wasted (24/7). This process works for me because I run other services on low-power devices such as Raspberry Pi4s or servers that use much less power when idling than my 'big' NAS. This proces reduces my energy bill considerably (primary motivation) and also seems great for hard drive longevity. Although zero drive failures to date is awesome, N=24 is not very representative and I could just be very lucky. Yet, it was the same story with the predecessor of this NAS, a machine with 20 drives (1 TB Samsung Spinpoint F1s (remember those?)) and I also had zero drive failures during its operational lifespan (~5 years). The motherboard (died once) Although the drives are still ok, I had to replace the motherboard a few years ago. The failure mode of the motherboard was interesting: it was impossible to get into the BIOS and it would occasionally fail to boot. I tried the obvious like removing the CMOS battery and such but to no avail. Fortunately, the [motherboard]1 was still available on Ebay for a decent price so that ended up not being a big deal. ZFS ZFS worked fine for all these years. I've switched operating systems over the years and I never had an issue importing the pool back into the new OS install. If I would build a new storage server, I would definitely use ZFS again. I run a zpool scrub on the drives a few times a year2. The scrub has never found a single checksum error. I must have run so many scrubs, more than a petabyte of data must have been read from the drives (all drives combined) and ZFS didn't have to kick in. I'm not surprised by this result at all. Drives tend to fail most often in two modes: Total failure, drive isn't even detected Bad sectors (read or write failures) There is a third failure mode, but it's extremely rare: silent data corruption. Silent data corruption is 'silent' because a disk isn't aware it delivered corrupted data. Or the SATA connection didn't detect any checksum errors. However, due to all the low-level checksumming, this risk is extremely small. It's a real risk, don't get me wrong, but it's a small risk. To me, it's a risk you mostly care about at scale, in datacenters4 but for residential usage, it's totally reasonable to accept the risk3. But ZFS is not that difficult to learn and if you are well-versed in Linux or FreeBSD, it's absolutely worth checking out. Just remember! Sound levels (It's Oh So Quiet) This NAS is very quiet for a NAS (video with audio). But to get there, I had to do some work. The chassis contains three sturdy 12V fans that cool the 24 drive cages. These fans are extremely loud if they run at their default speed. But because they are so beefy, they are fairly quiet when they run at idle RPM5, yet they still provide enough airflow, most of the time. But running at idle speeds was not enough as the drives would heat up eventually, especially when they are being read from / written to. Fortunately, the particular Supermicro motherboard I bought at the time allows all fan headers to be controlled through Linux. So I decided to create a script that sets the fan speed according to the temperature of the hottest drive in the chassis. I actually visited a math-related subreddit and asked for an algorithm that would best fit my need to create a silent setup and also keep the drives cool. Somebody recommended to use a "PID controller", which I knew nothing about. So I wrote some Python, stole some example Python PID controller code, and tweaked the parameters to find a balance between sound and cooling performance. The script has worked very well over the years and kept the drives at 40C or below. PID controllers are awesome and I feel it should be used in much more equipment that controls fans, temperature, and so on, instead of 'dumb' on/of behaviour or less 'dumb' lookup tables. Networking I started out with quad-port gigabit network controllers and I used network bonding to get around 450 MB/s network transfer speeds between various systems. This setup required a ton of UTP cables so eventually I got bored with that and I bought some cheap Infiniband cards and that worked fine, I could reach around 700 MB/s between systems. As I decided to move away from Ubuntu and back to Debian, I faced a problem: the Infiniband cards didn't work anymore and I could not figure out how to fix it. So I decided to buy some second-hand 10Gbit Ethernet cards and those work totally fine to this day. The dead power supply When you turn this system on, all drives spin up at once (no staggered spinup) and that draws around 600W for a few seconds. I remember that the power supply was rated for 750W and the 12 volt rail would have been able to deliver enough power, but it would sometimes cut out at boot nonetheless. UPS (or lack thereof) For many years, I used a beefy UPS with the system, to protect against power failure, just to be able to shutdown cleanly during an outage. This worked fine, but I noticed that the UPS used another 10+ watts on top of the usage of the server and I decided it had to go. Losing the system due to power shenanigans is a risk I accept. Backups (or a lack thereof) My most important data is backed up trice. But a lot of data stored on this server isn't important enough for me to backup. I rely on replacement hardware and ZFS protecting against data loss due to drive failure. And if that's not enough, I'm out of luck. I've accepted that risk for 10 years. Maybe one day my luck will run out, but until then, I enjoy what I have. Future storage plans (or lack thereof) To be frank, I don't have any. I built this server back in the day because I didn't want to shuffle data around due to storage space constraints and I still have ample space left. I have a spare motherboard, CPU, Memory and a spare HBA card so I'm quite likely able to revive the system if something breaks. As hard drive sizes have increased tremendously, I may eventually move away from the 24-drive bay chassis into a smaller form-factor. It's possible to create the same amount of redundant storage space with only 6-8 hard drives with RAIDZ2 (RAID 6) redundancy. Yet, storage is always expensive. But another likely scenario is that in the coming years this system eventually dies and I decide not to replace it at all, and my storage hobby will come to an end. I needed the same board, because the server uses four PCIe slots: 3 x HBA and 1 x 10Gbit NIC. ↩ It takes ~20 hours to complete a scrub and it uses a ton of power while doing so. As I'm on a dynamic power tariff, I run it on 'cheap' days. ↩ every time I listen to ZFS enthusiasts you get the impression you are taking insane risks with your data if you don't run ZFS. I disagree, it all depends on context and circumstances. ↩ enterprise hard drives used in servers and SANs had larger sector sizes to accommodate even more checksumming data to prevent against silent data corruption. ↩ Because there is little airflow by default, I had to add a fan to cool the four PCIe cards (HBA and networking) or they would have gotten way too hot. ↩
I've always been fond of the idea of the Raspberry Pi. An energy efficient, small, cheap but capable computer. An ideal home server. Until the Pi 4, the Pi was not that capable, and only with the relatively recent Pi 5 (fall 2023) do I feel the Pi is OK performance wise, although still hampered by SD card performance1. And the Pi isn't that cheap either. The Pi 5 can be fitted with an NVME SSD, but for me it's too little, too late. Because I feel there is a type of computer on the market, that is much more compelling than the Pi. I'm talking about the tinyminimicro home lab 'revolution' started by servethehome.com about four years ago (2020). A 1L mini PC (Elitedesk 705 G4) with a Raspberry Pi 5 on top During the pandemic, the Raspberry Pi was in short supply and people started looking for alternatives. The people at servethehome realised that these small enterprise desktop PCs could be a good option. Dell (micro), Lenovo (tiny) and HP (mini) all make these small desktop PCs, which are also known as 1L (one liter) PCs. These Mini PC are not cheap2 when bought new, but older models are sold at a very steep discount as enterprises offload old models by the thousands on the second hand market (through intermediates). Although these computers are often several years old, they are still much faster than a Raspberry Pi (including the Pi 5) and can hold more RAM. I decided to buy two HP Elitedesk Mini PCs to try them out, one based on AMD and the other based on Intel. The Hardware Elitedesk Mini G3 800 Elitedesk Mini G4 705 CPU Intel i5-6500 (65W) AMD Ryzen 3 PRO 2200GE (35W) RAM 16 GB (max 32 GB) 16 GB (max 32 GB) HDD 250 GB (SSD) 250 GB (NVME) Network 1Gb (Intel) 1Gb (Realtek) WiFi Not installed Not installed Display 2 x DP, 1 x VGA 3 x DP Remote management Yes No Idle power 4 W 10 W Price €160 €115 The AMD-based system is cheaper, but you 'pay' in higher idle power usage. In absolute terms 10 watt is still decent, but the Intel model directly competes with the Pi 5 on idle power consumption. Elitedesk 705 left, Elitedesk 800 right (click to enlarge) Regarding display output, these devices have two fixed displayport outputs, but there is one port that is configurable. It can be displayport, VGA or HDMI. Depending on the supplier you may be able to configure this option, or you can buy them separately for €15-€25 online. Click on image for official specs in PDF format Both models seem to be equipped with socketed CPUs. Although options for this formfactor are limited, it's possible to upgrade. Comparing cost with the Pi 5 The Raspberry Pi 5 with (max) 8 GB of RAM costs ~91 Euro, almost exactly the same price as the AMD-based mini PC3 in its base configuration (8GB RAM). Yet, with the Pi, you still need: power supply (€13) case (€11) SD card or NVME SSD (€10-€45) NVME hat (€15) (optional but would be more comparable) It's true that I'm comparing a new computer to a second hand device, and you can decide if that matters in this case. With a complete Pi 5 at around €160 including taxes and shipping, the AMD-based 1L PC is clearly the cheaper and still more capable option. Comparing performance with the Pi 5 The first two rows in this table show the Geekbench 6 score of the Intel and AMD mini PCs I've bought for evaluation. I've added the benchmark results of some other computers I've access to, just to provide some context. CPU Single-core Multi-core AMD Ryzen 3 PRO 2200GE (32W) 1148 3343 Intel i5-6500 (65W) 1307 3702 Mac Mini M2 2677 9984 Mac Mini i3-8100B 1250 3824 HP Microserver Gen8 Xeon E3-1200v2 744 2595 Raspberry Pi 5 806 1861 Intel i9-13900k 2938 21413 Intel E5-2680 v2 558 5859 Sure, these mini PCs won't come close to modern hardware like the Apple M2 or the intel i9. But if we look at the performance of the mini PCs we can observe that: The Intel i5-6500T CPU is 13% faster in single-core than the AMD Ryzen 3 PRO Both the Intel and AMD processors are 42% - 62% faster than the Pi 5 regarding single-core performance. Storage (performance) If there's one thing that really holds the Pi back, it's the SD card storage. If you buy a decent SD card (A1/A2) that doesn't have terrible random IOPs performance, you realise that you can get a SATA or NVME SSD for almost the same price that has more capacity and much better (random) IO performance. With the Pi 5, NVME SSD storage isn't standard and requires an extra hat. I feel that the missing integrated NVME storage option for the Pi 5 is a missed opportunity that - in my view - hurts the Pi 5. Now in contrast, the Intel-based mini PC came with a SATA SSD in a special mounting bracket. That bracket also contained a small fan(1) to keep the underlying NVME storage (not present) cooled. There is a fan under the SATA SSD (click to enlarge) The AMD-based mini PC was equipped with an NVME SSD and was not equipped with the SSD mounting bracket. The low price must come from somewhere... However, both systems have support for SATA SSD storage, an 80mm NVME SSD and a small 2230 slot for a WiFi card. There seems no room on the 705 G4 to put in a small SSD, but there are adapters available that convert the WiFi slot to a slot usable for an extra NVME SSD, which might be an option for the 800 G3. Noice levels (subjective) Both systems are barely audible at idle, but you will notice them (if you sensitive to that sort of thing). The AMD system seems to become quite loud under full load. The Intel system also became loud under full load, but much more like a Mac Mini: the noise is less loud and more tolerable in my view. Idle power consumption Elitedesk 800 (Intel) I can get the Intel-based Elitedesk 800 G3 to 3.5 watt at idle. Let that sink in for a moment. That's about the same power draw as the Raspberry Pi 5 at idle! Just installing Debian 12 instead of Windows 10 makes the idle power consumption drop from 10-11 watt to around 7 watt. Then on Debian, you: run apt install powertop run powertop --auto-tune (saves ~2 Watt) Unplug the monitor (run headless) (saves ~1 Watt) You have to put the powertop --auto-tune command in /etc/rc.local: #!/usr/bin/env bash powertop --auto-tune exit 0 Then apply chmod +x /etc/rc.local So, for about the same idle power draw you get so much more performance, and go beyond the max 8GB RAM of the Pi 5. Elitedesk 705 (AMD) I managed to get this system to 10-11 watt at idle, but it was a pain to get there. I measured around 11 Watts idle power consumption running a preinstalled Windows 11 (with monitor connected). After installing Debian 12 the system used 18 Watts at idle and so began a journey of many hours trying to solve this problem. The culprit is the integrated Radeon Vega GPU. To solve the problem you have to: Configure the 'bios' to only use UEFI Reinstall Debian 12 using UEFI install the appropriate firmware with apt install firmware-amd-graphics If you boot the computer using legacy 'bios' mode, the AMD Radeon firmware won't load no matter what you try. You can see this by issuing the commands: rmmod amdgpu modprobe amdgpu You may notice errors on the physical console or in the logs that the GPU driver isn't loaded because it's missing firmware (a lie). This whole process got me to around 12 Watt at idle. To get to ~10 Watts idle you need to do also run powertop --auto-tune and disconnect the monitor, as stated in the 'Intel' section earlier. Given the whole picture, 10-11 Watt at idle is perfectly okay for a home server, and if you just want the cheapest option possible, this is still a fine system. KVM Virtualisation I'm running vanilla KVM (Debian 12) on these Mini PCs and it works totally fine. I've created multiple virtual machines without issue and performance seemed perfectly adequate. Boot performance From the moment I pressed the power button to SSH connecting, it took 17 seconds for the Elitedesk 800. The Elitedesk 705 took 33 seconds until I got an SSH shell. These boot times include the 5 second boot delay within the GRUB bootloader screen that is default for Debian 12. Remote management support Some of you may be familiar with IPMI (ILO, DRAC, and so on) which is standard on most servers. But there is also similar technology for (enterprise) desktops. Intel AMT/ME is a technology used for remote out-of-band management of computers. It can be an interesting feature in a homelab environment but I have no need for it. If you want to try it, you can follow this guide. For most people, it may be best to disable the AMT/ME feature as it has a history of security vulnerabilities. This may not be a huge issue within a trusted home network, but you have been warned. The AMD-based Elitedesk 705 didn't came with equivalent remote management capabilities as far as I can tell. Alternatives The models discussed here are older models that are selected for a particular price point. Newer models from Lenovo, HP and Dell, equip more modern processors which are faster and have more cores. They are often also priced significantly higher. If you are looking for low-power small formfactor PCs with more potent or customisable hardware, you may want to look at second-hand NUC formfactor PCs. Stacking multiple mini PCs The AMD-based Elitedesk 705 G4 is closed at the top and it's possible to stack other mini PCs on top. The Intel-based Elitedesk 800 G3 has a perforated top enclosure, and putting another mini pc on top might suffocate the CPU fan. As you can see, the bottom/foot of the mini PC doubles as a VESA mount and has four screw holes. By putting some screws in those holes, you may effectively create standoffs that gives the machine below enough space to breathe (maybe you can use actual standoffs). Evaluation and conclusion I think these second-hand 1L tinyminimicro PCs are better suited to play the role of home (lab) server than the Raspberry Pi (5). The increased CPU performance, the built-in SSD/NVME support, the option to go beyond 8 GB of RAM (up to 32GB) and the price point on the second-hand market really makes a difference. I love the Raspberry Pi and I still have a ton of Pi 4s. This solar-powered blog is hosted on a Pi 4 because of the low power consumption and the availability of GPIO pins for the solar status display. That said, unless the Raspberry Pi becomes a lot cheaper (and more potent), I'm not so sure it's such a compelling home server. This blog post featured on the front page of Hacker News. even a decent quality SD card is no match (in terms of random IOPs and sequential throughput) for a regular SATA or NVME SSD. The fact that the Pi 5 has no on-board NVME support is a huge shortcomming in my view. ↩ in the sense that you can buy a ton of fully decked out Pi 5s for the price of one such system. ↩ The base price included the external power brick and 256GB NVME storage. ↩
Before Chat-GPT caused a sensation, big tech companies like Facebook and Apple were betting their future growth on virtual reality. But I'm convinced that virtual reality will never be a mainstream thing. If you ever used VR you know why: A heavy thing on your head that messes up your hair Nausea The focus on virtual reality felt like desperation to me. The desperation of big tech companies trying to find new growth, ideally a monopoly they control1, to satisfy the demands of shareholders. And then OpenAI dropped ChatGPT and all the big tech companies started to pivot so fast because in contrary to VR, AI doesn't involve making people nauseated and look silly. It's probably obvious that I feel it's not about AI itself. It is really about huge tech companies that have found a new way to sustain growth a bit longer, now that all other markets have been saturated. Flush with cash, they went nuts and bought up all the AI accelerator hardware2, which in turn uses unspeakable amounts of energy to train new large language models. Despite all the hype, current AI technology is at it's core a very sophisticated statistical model. It's all about probabilities, it can't actually reason. As I see it, work done by AI can't thus be trusted. Depending on the specific application, that may be less of an issue, but that is a fundamental limitation of current technology. And this gives me pause as it limits the application where it is most wanted: to control labour. To reduce the cost of headcount and to suppress wages. As AI tools become capable enough, it would be irresponsible towards shareholders not to pursue this direction. All this just to illustrate that the real value of AI is not for the average person in the street. The true value is for those bigger companies who can keep on growing, and the rest is just collateral damage. But I wonder: when the AI hype is over, what new hype will take it's place? I can't see it. I can't think of it. But I recognise that the internet created efficiencies that are convenient, yet social media weaponised this convenience to exploit our fundamental human weaknesses. As shareholder value rose, social media slowly chips away at the fabric of our society: trust. I've sold my Oculus Rift CV1 long ago, I lost hundreds of dollars of content but I refuse to create a Facebook/Meta account. ↩ climate change accelerators ↩
Introduction Victron Multiplus-II inverter/charges are configured with the veconfigure1 tool. Unforntunately this is a Windows-only tool, but there is still a way for Apple users to run this tool without any problems. Tip: if you've never worked with the Terminal app on MacOS, it might not be an easy process, but I've done my best to make it as simple as I can. A tool called 'Wine' makes it possible to run Windows applications on MacOS. There are some caveats, but none of those apply to veconfigure, this tool runs great! I won't cover in this tutorial how to make the MK-3 USB cable work. This tutorial is only meant for people who have a Cerbo GX or similar device, or run VenusOS, which can be used to remotely configure the Multipluss device(s). Step 1: install brew on macos Brew is a tool that can install additional software Visit https://brew.sh and copy the install command open the Terminal app on your mac and paste the command now press 'Enter' or return It can take a few minutes for 'brew' to install. Step 2: install wine Enter the following two commands in the terminal: brew tap homebrew/cask-versions brew install --cask --no-quarantine wine-stable Download Victron veconfigure Visit this page Scroll to the section "VE Configuration tools for VE.Bus Products" Click on the link "Ve Configuration Tools" You'll be asked if it's OK to download this file (VECSetup_B.exe) which is ok Start the veconfigure installer with wine Open a terminal window Run cd Enter the command wine Downloads\VECSetup_B.exe Observe that the veconfigure Windows setup installer starts Click on next, next, install and Finish veconfigure will run for the first time Click on the top left button on the video to enlarge These are the actual install steps: How to start veconfigure after you close the app Open a terminal window Run cd Run cd .wine/drive_c/Program\ Files\ \(x86\)/VE\ Configure\ tools/ Run wine VEConfig.exe Observe that veconfigure starts Allow veconfigure access to files in your Mac Download folder Open a terminal window Run cd run cd .wine/drive_c/ run ls -n ~/Downloads We just made the Downloads directory on your Mac accessible for the vedirect software. If you put the .RSVC files in the Downloads folder, you can edit them. Please follow the instructions for remote configuration of the Multiplus II. Click on the "Ve Configuration Tools" link in the "VE Configuration tools for VE.Bus Products" section. ↩
More in technology
Sharing a few data points because some people did not expect these drives to last that long.
I'm 100% for banning social media for children. I would even extend it to banning it all together in its current form. But I am also against the methods for enforcement. The problem with banning social media for one group and not another is that you have to differentiate the different groups somehow. And right now, in an age where information and personal data means everything, there are several entities that are salivating to get this data. I've written before about all the problems that come from uploading sensitive data online. It doesn't even matter if they delete the data afterward, as long as the data goes through the wire in the first place, there is a chance for a leak. What appears to be the same simple request "please verify your identity", becomes fundamentally different when mediated by technology. The question isn't whether these digital systems will be compromised, but when. And unlike that movie theater clerk who can't perfectly recall my birthdate minutes after seeing it, computers have perfect memory. They store, copy, backup, and transmit our most sensitive information through networks we don't control, to companies we've never heard of, under policies we'll never read. When kids are not allowed to use social media, it's the adult who pays the price. It's the adult who has to prove that they are not children. They have to risk uploading their government ID to a third party, or biometric data that may be saved and backed up forever. I reject this method. I also reject the method where I have to provide my age to the operating system. No system should compel me to provide personal data in order to use it. Especially if this feature is intended to provide the information to a third party. And what happens if I don't provide this information to my freebsd machine? Or worse, what if I provide false information? Am I going to get punished? How does that protect children in any way? Breaking news: Grown up arrested when OS reported forged identity in his laptop. One question I asked myself was what happens when a minor circumvents the ban. What are the consequences? At least in Australia, a country that has implemented a social media ban for those under 16, there are no consequences. The law places legal responsibility on platforms, not children or parents, and it currently applies to Facebook, Instagram, Kick, Reddit, Snapchat, Threads, TikTok, Twitch, X and YouTube. This almost sounds reasonable. Why should you care, if it becomes facebook responsibility when you don't even own a facebook account? But there is always a caveat. When facebook becomes a tool your school uses to communicate with parents. When Google classroom is the service students use to do their homework. When you are required to login on a system to receive government service. You may not care about a particular social media, but they raise the bar for every other entrant in the game. Every website frequented by a large number of people inherently become social media, and demand data you didn't need to provide before. Technology is not a magical thing that can just solve every problem. It is especially weak when it comes to social problems. However, society can use technology to solve their problems. The responsibility of children falls onto their parents. And there are great tools for parents that their kids access on their devices.
Introduction Hamming Distance Intuitive Estimation of Error Correction Bits Digging Deeper Comparing against BCH Codes What about Reed-Solomon Codes? BCH vs Reed-Solomon Codes References Footnotes Introduction I’ve been interested in error correcting codes (ECC) for a while now, and though I haven’t gotten very far, I keep on reading and, occasionally, writing about it. Except for really simple schemes, ECC theory gets complicated quickly. The two codes that I’ve looked at most are Reed-Solomon (RS) and binary Bose–Chaudhuri–Hocquenghem (BCH) codes. They are closely related: both use polynomial math that is normally performed over a Galois Field, though, as shown in my earlier Reed-Solomon blog post, that’s not strictly required. The encoding algorithms are relatively straightforward but the decoding algorithms are significantly more complicated. RS and binary BCH codes differ in the size of their symbols, their alphabet: a symbol of the former consists of multiple bits, often 8 bits, while a symbol of the latter is a bit. Errors are corrected with a granularity of a symbol. One key ECC question is “How many additional bits do I need to correct a given number of errors?” In this blog post, I check if an intuitive approach that I’ve been using matches reality. I’m trying hard to stay away from theory here, which makes the subject hopefully more accessible, but has the flip side that it’s far from being rigorous and exact. Hamming Distance When you look at the theory of error detecting and correcting codes, the discussion immediately turns to Hamming distance: the number of bit flips required to convert one vector of bits to another. The goal of an error correcting code is to transform the input vector, the message, to an output vector, the codeword, so that the minimum Hamming distance between valid codewords is guaranteed to be at least a certain number. The higher the Hamming distance, the higher number of bits that can be flipped in a codeword without destroying the ability to recover the original message. If \(t\)1 is the number of errors that we want to correct, error coding theory gives us the following formula: In words: to correct a codeword with a maximum number of \(t\) bit errors, the Hamming distance between any 2 codewords must be larger than two times \(t\). The theory also tells us the minimum Hamming distance to detect an error without correcting it: In the rest of this blog post, though, we’ll only care about correction, not detection. Here are 2 quick examples where the message is one bit. Adding 1 bit to the message For a one bit message, there are only 2 possible messages, 0 and 1. Let’s use a code that repeats this bit once: 0 becomes 00 and 1 becomes 11. The Hamming distance between these 2 valid codewords is 2 because you need to flip 2 bits to convert 00 into 11. The equation above tells us that a distance of 2 is not sufficient to fix a one-bit error. Indeed, when we receive 01 as a codeword, it’s impossible to tell if the transmitted codeword was 00 or 11: both have a Hamming distance of 1 to the received word of 01. Adding 2 bits to the message When we expand the 1 bit message to 3 bits and repeat the message bit twice, things are different: 0 transforms into 000 and 1 into 111. The Hamming distance between all possible codewords is 3. The theory says that we should be able to correct a codeword for which only 1 bit was flipped. And indeed, if we receive 001 and there is only 1 bit corrupted, we know for sure that the original message was 0. Intuitive Estimation of Error Correction Bits The concept of a minimum required Hamming distance to correct a number of errors is not particularly complicated, but it’s a different matter to use it to determine the number of additional ECC bits for, say, a BCH code. In fact, I still don’t know how to do it. As an alternative, I found myself using a simple model that works like this: Participant A transmits 1000 bits to participant B over a noisy channel B transmits the same 1000 bits back to A over a perfect channel A transmits a 10 bit error index to signal to B which bit it received wrong Or more general: If a message of \(k\) bits has 1 incorrect bit, you need a number of \(r = \lceil \log_2(k) \rceil\) 2 bits to point out the incorrect bit. Let’s call that number the error index3. To correct \(t\) bits, you need up to \(r = t \cdot \lceil \log_2(k) \rceil\) additional bits. That’s it! The model is simple, it’s not exactly practical, and the number of parity bits has to be only a first order estimation, but I felt like it had to be pretty close to reality. But is it? Digging Deeper Let’s consider some of the aspects of this model: There is no way to signal that there was no corruption That can be solved by allocating one error index value extra to signal a perfect transmission and calculate the number of bits as \(r = \lceil \log_2(k+1) \rceil\) instead of \(r = \lceil \log_2(k) \rceil\). E.g. for a 1000-bit message, an error index from 0 to 999 indicates an error location, and 1000 signals perfect transmission. For messages that are not a power of 2, the number of error indicator bits remains the same. The transmitted error indicator bits can be corrupt as well One way to solve this is by going through the same procedure to verify the error index. For a first iteration, it would cost an additional \(\lceil \log_2(\lceil \log_2(k+1) \rceil ) \rceil\) bits, 4 bits for a 10 bit error index. Of course, those 4 bits might be corrupt too! A better approach is to work backwards: start out with the length of the codeword and the number of errors you want to correct and determine the number of input message bits from there. The number of error indicator bits is then calculated over the length of the codeword and you get \(r = \lceil \log_2(n+1) \rceil\). The length of the message is \(k = n - r\). This breaks the simple model where you transmit message and error indicator separately, but that was not a realistic scenario anyway. It doesn’t consider that the same error index can appear only once If the code can correct 2 errors, you’d need \(\lceil \log_2(k+1) \rceil\) bits for the first error index to indicate zero errors or the location of the first error. For the second error, you need one less value because one location has already been corrected, so the number of bits needed is \(\lceil \log_2(k) \rceil\). This suggests that the number of error indicator bits might be an upper bound. It only applies to error correction the way BCH does it Any random bit can be corrected, but it doesn’t work for RS coding because there a whole symbol can be incorrect. That’s fixable though, I get to that later. You don’t need to signal the correct value Since the symbol is 1 bit in size, the location of the error is sufficient. To fix it, the receiver just needs to invert the bit. This also doesn’t work for RS. Comparing against BCH Codes Forget about using the Wikipedia article to compare the intuition against theory: you won’t find anything explicit. I found this presentation from the National Taipei University useful. The key formulas are on the first slide: Block length: \(n = 2^m -1\) Number of parity-check bits: \(n - k \le m t\) Minimum distance: \(d_{min} \ge 2t + 1\)4 In the equations above, \(k\) is still the length of the original message, \(t\) is the number of bits that can be corrected, \(n\) is the length of the codeword. If we rearrange the first equation, we get: Substitute \(m\) in the second equation: \(r\) is the number of error correction bits, so \(r = n - k\) and: Compare this with the intuitive formula that I started with: The only differences are: I used \(k\) instead of \(n+1\). That accounts for the fact that parity bits themselves need error correction too and that a perfect transmission must be signaled. the BCH equation assumes that the total codeword length is always one less than a power of two. instead of an approximation, it defines an upper bound. Theoretically, all BCH and RS codewords have a length \(n = 2^m -1\) symbols. In practice, the transmitted codewords are often shortened by setting a bunch of the symbols in the input message to zero and not transmitting them.5 When using shortened codes, you still have to use the \(m\) value of the parent code for which \(n = 2^m -1\). The intuitive formula works for long codewords and a limited number of errors to correct: \(n \approx k\). What about Reed-Solomon Codes? RS codes correct symbols not bits but you can still use the intuitive formula to identify the corrupted symbol: in a message with, say, 223 bytes you’d need 8 bits to pinpoint an incorrect byte. However, you also need an additional byte to fix the corruption and that is the case for every error byte that needs to be corrected. To repair \(t\) symbols that are \(s\) bits in size in a message of \(k\) symbols, you’d need this number of bits for the error location: And a bunch of bits with the correct values: Note that \(r_{location}\) points to a symbol, not a bit. Just like BCH codes, \(n = 2^s - 1\) for RS codes. Change \(r_{location}\) to use \(n\) instead of \(k\) and you get: The final result is: It turns out, that is exactly what you’ll find in all texts about RS coding: the number of parity symbols must be at least 2 times the number of corrupt symbols you want to correct. BCH vs Reed-Solomon Codes I’m getting a bit off-topic here but with the intuitive formula validated, it’s interesting to see when RS codes are better than BCH and vice versa. Let’s create an example where codewords are 255 bytes or 2040 bits and where we reserve 32 bytes or 256 bits for error correction. For RS, this allows us to correct 16 corrupt bytes that can contain up to 128 corrupt bits, as long as they’re contained within those 16 bytes. For BCH, we need 11 bits per error or \(11 \cdot 23 = 253\) parity bits to correct 23 corrupt bits which fits within the 256 bits that were allocated for RS. RS is the winner if errors happen in bursts: it doesn’t matter if a burst is 1 or 8 bits long, it counts as a single corruption when they fall within the same byte. BCH wins when errors are spread randomly: instead of fixing 16 bit flips, it can fix 23. In practice, communication protocols often used 2 codes in the following scheme: Transmitter: RS Encoder -> Interleaving -> Convolutional Encoder Receiver: Viterbi Decoder -> De-interleaving -> RS Decoder In this setup, RS is called an outer code and convolutional coding the inner code, because they wrap around the transmission in that order. In many transmission mediums, bit error are uncorrelated and burst errors unlikely. One would think that RS coding is the wrong choice then. However, the Viterbi decoder can introduce bursts of errors when it makes a mistake. The RS decoder is used to clean those up. The burst error length can be so long that a single RS codeword can’t deal with them. That’s why there’s an interleaver to increase the length of burst errors that can be corrected, at the expense of latency and memory storage, but that’s a topic for a different blog post. Modern communication protocols, such as DVB-S2 that’s used by satellite TV, use a low density parity check (LDPC) code instead of a convoluational code as inner code. A LDPC decoder has a lower tendency to introduce long error bursts in case of a mistake and instead introduces single bit errors that are spread randomly within a large codeword. This is where BCH codes shine which is why it’s used instead of RS. All words in this blog posts were written by a human. References Reed-Solomon Error Correcting Codes from the Bottom Up National Taipei University - BCH code CMU Introduction to Coding Theory - Notes 4: Elementary bounds on codes NASA - Bandwidth efficient CCSDS coding standard proposals Discusses outer coding/interleaving/inner coding for space communication. There’s an abundance of information in there, which is what you’d expect from a 70 page document! Footnotes In literature, you’ll often see \(k\) being used for this formula, but further down, \(k\) is used as the length of the original message, which is also a literature convention. So I decided to be consistent within this blog post. ↩ \(\lceil x \rceil\) means: round up to nearest integer. ↩ I came up with “error index” myself, but it turns out to be a term that is commonly used in error coding theory and its meaning is exactly what I had in mind! ↩ This is the minimum Hamming distance equation for error correction. ↩ This assumes that a systematic coding scheme is used, where the original message appears unmodified in the codeword. This is the case for most BCH and RS coding schemes that are used in the real world. ↩
Plus Thomas Ableman on what we can learn about AI from the Victorian railway rollout
Water-resistant microneedles made from food-grade silk could deliver vaccines to help prevent the spread of disease in aquaculture.