Full Width 
Shortcuts 
Sign Up 
Log In 
Articles 
Reading List More from heavymeta.org
Just after midnight Eastern Time on July 19, 2024, the enterprise cybersecurity company CrowdStrike YOLOed a software update to millions of Windows machines. Or as they put it: On July 19, 2024 at 04:09 UTC, as part of ongoing operations, CrowdStrike released a sensor configuration update to Windows systems. That sensor configuration update caused the largest IT outage in history. Overnight, about 8.5 million computers blue screened, affecting hospitals, banks, 911 systems–as the New York Times put it, “It is more apt to ask what was not affected.” The answer is Linux, Macs, and phones. The outage highlighted a different kind of digital divide. On one side, gmail, Facebook, and Twitter kept running, letting us post photos of blue screens located on the other side: the Windows machines responsible for actually doing things in the world like making appointments, opening accounts, and dispatching police. They also run airlines. Here’s a visualization of the chaos that CrowdStrike caused for airlines from the New York Times: Airline cancellations is a good metric, but I want to look directly at air traffic: How many planes were in the air? How many planes should have been in the air? At about noon UTC, 8 hours after the CrowdStrike update hit, someone posted a video to Twitter that they made with FlightRadar24 showing air traffic over the United States. It was described as a 12-hour timelapse of American Airlines, Delta, and United plane traffic that showed the nationwide ground stop of the three airlines due to CrowdStrike. Here’s the video: It’s not a good visualization of the impact because there’s no basis for comparison. It clearly shows fewer planes flying at night, but that happens every day. Was that night different from any other night? There’s no way to tell. In Bellingcat’s “OSHIT: Seven Deadly Sins of Bad Open Source Research”, sin #4 is “Lacking Context for Occurrences, Common or Otherwise”. In this post I’ll show the effects CrowdStrike had on air traffic, with enough context to make the significance clear. Impact on U.S. Aviation CrowdStrike hit on July 19. This chart shows the number of aircraft that took off in the United States, hour by hour, on that day. It also shows the same numbers for July 12, the previous Friday. The same day one week previously seems to be a good basis for comparison–both days are Fridays, and there aren’t any major holidays on either day. I also plotted the stats for July 18, the day before CrowdStrike, but it was very similar so I’ll continue to compare to the previous week. Note that the chart is for all of aviation in the United States, including fire fighting aircraft, police, military, and general aviation as well as commercial aviation. From about 0600 to 1300 there seems to have been a small decrease in the number of flights, and then a small increase in the rest of the day. Looking at the cumulative statistics starting from 0400, when the CrowdStrike update was pushed, flights were up 2.6% compared to the same period on the previous Friday. This chart shows the percentage change in flights, comparing each hour on July 19 to the matching hour of the previous Friday as the baseline: This chart brings CrowdStrike’s effects into greater relief. The hour with the largest percent decrease was from 0800 to 0900, which had only 261 flights compared to the previous Friday’s 378 flights, a 31% reduction. Airline Statistics Now let’s look at the statistics for the top 4 U.S. airlines: Delta, United, American, and Southwest. Delta Air Lines Change during CrowdStrike: -1087 flights (-46%) United Airlines Change during CrowdStrike: -596 flights (-36%) American Airlines Change during CrowdStrike: -376 flights (-16%) Southwest Airlines Change during CrowdStrike: +101 flights (+3%) Airlines Summary Delta was hardest hit, then United, and to a significantly smaller degree American. Southwest didn’t seem to be affected at all. Apparently Southwest Airlines’ ingenious strategy of never upgrading from Windows 3.1 allowed it to remain unscathed. This seems to be false, BTW. This ABC News article says that Southwest wasn’t affected because they don’t use CrowdStrike.] Delta Air Lines took an extended time to recover, canceling thousands of flights in the days following the CrowdStrike update. Why were other airlines able to get back to normal so much faster than Delta? A terrible article from ABC News said this: The reason for the prolonged recovery from the outage was because the CrowdStrike update disruption required a manual fix at each individual computer system, experts told ABC News. While each fix can be completed in no more than 10 minutes, the vast number of Delta’s digital terminals required significant manpower to address, expert said. I’m reminded of sin #4 again–How is this different from any other airline? ABC News has no idea. A random redditor gave an unsourced explanation that might be wrong but at least attempts to answer the question “Why Delta so bad?” (DR = disaster recovery): These “experts” are completely wrong. The core issue was Delta did NOT have a proper DR plan ready and did NOT have a proper IT business continuity plan ready. UA, AA, and F9 recovered so fast because they had plans on stand-by and engaged them immediately. After the SWA IT problem, UA and AA put in robust DR plans staged everywhere from the server farms, to cloud solutions, to end-user stations at airports. They had plans on how to recover systems. DL outsources a lot of their IT. UA and AA engaged those plans quickly. They did not hold back paying OT for staff. UA and AA have just as much reliance on Windows as Delta. AA was recovered by end of data Friday and resumed normal operations Saturday. UA was about 12 hours behind them having it resolved by Saturday morning resuming normal schedules Saturday afternoon. The ONUS is 100% on DL C+ level in their IT decisions. Data and Analysis I took raw ADS-B data from ADS-B Exchange and processed it through my custom code to detect aircraft takeoffs. I’m assuming that a takeoff is roughly equivalent to a flight, which isn’t actually true but is close enough for these purposes. It tends to undercount the number of aircraft flying, e.g. in the case where an aircraft took off from a field outside of ADS-B Exchange’s coverage, but it does so in a systematic way that still allows for valid comparisons between time periods. That is, the absolute numbers of flights may be too low, but the percent changes in numbers are accurate. I counted takeoffs instead of counting flying aircraft because I already had code to detect takeoffs and didn’t want to write new code–this was just a quick weekend project.
More in technology
It is a story as old as time (or at least the 1960s): kid gets an RC car for Christmas and excitedly takes it for spin, but crashes it into a wall within five minutes and tears ensue. The automotive industry has cut down on accidents by implementing automatic emergency braking safety features, so why […] The post This automatic emergency braking system protects RC cars appeared first on Arduino Blog.
The shortest distance between your thoughts and the printed word.
It isn’t a secret that many kids find math to be boring and it is easy for them to develop an attitude of “when am I ever going to use this?” But math is incredibly useful in the real world, from blue-collar machinists using trigonometry to quantum physicists unveiling the secrets of our universe through […] The post This unique electronic toy helps children learn their shapes appeared first on Arduino Blog.
I have worked with a few software developers who made the switch to this industry in the middle of their careers. A major change like that can be scary and raise a lot of fears and doubts, but I can attest that this can work out well with the right personality traits and a supporting environment. Here’s what I’ve observed. To keep the writing concise, I’ll be using the phrase “senior junior”1 to describe those that have made such a career switch. Overcoming the fear Fear is a natural reaction to any major change in life, especially when there’s risk of taking a financial hit while you have a family to support and a home loan to pay. The best mitigation that I’ve heard is believing that you can make the change, successfully. It sounds like an oversimplification, sure, as all it does is that it removes a mental blocker and throws out the self-doubt. And yet it works unreasonably well. It also helps if you have at least some savings to help mitigate the financial risk. A years’ worth of expenses saved up can go a long way in providing a solid safety net. What makes them succeed A great software developer is not someone that simply slings some code over the wall and spends all of their day working only on the technical stuff, there are quite a few critical skills that one needs to succeed. This is not an exhaustive list, but I’ve personally observed that the following ones are the most critical: ability to work in a team great communication skills conflict resolution ability to make decisions in the context of product development and business goals maintaining an environment of psychological safety Those with more than a decade of experience in another role or industry will most likely have a lot of these skills covered already, and they can bring that skill set into a software development team while working with the team to build their technical skill set. Software development is not special, at the end of they day, you’re still interacting with humans and everything that comes with that, good or bad. After working with juniors that are fresh out of school and “senior juniors” who have more career experience than I do, I have concluded that the ones that end up being great software developers have one thing in common: the passion and drive to learn everything about the role and the work we do. One highlight that I often like to share in discussions is one software developer who used to work in manufacturing. At some point they got interested in learning how they can use software to make work more efficient. They started with an MVP solution involving a big TV and Google Sheets, then they started learning about web development for a solution in a different area of the business, and ended up building a basic inventory system for the warehouse. After 2-3 years of self-learning outside of work hours and deploying to production in the most literal sense, they ended up joining my team. They got up to speed very quickly and ended up being a very valuable contributor in the team. In another example, I have worked with someone who previously held a position as a technical draftsman and 3D designer in a ship building factory (professionals call it a shipyard), but after some twists and turns ended up at a course for those interested in making a career switch, which led to them eventually working in the same company I do. Now they ship builds with confidence while making sure that the critical system we are working on stays stable. That developer also kicks my ass in foosball about 99% of the time. The domain knowledge advantage The combination of industry experience and software development skills is an incredibly powerful one. When a software developer starts work in a project, they learn the business domain piece by piece, eventually reaching a state where they have a slight idea about how the business operates, but never the full picture. Speaking with their end users will help come a long way, but there are always some details that get lost in that process. Someone coming from the industry will have in-depth knowledge about the business, how it operates, where the money comes from, what are the main pain points and where are the opportunities for automation. They will know what problems need solving, and the basic technical know-how on how to try solving them. Like a product owner, but on steroids. Software developers often fall into the trap of creating a startup to scratch that itch they have for building new things, or trying out technologies that have for a very long time been on their to-do list. The technical problems are fun to solve, sure, but the focus should be on the actual problem that needs fixing. If I wanted to start a new startup with someone, I’d look for someone working in an industry that I’m interested in and who understands the software development basics. Or maybe I’m just looking for an excellent product owner. How to help them succeed If you have a “senior junior” software developer on your team, then there really isn’t anything special you’d need to do compared to any other new joiner. Do your best to foster a culture of psychological safety, have regular 1-1s with them, and make sure to pair them up with more experienced team members as often as possible. A little bit of encouragement in challenging environments or periods of self-doubt can also go a long way. Temporary setbacks are temporary, after all. What about “AI”? Don’t worry about all that “AI”2 hype, if it was as successful in replacing all software development jobs as a lof of people like to shout from the rooftops, then it would have already done so. At best, it’s a slight productivity boost3 at the cost of a huge negative impact on the environment. Closing thoughts If you’re someone that has thought about working as a software developer or who is simply excited about all the ways that software can be used to solve actual business problems and build something from nothing, then I definitely recommend giving it a go, assuming that you have the safety net and risk appetite to do so. For reference, my journey towards software development looked like this, plus a few stints of working as a newspaper seller or a grocery store worker. who do you call a “senior senior” developer, a senile developer? ↩︎ spicy autocomplete engines (also known as LLM-s) do not count as actual artificial intelligence. ↩︎ what fascinates me about all the arguments around “AI” (LLM-s) is the feeling of being more productive. But how do you actually measure developer productivity, and do you account for possible reduced velocity later on when you’ve mistaken code generation speed as velocity and introduced hard to catch bugs into the code base that need to be resolved when they inevitably become an issue? ↩︎
I uploaded YouTube videos from time to time, and a fun comment I often get is “Whoa, this is in 8K!”. Even better, I’ve had comments from the like, seven people with 8K TVs that the video looks awesome on their TV. And you guessed it, I don’t record my videos in 8K! I record them in 4K and upscale them to 8K after the fact. There’s no shortage of AI video upscaling tools today, but they’re of varying quality, and some are great but quite expensive. The legendary Finn Voorhees created a really cool too though, called fx-upscale, that smartly leverages Apple’s built-in MetalFX framework. For the unfamiliar, this library is an extensive of Apple’s Metal graphics library, and adds functionality similar to NVIDIA’s DLSS where it intelligently upscales video using machine learning (AI), so rather than just stretching an image, it uses a model to try to infer what the frame would look like at a higher resolution. It’s primarily geared toward video game use, but Finn’s library shows it does an excellent job for video too. I think this is a really killer utility, and use it for all my videos. I even have a license for Topaz Video AI, which arguably works better, but takes an order of magnitude longer. For instance my recent 38 minute, 4K video took about an hour to render to 8K via fx-upscale on my M1 Pro MacBook Pro, but would take over 24 hours with Topaz Video AI. # Install with homebrew brew install finnvoor/tools/fx-upscale # Outputs a file named my-video Upscaled.mov fx-upscale my-video.mov --width 7680 --codec h265 Anyway, just wanted to give a tip toward a really cool tool! Finn’s even got a [version in the Mac App Store called Unsqueeze](https://apps.apple.com/ca/app/unsqueeze/id6475134617 Unsqueeze) with an actual GUI that’s even easier to use, but I really like the command line version because you get a bit more control over the output. 8K is kinda overkill for most use cases, so to be clear you can go from like, 1080p to 4K as well if you’re so inclined. I just really like 8K for the future proofing of it all, in however many years when 8K TVs are more common I’ll be able to have some of my videos already able to take advantage of that. And it takes long enough to upscale that I’d be surprised to see TVs or YouTube offering that upscaling natively in a way that looks as good given the amount of compute required currently. Obviously very zoomed in to show the difference easier If you ask me, for indie creators, even when 8K displays are more common, the future of recording still probably won’t be in native 8K. 4K recording gives so much detail still that have more than enough details to allow AI to do a compelling upscale to 8K. I think for my next camera I’m going to aim for recording in 6K (so I can still reframe in post), and then continue to output the final result in 4K to be AI upscaled. I’m coming for you, Lumix S1ii.
