Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
17
A lot of people these days know about collaborative filtering. It's that Netflix Prize thing, right? People rate things 1-5 stars and then you have to predict missing ratings. While there's no doubt that the Netflix Prize was successful, I think it created an illusion that all recommender systems care about explicit 1-5 ratings and RMSE as the objective.
over a year ago

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from Home on Erik Bernhardsson

It's hard to write code for computers, but it's even harder to write code for humans

Writing code for a computer is hard enough. You take something big and fuzzy, some large vague business outcome you want to achive. Then you break it down recursively and think about all the cases until you have clear logical statements a computer can follow.

9 months ago 23 votes
Predicting solar eclipses with Python

As I am en route to see my first total solar eclipse, I was curious how hard it would be to compute eclipses in Python. It turns out, ignoring some minor coordinate system head-banging, I was able to get something half-decent working in a couple of hours.

a year ago 24 votes
Simple sabotage for software

CIA produced a fantastic book during the peak of World War 2 called Simple Sabotage. It laid out various ways for infiltrators to ruin productivity of a company. Some of the advice is timeless, for instance the section about “General interference with Organizations and Production”:

a year ago 22 votes
What I have been working on: Modal

Long story short: I'm working on a super cool tool called Modal. Please check it out — it lets you run things in the cloud without having to think about infrastructure. Scaling out, scheduling, containerization, using GPUs, setting up webhooks, and all kinds of other stuff.

over a year ago 22 votes
We are still early with the cloud: why software development is overdue for a change

This is is in many respects a successor to a blog post I wrote last year about what I want from software infrastructure, but the ideas morphed in my head into something sort of wider.

over a year ago 23 votes

More in technology

A slept on upscaling tool for macOS

I uploaded YouTube videos from time to time, and a fun comment I often get is “Whoa, this is in 8K!”. Even better, I’ve had comments from the like, seven people with 8K TVs that the video looks awesome on their TV. And you guessed it, I don’t record my videos in 8K! I record them in 4K and upscale them to 8K after the fact. There’s no shortage of AI video upscaling tools today, but they’re of varying quality, and some are great but quite expensive. The legendary Finn Voorhees created a really cool too though, called fx-upscale, that smartly leverages Apple’s built-in MetalFX framework. For the unfamiliar, this library is an extensive of Apple’s Metal graphics library, and adds functionality similar to NVIDIA’s DLSS where it intelligently upscales video using machine learning (AI), so rather than just stretching an image, it uses a model to try to infer what the frame would look like at a higher resolution. It’s primarily geared toward video game use, but Finn’s library shows it does an excellent job for video too. I think this is a really killer utility, and use it for all my videos. I even have a license for Topaz Video AI, which arguably works better, but takes an order of magnitude longer. For instance my recent 38 minute, 4K video took about an hour to render to 8K via fx-upscale on my M1 Pro MacBook Pro, but would take over 24 hours with Topaz Video AI. # Install with homebrew brew install finnvoor/tools/fx-upscale # Outputs a file named my-video Upscaled.mov fx-upscale my-video.mov --width 7680 --codec h265 Anyway, just wanted to give a tip toward a really cool tool! Finn’s even got a [version in the Mac App Store called Unsqueeze](https://apps.apple.com/ca/app/unsqueeze/id6475134617 Unsqueeze) with an actual GUI that’s even easier to use, but I really like the command line version because you get a bit more control over the output. 8K is kinda overkill for most use cases, so to be clear you can go from like, 1080p to 4K as well if you’re so inclined. I just really like 8K for the future proofing of it all, in however many years when 8K TVs are more common I’ll be able to have some of my videos already able to take advantage of that. And it takes long enough to upscale that I’d be surprised to see TVs or YouTube offering that upscaling natively in a way that looks as good given the amount of compute required currently. Obviously very zoomed in to show the difference easier If you ask me, for indie creators, even when 8K displays are more common, the future of recording still probably won’t be in native 8K. 4K recording gives so much detail still that have more than enough details to allow AI to do a compelling upscale to 8K. I think for my next camera I’m going to aim for recording in 6K (so I can still reframe in post), and then continue to output the final result in 4K to be AI upscaled. I’m coming for you, Lumix S1ii.

21 hours ago 3 votes
From building ships to shipping builds: how to succeed in making a career switch to software development

I have worked with a few software developers who made the switch to this industry in the middle of their careers. A major change like that can be scary and raise a lot of fears and doubts, but I can attest that this can work out well with the right personality traits and a supporting environment. Here’s what I’ve observed. To keep the writing concise, I’ll be using the phrase “senior junior”1 to describe those that have made such a career switch. Overcoming the fear Fear is a natural reaction to any major change in life, especially when there’s risk of taking a financial hit while you have a family to support and a home loan to pay. The best mitigation that I’ve heard is believing that you can make the change, successfully. It sounds like an oversimplification, sure, as all it does is that it removes a mental blocker and throws out the self-doubt. And yet it works unreasonably well. It also helps if you have at least some savings to help mitigate the financial risk. A years’ worth of expenses saved up can go a long way in providing a solid safety net. What makes them succeed A great software developer is not someone that simply slings some code over the wall and spends all of their day working only on the technical stuff, there are quite a few critical skills that one needs to succeed. This is not an exhaustive list, but I’ve personally observed that the following ones are the most critical: ability to work in a team great communication skills conflict resolution ability to make decisions in the context of product development and business goals maintaining an environment of psychological safety Those with more than a decade of experience in another role or industry will most likely have a lot of these skills covered already, and they can bring that skill set into a software development team while working with the team to build their technical skill set. Software development is not special, at the end of they day, you’re still interacting with humans and everything that comes with that, good or bad. After working with juniors that are fresh out of school and “senior juniors” who have more career experience than I do, I have concluded that the ones that end up being great software developers have one thing in common: the passion and drive to learn everything about the role and the work we do. One highlight that I often like to share in discussions is one software developer who used to work in manufacturing. At some point they got interested in learning how they can use software to make work more efficient. They started with an MVP solution involving a big TV and Google Sheets, then they started learning about web development for a solution in a different area of the business, and ended up building a basic inventory system for the warehouse. After 2-3 years of self-learning outside of work hours and deploying to production in the most literal sense, they ended up joining my team. They got up to speed very quickly and ended up being a very valuable contributor in the team. In another example, I have worked with someone who previously held a position as a technical draftsman and 3D designer in a ship building factory (professionals call it a shipyard), but after some twists and turns ended up at a course for those interested in making a career switch, which led to them eventually working in the same company I do. Now they ship builds with confidence while making sure that the critical system we are working on stays stable. That developer also kicks my ass in foosball about 99% of the time. The domain knowledge advantage The combination of industry experience and software development skills is an incredibly powerful one. When a software developer starts work in a project, they learn the business domain piece by piece, eventually reaching a state where they have a slight idea about how the business operates, but never the full picture. Speaking with their end users will help come a long way, but there are always some details that get lost in that process. Someone coming from the industry will have in-depth knowledge about the business, how it operates, where the money comes from, what are the main pain points and where are the opportunities for automation. They will know what problems need solving, and the basic technical know-how on how to try solving them. Like a product owner, but on steroids. Software developers often fall into the trap of creating a startup to scratch that itch they have for building new things, or trying out technologies that have for a very long time been on their to-do list. The technical problems are fun to solve, sure, but the focus should be on the actual problem that needs fixing. If I wanted to start a new startup with someone, I’d look for someone working in an industry that I’m interested in and who understands the software development basics. Or maybe I’m just looking for an excellent product owner. How to help them succeed If you have a “senior junior” software developer on your team, then there really isn’t anything special you’d need to do compared to any other new joiner. Do your best to foster a culture of psychological safety, have regular 1-1s with them, and make sure to pair them up with more experienced team members as often as possible. A little bit of encouragement in challenging environments or periods of self-doubt can also go a long way. Temporary setbacks are temporary, after all. What about “AI”? Don’t worry about all that “AI”2 hype, if it was as successful in replacing all software development jobs as a lof of people like to shout from the rooftops, then it would have already done so. At best, it’s a slight productivity boost3 at the cost of a huge negative impact on the environment. Closing thoughts If you’re someone that has thought about working as a software developer or who is simply excited about all the ways that software can be used to solve actual business problems and build something from nothing, then I definitely recommend giving it a go, assuming that you have the safety net and risk appetite to do so. For reference, my journey towards software development looked like this, plus a few stints of working as a newspaper seller or a grocery store worker. who do you call a “senior senior” developer, a senile developer? ↩︎ spicy autocomplete engines (also known as LLM-s) do not count as actual artificial intelligence. ↩︎ what fascinates me about all the arguments around “AI” (LLM-s) is the feeling of being more productive. But how do you actually measure developer productivity, and do you account for possible reduced velocity later on when you’ve mistaken code generation speed as velocity and introduced hard to catch bugs into the code base that need to be resolved when they inevitably become an issue? ↩︎

9 hours ago 1 votes
Computer Games mag Interviews Don Bluth (1984)

Talks about the famous Dragon's Lair

2 days ago 3 votes
Refurb weekend: Gremlin Blasto arcade board

totally unreasonable price for a completely untested item, as-was, no returns, with no power supply, no wiring harness and no auxiliary daughterboards. At the end of this article, we'll have it fully playable and wired up to a standard ATX power supply, a composite monitor and off-the-shelf Atari joysticks, and because this board was used for other related games from that era, the process should work with only minor changes on other contemporary Gremlin arcade classics like Blockade, Hustle and Comotion [sic]. It's time for a Refurb Weekend. a July 1982 San Diego Reader article, the locally famous alternative paper I always snitched a copy of when I was downtown, and of which I found a marginally better copy to make these scans. There's also an exceptional multipart history of Gremlin you can read but for now we'll just hit the highlights as they pertain to today's project. ported to V1 Unix and has a simpler three-digit variant Bagels which was even ported to the KIM-1. Unfortunately his friends didn't have minicomputers of their own, so Hauck painstakingly put together a complete re-creation from discrete logic so they could play too, later licensed to Milton Bradley as their COMP IV handheld. Hauck had also been experimenting with processor-controlled video games, developing a simple homebrew unit based around the then-new Intel 8080 CPU that could connect to his television set and play blackjack. Fogleman met Hauck by chance at a component vendor's office and hired him on to enhance the wall game line, but Hauck persisted in his experiments, and additionally presented Fogleman with a new and different machine: a two-player game played with buttons on a video TV display, where each player left a boxy solid trail in an attempt to crowd out the other. To run the fast action on its relatively slow ~2MHz CPU and small amount of RAM, a character generator circuit made from logic chips painted a 256x224 display from 32 8x8 tiles in ROM specified by a 32x28 screen matrix, allowing for more sophisticated shapes and relieving the processor of having to draw the screen itself. (Does this sound like an early 8-bit computer? Hold that thought.) patent application was too late and too slow to stop the ripoffs. (For the record, Atari programmer Dennis Koble was adamant he didn't steal the idea from Gremlin, saying he had seen similar "snake" games on CompuServe and ARPANET, but Nolan Bushnell nevertheless later offered Gremlin $100,000 in "consolation" which the company refused.) Meanwhile, Blockade orders evaporated and Gremlin's attempts to ramp up production couldn't save it, leaving the company with thousands of unused circuit boards, game cabinets and video monitors. While lawsuits against the copycats slowly lumbered forward, Hauck decided to reprogram the existing Blockade hardware to play new games, starting with converting the Comotion board into Hustle in 1977 where players could also nab targets for additional points. The company ensured they had a thousand units ready to ship before even announcing it and sales were enough to recoup at least some of the lost investment. Hauck subsequently created a reworked version of the board with the same CPU for the more advanced game Depthcharge, initially testing poorly with players until the controls were simplified. This game was licensed to Taito as Sub Hunter and the board reworked again for the target shooter Safari, also in 1977, and also licensed by Taito. For 1978, Gremlin made one last release using the Hustle-Comotion board. This game was Blasto. present world record is 8,730), but in two player mode the players can also shoot each other for an even bigger point award. This means two-player games rapidly turn into active hunts, with a smaller bonus awarded to a player as well if the other gets nailed by a mine. shown above with a screenshot of the interactive on-board assembler. Noval also produced an education-targeted system called the Telemath, based on the 760 hardware, which was briefly deployed in a few San Diego Unified elementary schools. Alas, they were long gone before we arrived. Industry observers were impressed by the specs and baffled by the desk. Although the base price of $2995 [about $16,300] was quite reasonable considering its capabilities, you couldn't buy it without its hulking enclosure, which made it a home computer only to the sort of people who would buy a home PDP-8. (Raises hand.) Later upgrades with a Z80 and a full 32K didn't make it any more attractive to buyers and Noval barely sold about a dozen. Some of the rest remained at Gremlin as development systems (since they practically were already), and an intact upgraded unit with aftermarket floppy drives lives at the Computer History Museum. The failure of Noval didn't kill Gremlin outright, but Fogleman was concerned the company lacked sufficient capital to compete more strongly in the rapidly expanding video game market, and Noval didn't provide it. With wall game sales fading fast and cash flow crunched, the company was slowly approaching bankruptcy by the time Blasto hit arcades. At the same time, Sega Enterprises, Inc., then owned by conglomerate Gulf + Western (who also then owned Paramount Pictures), was looking for a quick way to revive its failing North American division which was only surviving on the strength of its aggressively promoted mall arcades. Sega needed development resources to bring out new games States-side, and Gremlin needed money. In September 1978 Fogleman agreed to make Gremlin a Sega subsidiary in return for an undisclosed number of shares, and became a vice chairman. Sega was willing to do just about anything to achieve supremacy on this side of the Pacific. In addition to infusing cash into Gremlin to make new games (as Gremlin/Sega) and distribute others from their Japanese peers and partners (as Sega/Gremlin), Sega also perceived a market opportunity in licensing arcade ports to the growing home computer segment. Texas Instruments' 99/4 had just hit the market in 1979 to howls there was hardly any software, and their close partner Milton Bradley was looking for marketable concepts for cartridge games. Blasto had simple fast action and a good name in the arcades, required only character graphics (well within the 9918 video chip's capabilities) and worked for both one or two players, and Sega had no problem blessing a home port of an older property for cheap. Milton Bradley picked up the license to Hustle as well. Bob Harris for completion, and TI house programmer Kevin Kenney wrote some additional features. 1 to 40 (obviously some thought was given to using the same PCB as much as possible). The power header is also a 10-pin block and the audio and video headers are 4-pin. Oddly, the manual doesn't say anywhere what the measurements are, so I checked them with calipers and got a pitch of around 0.15", which sounds very much like a common 0.156" header. I ordered a small pack of those as an experiment. 0002 because of the control changes: if you have an 814-0001, then you have a prototype. The MAME driver makes reference to an Amutech Mine Sweeper which is a direct and compatible ripoff of this board — despite the game type, it's not based on Depthcharge.) listed with the part numbers for the cocktail, but the ROM contents expected in the hashes actually correspond to the upright. Bipolar ROMs and PROMs are, as the name suggests, built with NPN bipolar junction transistors instead of today's far more common MOSFETs ("MOS transistors"). This makes them lower density but also faster: these particular bipolar PROMs have access times of 55-60ns as opposed to EPROMs or flash ROMs of similar capacity which may be multiple times slower depending on the chip and process. For many applications this doesn't matter much, but in some tightly-timed systems the speed difference can make it difficult to replace bipolar PROMs with more convenient EPROMs, and most modern-day chip programmers can't generate the higher voltage needed to program them (you're basically blowing a whole bunch of microscopic Nichrome metal fuses). Although modern CMOS PROMs are available at comparable speeds, bipolars were once very common, including in military environments where they could be manufactured to tolerate unusually harsh operating conditions. The incomparable Ken Shirriff has a die photo and article on the MMI 5300, an open-collector chip which is one of the military-spec parts from this line. Model 745 KSR and bubble memory Model 763 ASR, use AMD 8080s! The Intel 8080A is a refined version of the original Intel 8080 that works properly with more standard TTL devices (the original could only handle low-power TTL); the "NL" tag is TI's designation for a plastic regular-duty DIP. Its clock source is a 20.79MHz crystal at Y1 which is divided down by ten to yield the nominal clock rate of 2.079MHz, slightly above its maximum rating of 2MHz but stable enough at that speed. The later Intel 8080A-1 could be clocked up to 3.125MHz and of course the successor Intel 8085 and Zilog Z80 processors could run faster still. An interesting absence on this board is an Intel 8224 or equivalent to generate the 8080A's two-phase clock: that's done directly off the crystal oscillator with discrete logic, an elegant (and likely cheaper) design by Hauck. The video output also uses the same crystal. Next to the CPU are pads for the RAM chips. You saw six of them in the last picture under the second character ROM (316-0100M), all 2102 (1Kbit) static RAM. These were the chips I was most expecting to fail, having seen bad SRAM in other systems like my KIM-1. The ones here are 450ns Fairchild 21021 SRAMs in the 21021PC plastic case and "commercial" temperature range, and six of them adds up to 768 bytes of memory. NOS examples and equivalents are fortunately not difficult to find. Closer to the CPU in this picture, however, are two more RAM chip pads that are empty except for tiny factory-installed jumpers. On the Hustle and Blasto boards (both), they remain otherwise unpopulated, and there is an additional jumper between E4 and E5 also visible in the last picture. The Comotion board, however, has an additional 256 bytes of RAM here (as two more 1024x1 SRAMs). On that board these pads have RAM, there are no jumpers on the pads, and the jumper is now between E3 (ground) and E5. This jumper is also on Blockade, even though it has only five 2102s and three dummy jumpers on the other pads. That said, the games don't seem to care how much RAM is present as long as the minimum is: the current MAME driver gives all of them the full 1K. this 8080 system which uses a regulator). Tracing the schematic out further, the -12V line is also used with the +5V and +12V lines to run the video circuit. These are all part of the 10-pin power header. almost this exact sequence of voltages? An AT power supply connector! If we're clever about how we put the two halves on, we can get nearly the right lines in the right places. The six-pin AT P9 connector reversed is +5V, +5V, +5V, -5V, ground, ground, so we can cut the -5V to be the key. The six-pin AT P8 connector not reversed is power-good, +5V (or NC), +12V, -12V, ground, ground, so we cut the +5V to be the key, and cut the power-good line and one of the dangling grounds and wire ground to the power-good pin. Fortunately I had a couple spare AT-to-ATX converter cables from when we redid the AT power supply on the Alpha Micro Eagle 300. connectors since we're going to modify them anyway. A quick couple drops of light-cured cyanoacrylate into the key hole ... Something's alive! An LED glows! Time now for the video connector to see if we can get a picture! a nice 6502 reset circuit). The board does have its own reset circuit, of a sort. You'll notice here that the coin start is wired to the same line, and the manual even makes reference to this ("The circuitry in this game has been arranged so that the insertion of a quarter through the coin mechanism will reset the restart [sic] in the system. This clears up temporary problems caused by power line disturbances, static, etc."). We'll of course be dealing with the coin mechanism a little later, but that doesn't solve the problem of bringing the machine into the attract mode when powered on. I also have doubts that people would have blithely put coins into a machine that was obviously on the fritz. pair is up and down, or left and right, but not which one is exactly which because that depends on the joystick construction. We'll come back to this. Enterprises) to emphasize the brand name more strongly. The company entered a rapid decline with the video game crash of 1983 and the manufacturing assets were sold to Bally Midway with certain publishing rights, but the original Gremlin IP and game development teams stayed with Sega Electronics and remained part of Gulf+Western until they were disbanded. The brand is still retained as part of CBS Media Ventures today though modern Paramount Global doesn't currently use the label for its original purpose. In 1987 the old wall game line was briefly reincarnated under license, also called Gremlin Industries and with some former Gremlin employees, but only released a small number of new machines before folding. Meanwhile, Sega Enterprises separated from Gulf+Western in a 1984 management buyout by original founder David Rosen, Japanese executive Hayao Nakayama and their backers. This Sega is what people consider Sega today, now part of Sega Sammy Holdings, and the rights to the original Gremlin games — including Blasto — are under it. Lane Hauck's last recorded game at Gremlin/Sega was the classic Carnival in 1980 (I played this first on the Intellivision). After leaving the company, he held positions at various companies including San Diego-based projector manufacturer Proxima (notoriously later merging with InFocus), Cypress Semiconductor and its AgigA Tech subsidiary (both now part of Infineon), and Maxim Integrated Products (now part of Analog Devices), and works as a consultant today. I'm not done with Blasto. While I still enjoy playing the TI-99/4A port, there are ... improvements to be made, particularly the fact it's single fire, and it was never ported to anything else. I have ideas, I've been working on it off and on for a year or so and all the main gameplay code is written, so I just have to finish the graphics and music. You'll get to play it. And the arcade board? Well, we have a working game and a working harness that I can build off. I need a better sound amplifier, the "boom" circuit deserves a proper subwoofer, and I should fake up a little circuit using the power-good line from the ATX power supply to substitute for the power interrupt board. Most of all, though, we really need to get it a proper display and cabinet. That's naturally going to need a budget rather larger than my typical projects and I'm already saving up for it. Suggestions for a nice upright cab with display, buttons and joysticks that I can rewire — and afford! — are solicited. On both those counts, to be continued.

2 days ago 6 votes
This machine automatically scans books from cover to cover

Hard data is hard to find, but roughly 100 million books were published prior to the 21st century. Of those, a significant portion were never available in a digital format and haven’t yet been digitized, which means their content is effectively inaccessible to most people today. To bring that content into the digital world, Redditor […] The post This machine automatically scans books from cover to cover appeared first on Arduino Blog.

4 days ago 5 votes