Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]
11
I’ve been to plenty of bad interviews. Sometimes, only some questions are bad, but usually it goes further than that. Bizarre questions like “what’s the difference between a number and an array” are just a symptom of deeper issues. Let’s take a step back — why are we interviewing? To hire someone — in our case, a software developer — good at building working software (or at least, ahem, with a good outlook), in our team. A good interview process is precise (not hire people who can’t build software, and not reject people who can), and fast (to save our team’s time, and hire the candidate before anyone else does). That’s it, really. Hire good candidates with a reasonable precision / time ratio. If this sounds trivial, then why do some interviewers… Enjoy making the candidate look stupid? Ask questions that every sane person googles in real-life? Not help a struggling candidate as they would a struggling teammate? Ask random questions off the top of their head instead of planning...
over a year ago

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from Vladimir Klepov as a Coder

Growing my team 4x has been a pain. Can we do better?

My name is Vladimir, and I'm an engineering manager of a team building a banking app. Following the success of our core banking product, we've decided to expand to other financial services. In the last four months my team has grown 4x, going from a 4-person team to 4 teams totalling 15 people. It was, frankly, a shitshow, and now I see many things we could have done better. At first sight, increasing your team is a perfect way to speed up your product development. In practice, scaling is very challenging. Today, I'll share the pains of growth we've ran into: Disbalanced growth across functions. Teams becoming too large to manage. Processes breaking down. Mass onboarding challenges. Seniority skew. Now that it's all done, I can't say these problems were unexpected, or that our solutions have been incredibly inventive, but a first-hand account is worth writing down. Whether you're a leader expecting your team to scale, or just curious about the daily challenges of an engineering manager, hope you'll find something for yourself. Let's go! Scale evenly across functions You lead a 5-man engineering team. You'd like to build 2x the stuff you're building. The obvious solution is to hire 5 more engineers. Problem solved? Not so fast. Anything you build flows across several stages, each handled by a certain role, for example: product manager -> design -> engineering -> QA -> product analytics -> (back to) product manager. If you have a balanced flow with your current team, enlarging just the engineering will not make you more productive: Downstream functions (QA / analytics) can't keep up with the increased production — they must either put in overtime or downgrade quality standards. Upstream functions (product / design) can't fill the backlog fast enough, and the eng team has nothing to focus on, either slacking or refactoring the refactorings. With rapid growth, some temporary disbalances are unavoidable. It's fine to spend a few weeks or months in a disproportionate state, but overall aim for balanced scaling. Here are some tips to smooth the transition and even use the disbalance to your advantage: If you have any control over it, give product & design a hiring head-start. You can get their artifacts ready for development with a limited series of grooming meetings, and once the new engineers are on board, you'll have some great useful tasks to feed them. Catch up on your tech debt. Have too many engineers and not enough product tasks? Don't despair: use the time to clean up some old bugs, do the overdue refactorings, and prepare the codebase for the speed-up. Expand the area of responsibility. If you lack QA specialists, it might be time for the engineers to practice their testing skills. Oversized non-technical component (product / design) is more problematic, but you can give them some no-code tools to replace the eng team in some scenarios — e.g. build an admin UI where PM can edit the texts, create new banners, and so on, without involving your team. Update team structure Say you have a normal-sized team (4–7 engineers) with your average meeting structure (whole-team planning, grooming, and retro + weekly 1x1 with every team member). Making it a 15-person team won't work at all. An hour-long 4-person retro has 10 minutes of speaking time per member — enough to make a point. In a 15-person retro, it's 3 minutes — not practical. You either exclude some people, or extend the meetings — both poor options. It's harder to agree on any decision, because you now have 3x the possible objections. The 1x1s alone eat up 7 hours (almost a full day!) of your time a week. Managing communication of 15 people plus all the external stakeholders is time-consuming. The team must be split. What does it mean to be a separate team, anyways? Ever heard of "high cohesion and low coupling" principle in software architecture? I think this also applies to teams: Shared information space. Team members know what you're working on, where you're going, who your peers are, the system structure and so on. Own a well-defined part of the product and the codebase. A look at a random feature is enough to guess the responsible team. Control your processes — meeting structure, releases, etc. You can't be responsible for what you can't control. Have all the capabilities needed for your day-to-day work. Begging someone each time you need to deploy, change the API or add a banner is not very effective. It's best to split by product domain: customer acquisition team owns the landing page and signup, daily banking team owns the main app, and so on. You could split by layer (product + infra) or by function (backend + frontend + mobile) — I feel these compromise points 2 and 4, but let's not die on this hill today. At any rate, a team over 8–10 people must be split into sane-sized chunks to keep going. A note on grouping You can split a team by (A) building the new team out of newbies or (B) mixing newbies and oldies in each new part. Prefer mixing: it distributes the knowledge across the organization, and the social connections from the original team prevent siloing. You could argue that (A) keeps the original high-performing team intact, but it does so by slowing down the new team and undermining your long-term flexibility. Where to get new leaders? Splitting a team into 3 parts creates 2–4 leadership positions, depending on your place in the new structure. Ideally, you have senior members of the original team to lead the new teams, because it's a rare opportunity for career growth on a management track, and they can easily hire and onboard new members of their teams. If you don't have a suitable candidate (everyone is either very junior, or hates management), it's fine to hire externally — following a few hiccups, I recommend hiring people with prior leadership experience, because adjusting to a new product and a new role at once can be too much. Split iteratively You don't have to produce a fully separate team right away. As usual, move step by step — you get faster results, and can adapt to the issues that arise. Here's one possible sequence: Assemble a domain team, appoint one as the lead (you can call it trial-lead, to give them a chance to cop out). See how they like their new roles, and if the headcount needs tweaking. Run a retro for the new team to catch communication issues or cross-team dependencies early. Separate the domain backlog and kanban boards (or wherever you track the tasks). You'll need it for further process splitting, and to assess the team's load and velocity. Split planning and daily meetings, so that the teams don't waste each other's time on discussing irrelevant tasks. Gradually transfer the remaining processes (1x1s, onboarding, postmortems) to the new lead. Split the codebase, so that the new team can fully own its service. Update your processes Just like your single-team structure, the processes you have will likely fail for a larger team, especially if you have many newbies. Example: we had a liberal release process — if you want some feature in production, you deploy it. As the team grew, the release frequency dropped — the newbies were afraid to touch prod, the oldies were hoping one of the other 14 people would do it. Before I give you my solution to this puzzle, let's look at the general advice for process scaling. Localize processes to the new teams. Owning processes makes a team more effective. Retros, plannings, kickoffs, demos, daily stand-ups, releases, documenting, on-call duty, whatever you can split, do split. Yes, your overall team loses sight of the stuff going on across the system, causing duplication and poor decisions, but in return we can focus on a specific business area, and maximize productive time instead of drowning in discussions. If you need to offset the downsides, introduce a cross-team sync here and there (still working on this one). Let the new teams experiment with processes. What worked for your original team doesn't matter, that team is dead and gone. What works for one of your teams won't necessarily work for the other, because they work in different conditions. For example, our CA team has many time-bound tasks from marketing. The core banking team focuses on building quality software, and fixing the bugs as they arise. Very different teams. Start with a copy of your current processes (just to start somewhere!), and introduce team retros as early as possible to tweak the process as needed. Stricten the centralized processes. Back to our problem with deployment — we couldn't isolate the release process to sub-teams, because splitting a monolithic front-end into independently deployable parts is technically challenging. We introduced a more structured release process: The releases are automatically built and ready to deploy every morning. No more decisions to make. Daily rotation of release managers responsible for getting the release to production. With regular training, you get better at releasing. The release process is clearly documented. The newbies have a clear path to follow, making it less stressful. This works for other centralized processes — writing documentation, debugging with customer support, maintaining shared libs. Overall: hand over as much process as you can to the sub-teams, and introduce clear rules for the remaining centralized processes. The onboarding valley Surprisingly, fast hiring can reduce your team's productivity in the short term. The newbies are not yet up to speed, and the oldies now spend time explaining your codebase and reviewing code. This will fix itself over time, but here are a few strategies to get past the bump faster. Prefer slower growth. Adding one person every few weeks is much better than adding 6 people simultaneously, because: The "onboarding load" stretched over time occupies a smaller share of your team's resource. A few weeks in, new hires can already help onboard someone else. In some cases, they'll do a better job than any oldie, because their memories of one-off tasks like setting up the dev environment are fresh. Every onboarding exposes new roadblocks in your process, helping you smooth the next ones. On paper, batch onboarding might seem like a time-saver, as you can make a lecture explaining the basics to many people at once. In practice, unless your product is very small, or the tasks are very repetitive, every newbie faces very different challenges, drowning you with a wave of questions. Encourage peer-to-peer onboarding. As a leader, you might think onboarding is your personal responsibility. I call BS — peer-to-peer onboarding is clearly better: More "onboarding resource" leads to faster, better onboarding. The load on you, personally, decreases, freeing time to do other impactful things and, you know, live. Team members get a safe environment to practice their mentorship skills. People get to know each other, instead of only talking to you. ICs with recent hands-on experience do a better job at explaining the specifics than you. You can pair a formal "mentor" to every newbie, or direct questions to a team group chat. If you want to control the overall onboarding, at least route specific questions to team members experienced in that area instead of trying to come up with all the answers yourself. Write the docs. The best way to make onboardings cheaper is writing stuff down instead of explaining it over and over again, with your mouth. Some particular things to focus on: Onboarding checklist — the things every new team member must do: get a VPN certificate from the security dude, join this and that chat, clone a repo here and there, boom you're done. Only include essential steps — adding somewhat useful stuff obscures the actually important things. Document your existing business processes, system architecture, technical conventions, team and communication structure. It's better than explaining in real-time, because you get higher-quality charts, relevant links, and you can collaborate to put the knowledge of multiple team members in one place. Tooling and automation. The more automated a process or convention is, the less onboarding you need. Example: if you build your releases locally and upload somewhere via FTP using the keys you get from Piotr the devops, it's time to set up decent CD instead of documenting the current state of affairs. Pro tip: encourage newbies to improve and update the docs as they follow along — it's a great first contribution to your team! Senior vs junior hires It seems sensible to focus exclusively on senior hires. Experienced engineers get up to speed quicker, because they're already familiar with the basics, have a lower risk of making catastrophically poor decisions, and can bring good practices and ideas from across the industry to your team. Not so fast — here are some reasons to hire junior developers. With little exposure to the industry, they can easily adapt to whatever culture and processes you have. Anyone on your team can mentor a junior hire, while getting e.g. a junior engineer with decent knowledge of the product to mentor a newly hired senior engineer might be awkward, not very productive, or even taken as an insult. And of course, you can hire more junior engineers on the same budget. Overall, aim for a balanced team composition in the mid-term. You don't want your team to be a kindergarten, but a nursing home is no good either. Remember that people tend to gain experience, so the junior engineers you hire will become middle in no time. Today, we've discussed the challenges of rapid team growth — and ways to address them: Hiring more engineers won't speed you up unless product, design and QA grow to match. Start by growing product & design. If product lags behind, use the spare time to clean up the tech debt. Expand the area of responsibility of the oversized roles. Teams over 8–10 people are hard to manage. Split into chunks of 3–7 people, preferably by business domain. Mix old and new members in each team. Split step-by-step instead of going all in. The processes of your original team won't accommodate a larger team. Localize the processes to the sub-teams as much as possible. Let the teams tweak their processes to suit their needs. Stricten the remaining centralized processes. Onboarding is time-consuming, and can slow you down. Go slow: onboarding a person every week is easier than 6 people at once. Write the docs instead of explaining stuff over and over. Peer-to-peer knowledge transfer is better than onboarding everyone personally. Hiring only senior engineers is not a silver bullet. Aim for a healthy experience distribution in the mid-term. Hope these tips help you get past the scaling issues and up to speed in no time.

10 months ago 28 votes
From engineer to manager: what I love, what I hate

It's been almost 2 years since I moved to a team lead role, then to a full-time engineering management position after the expansion of our team. I've been a front-end developer for 7 years before that, and initially I took the "advanced individual contributor" career track before doing the management turnaround. How's it been? Bumpy, but fun. In this article, I'll share the things I love and hate about my current job. Love Let's start with the positive side of management positions. There's plenty to love, honestly. Impact First things first, I adore the power to improve the product we're building, and the overall team well-being that comes with a management position. As an engineer, you'll sometimes find yourself in a tough spot with little to no power to change things. Early morning standups are a chore? Code quality sucks? The new feature makes no sense? As a manager, the power to change is yours: you get both the formal and informal authority to change things for the better, and to make yourself and your team happier. Your words have weight. If you, an IC, say "guys, we really should write tests", everybody goes "oh, crazy old Vladimir, all grumpy again, haha, where do you see tests fit here?". If you, an EM, casually say "guys, we really should write tests", you might be surprised to find the tests unexpectedly growing in different places. Pleasant. Career opportunities Being an engineering manager is a more promising career opportunity than an engineering track. This might be controversial, but hear me out: The EM is normally higher-paid than the basic team-level engineering grades (junior / middle / senior). There are higher IC grades (staff, principal, president of code, whatever) in the EM+ salary bands. These staff+ IC jobs are concentrated in larger tech companies, because those have tougher technical challenges. Almost every software team in every company has a leadership position. The management career ladder is "taller" than that of ICs. Yes, the chances of becoming a CTO are slim, but it's an opportunity that's just not there for a pure IC with no management experience. All in all, I believe the demand for EMs to be steadier than for staff+ engineers, and this path gives you more opportunities at the later stages of your career. On a related note... Transferable skills Management skills are more widely useful than an IC engineering role. A decent front-end engineer with React experience won't have trouble moving to another front-end framework, and can probably transition to a back-end / mobile engineering role with -1 grade (a year handicap or so). That's not bad. What roles are available to someone with engineering management experience? First, you can easily take on a team with a wildly different focus — mobile developers, infrastructure, ML engineers. You'd need some time to get up to speed on the big-picture technical struggles of your new team, but most companies would take this shot. If you don't want to be an EM any more, you're well-positioned to move to a project or product management role. If the entire tech market falls into decline, many management skills would still work for other industries. While I don't see a big flow of tech managers moving into construction business (tech does pay well), there's one alternate path to consider — entrepreneurship. Involvement with people and business decisions makes for great training before starting your own business. So, being a manager gives you quite a bit of career flexibility, and makes you less vulnerable to future technological shifts. Less knowledge rot Suffering from front-end fatigue? Can't keep up with the newest shiniest frameworks and tools? Management's got you covered! The "hot new" agile / kanban / scrum methodologies are 20–30 years old. The basic meeting types (demos, dailies, 1-on-1s) have been developing for centuries. At the core, you have teamwork and human interactions, which haven't changed that much since the beginning of humanity. My grandfather was a big railroad boss in the 70s, and we can sensibly discuss some of my work challenges. "Oh, you have this talented slacker? Give him some big important task, let's see what he's worth." When it comes to computers, he's more like "I'd like a shovel big enough to throw all your silly gadgets into stratosphere." So, if you're tired of keeping up with the latest hot thing in tech, a management role can provide a well-deserved relief. Do keep an eye on what's happening on the tech side of things, but there's no urgency, and no need to get real deep. New challenges Frankly, after 4–5 years of working in a particular tech area, you can solve the vast majority of practical problems well enough. If you want some work challenge, you can: Slightly alter your stack — say, a new FE framework. But it's unlikely to keep you engaged very long. Make a broader career shift — e.g. frontend to backend. This would probably give you another couple years of fun, but such transitions are, in my experience, either random (e.g. your BE dev quits and someone has to fill the role), or hit your salary. Invent problems out of thin air — rewrite everything using a new library, or handle 9000 RPS "for the future". Fun, but most of the time it's more harm than good for your team and business. Of all the possible career moves a seasoned engineer can make, switching to management gives you the most new challenges (years worth of new stuff to learn) without hitting your salary. Hate As much as I like the challenges and impact of my new role, and the practical career benefits, I'll be the first one to admit it has downsides as well. Corporate BS As an engineer, I hated bloody corporate BS: individual performance reviews, useless deadlines, company-enforced restrictions on processes and tech stack. Well, congratulations, as a manager you are the sheriff of these practices, whether you believe in them or not. As a leader of a team in an org with performance calibrations, I must nominate 1 person who hasn't been working hard enough every 6 months. This human chess is soul-sucking, but I can't make it go away — if I don't offer a sacrificial teammate, someone will be picked randomly further down the process. Crazy shit. Sometimes you can negotiate a bit, or hack the process, e.g. assign "below-expected performance" on a round-robin basis, but to your team you'll sometimes be the corporate monster. Sigh. And I haven't even been through the real tough stuff like layoffs, closures and reorganizations. Awkward social situations I've made it to an engineering management position by being good at building stuff. I've been prepared to help with technical decisions, give career guidance, tune processes and set up automation as needed. In fact, a large portion of my job is debugging social tensions and psychological insecurities of people. Your junior engineer comments out a few tests to deploy a feature preview. The QA person sees this, and is very pissed because your whole team apparently does not respect the QA role and the value they provide. Restore trust. A project manager makes an unsuccessful joke that hurts your designer, who's now crying. Make the PM apologize. Am I a kindergarten teacher or something? Boy, I'm no psychologist, and I can't say I'm exceptionally good with people. This part of my job is quite hard, trying to fake it til I make it here. Office hours Life of an engineer is relatively relaxed. If you don't have anything urgent, you can go lay on the grass for half a day, thinking about the future of your project or something. You can miss a few meetings on short notice, no questions asked. Now, you're an EM. Try going and lying on the grass for a few hours. You come back to a messenger full of problems: your intern can't work because she forgot how to npm install; a senior manager wants to discuss some potential feature; release has derailed. Also, you can't really skip a meeting you're supposed to facilitate / organize without some up-front preparation. This might improve as your team matures and builds better processes, but in general you feel office hours much more as a manager, and your work-life balance directly depends on how good you are at your job. Long feedback loop The final thing I hate about management is the long feedback loop of your actions. Most engineering tasks show the result quite fast: new features take weeks to months, and if that's too long for you — fix a bug and see happy users the next day, or refactor some code and watch complexity decrease in a few hours. Amazing! You're a manager? Well, very few of your actions produce a visible result in under a month. Suppose your team has grown too large, and you want to split it up. You must pick a well-rounded set of engineers for the new team, talk to everybody involved to see how they feel about such a change, arrange new regular meetings, set up processes and communications, do some jira magic, maybe isolate the codebases of sub-products. If you think it can be done in a week, well, you're wrong. Then, even the right changes can make things get worse before they get better. Say you're understaffed, and you decide to hire. In the short term, you spend hours and hours interviewing, and a new team member won't get up to speed right away, sucking out precious time for onboarding. It's sometimes hard to see the long-term goal behind the short-term inconvenience. So, while engineering problem-solving is often fairly straightforward, management changes are more similar to large-scale refactorings. You won't see any quick improvements, which can be frustrating. To sum up, moving from an IC engineering role to a management position has been a rollercoaster ride for me, with both bright and bleak spots. Here's what I love: The wider impact on the product and team. Management is a great long-term career track: it gives you more job opportunities than a staff+ IC, the flexibility to move between different technical areas and roles, and skills that will be relevant across various industries for years to come. If you're bored with your field of tech expertise, moving to a management role is a great way to bring the challenge back into your job. And here's what I hate: Enforcing corporate decisions and policies can be soul-sucking. Dealing with social tensions and psychological insecurities of people isn't something I was ready for. It's hard to go offline even for a few hours without preparing in advance. Your actions have long and non-linear feedback loops with very delayed gratification. Now, is this career move the right one for you? If you enjoy challenge and responsibility, and you get an opportunity — I'd say go for it! Yes, management is not a fit for everybody (I'm not even sure it fits me TBH), but it's a great experience that would surely expand your skill set and make you see engineering work from a new angle. If you totally hate it, you have plenty of time to go back into coding =)

a year ago 18 votes
The most useful programming language

Aspiring developers often ask me what's the best programming language to learn. Personally, I mostly work with JS — solid choice, but everyone and their dog learns JS these days, so it might be time to add some diversity. I'm curious — which single programming language covers the most bases for you, and gives you most career opportunities for years to come? That's the question we'll try to answer today. Here's the plan. I made a list of 8 tech specializations: 2 web development areas: back- and front-end. Both pretty big areas, and ones I have most experience with. Mobile and desktop native app development. Native app development (especially desktop apps) seems to have fallen out of favor, but there's still enough work in these areas. Quality assurance automation. QA grows along with engineering, and increasingly relies on automated tests. Embedded systems. We'll focus on microcontroller programming, not fat boxes with a full windows / linux OS. Quite a promising area with the growth of IoT. Game development. Granted, I don't know much about this area, but I'll do my best to cover it as well, as many developers dream of building a fun game someday. Data analysis and Machine Learning. One of the most hyped areas of the last decade. The contenders are the usual suspects from TIOBE top 20: python, C, C++, Java (grouped with Kotlin and other JVM languages), C# (again, throw in VB and other .NET languages), JavaScript (and TypeScript), PHP, Go, Swift, Ruby, Rust. I left out SQL and Scratch, because they're not general-purpose languages, and Fortan with Matlab, because they aren't really used outside of scientific / engineering computing. A language scores 1 point by being the industry standard in the area — vast community and ecosystem, abundant jobs. Being useful for certain tasks in the area gets you 0.5 points. So, let's see what languages will make you the most versatile engineer, shall we? Backend Let's start with the simple one — Java, C#, Python, PHP, Go and Ruby are all excellent back-end programming languages. Of these, I'd say PHP is slightly more useful as many low-code solutions rely on it, and Ruby is steadily declining. Still, all these languages have earned 1 point. Next, 0.5 points go to: C++, used in high-load and time-critical scenarios, JS — node.js is often used to support front-end, but there aren't that many strictly back-end jobs for JS developers. Rust — still not that widely used, but growing fast. The only languages to fail here are Swift (technically usable on server via e.g. vapor, but I couldn't find any jobs in this stack) and C. Frontend Obviously, JavaScript is the language for front-end developers, which runs natively in browsers. But, surprise, other languages still qualify! All solid back-end languages (Java, C#, Python, PHP, Go, Ruby) get 0.5 points, because you can solve many UI problems by rendering HTML server-side the old-school way. C# has a slight edge here, since blazor is quite smart and popular. C, C++, and Rust score 0.5 points because they can be compiled to WebAssembly and run in the browser — just look at figma. Rust also powers some cool JS tooling, like biome and swc The only language to fail here is, again, Swift. QA automation The topic of QA automation is really simple. Java and python get the cake — Allure, Selenium, JUnit, and pytest are the most sought-after automation tools on the market right now. JS gets 0.5 points for playwright and cypress — the preferred tools for testing complex web front-ends. A few automation tools support C# — worth 0.2 points. Mobile apps Another straightforward area. Android apps are written in JVM languages (Java / Kotlin), iOS is integrated with Swift (finally). JS scores 0.5 points, because you can effectively build apps with React Native, and you can get pretty far with PWA or a good old WebView. Another 0.5 point for C#, thanks to Xamarin and MAUI. Desktop apps (windows / linux / MacOS) The three kings here are C++, C#, and Java. JS gets 0.5 points, again, for electron — disgusting or not, it's widely used. Another 0.5 points for Swift, because that's what you build MacOS apps with, but MacOS computers are relatively niche. Rust has the highly-hyped Tauri project for building desktop apps, but it's not that widespread, and I'm not aware of any high-profile apps using it. Let's give each 0.2 points for the effort and check back later. Embedded systems Embedded systems are usually tight on resources, so compiled languages are the way to go here. Basically any embedded job requires C and C++. Rust is, as usual, very promising, but not that popular yet, so 0.5 points. Another half-point for Python — used for edge computer vision and prototyping, but struggling with high memory requirements. Game development The primary languages in big gamedev are C++ (used in Unreal Engine) and C# (for Unity). Since mobile games are a thing, Java and Swift get 0.5 points each, because that's what you'll likely use here. Another 0.5 points for JS (browser games). Rust should be quite a good fit for games, but (as expected by now) it's not quite there yet. Data Analysis & Machine Learning It's no secret that Python is the language of choice for anything data-related, and most of the cutting edge stuff happens, well deserved 1 point here. But do you know there's another top language to get your piece of Data & ML hype? Big companies have a lot of data, right? And big companies love Java. So, many big data tools (especially coming from Apache — Hadoop, Spark, Jena) work with Java, and most data jobs require experience with python or java, so another 1 point for java. On to more surprises. Large chunks of data-heavy python libraries are actually written in C / C++ — e.g. over a third of numpy, or most of LlamaCPP — which earns both half-a-point. As you'd expect, Rust is also gaining traction for this use case with stuff like pola.rs, so another 0.2 points! The final half-a-point goes to JS for powering much of the UI / visualization stuff (see e.g. bokeh). Before we reveal our final ranking, let's weigh the categories, because they're not the same size. I've used some back-of-the-napkin analysis of job postings and sizing of reddit / linkedin groups and my personal experience. With backend as our reference, I'd say frontend is roughly the same size. Mobile development is surprisingly sizable — let's give it a 0.6 weight. For QA, I'd say 0.2 makes sense, as 1 QA per 3–5 devs is a normal ratio, and manual QA is still a thing. Desktop is easily the smallest area, looks like a 0.1 to me. For gamedev, 0.5 is just my random guess. Finally, there are surprisingly many data people — with the good salaries, let's make it a 0.6. Putting it all together: Java takes the first spot by a good margin by topping 5 categories, and having some gamedev / frontend capabilities. Place other JVM languages (especially Kotlin) around here, but with a discount since they're not as widely used. The next three are really close, but JS gets slighly ahead by being average at everything except embedded, even though it's only the top choice for front-end development. Python and C# tie for the third place. Both are top-tier backend languages with other strong areas (QA / ML for python, desktop and gamedev for C#). C++ is not that far behind either, as it's still the top language when it comes to efficiency. It also steps into other languages' realms when they need some speedup (WebAssembly / ML). Next come "three backend friends" — Go, PHP, and Ruby. All top-notch languages for building web backends, but not much else beyond that. Of these, Ruby is on the decline, and PHP and Go both have their separate niches. Rust does not score that well, but still makes it into the top 10 — not bad for such a new language. It has great growth potential by eating at the traditional C++ areas, super excited to see where it gets in 3–5 years. We all love good old C, but C++ looks like a better fit for complex systems. Swift comes in last — fair enough for a language that's only useful for the products of one single company. Perhaps surprisingly, the single most useful language is Java. Python and JS, beginner favorites, come strong, with a very different focus. C# perhaps deserves a bit more attention. Overall, today we've learnt about many amazing technologies that allow languages to sneak into each other's territory. If you were to start anew, what language would you learn?

a year ago 19 votes
I conducted 60 interviews in 2 months — here's what I learned

It's hard to believe, but, starting mid-october 2023 I conducted 60 technical interviews and hired 10 people into our team. It's been extremely tiring: around 80 hours of active interviewing, plus writing interview reports, plus screening CVs and take-home assignments, plus onboarding new members — all while doing my normal work stuff. Still, I feel like I learnt a lot in the process — things that would help me as a candidate in the future, and might help you land your next job. Note that I'm a fairly relaxed interviewer, and, as an internal startup of a large tech company, we generally have a more humane hiring process, so your mileage may vary. Still, I've done my best to pick the tips that I feel are universally applicable. Here are nine insights I took out of this experience, in no particular order: Be generous with your "expected income". Say you're a solid higher-middle engineer, and you ask for a senior salary. My thought process: OK buddy, it's a bit more than reasonable now, but I won't have to fight for your promotion 8–12 months from now when you get there, and I don't have to spend another 12 hours of my own time (and leave my team understaffed for another few weeks) looking for a real hardcore senior, so I'll let you have it. Now suppose you ask for a junior salary. It's suspicious — why is your bar so low? Is there someting about your work performance you're not telling us? So, do your research on reasonable salaries for your level of experience, and aim slightly above that. Ask the right questions. I always leave time for the candidate to ask me questions — obviously, this lets the candidate probe what it's like to work at our team, but it's also the best opportunity for me to learn what really matters to the candidate. I've never been much of an asker myself, but now I see that "Thanks, I have no questions" does not look good — if anything, it paints you as someone who doesn't care. Here's a short list of good questions: What does the daily work in this role look like? Harmless. What features are you building next? Caring about the overall product, nice. Sometimes the answer is "I can't disclose this secret", but not that often. Anyting about processes or team structure: how many people are on the team? How often do you release? What regular meetings do you have? Interested in organization, might want to be a team lead someday, great. Anything tech-related: which framework do you use? Why did you pick framework X? How do you test your app? Especially suitable for junior- to middle developers who are most involved in hands-on work. What kind of tasks do you see me doing? Again, just a good neutral question, because responsibilities for any role differ wildly between companies. What growth / promotion opportunities does this position have? Cool trick, flipping the feared "where do you see yourself in 5 years" question against the hiring manager. Here are a few questions that are not very good: Do you use jira and github? It's a minor detail, won't you be able to work with youtrack and gitlab? Do you sometimes work late? Only if something breaks, but overall this question makes you seem a bit lazy. People on poor teams that routinely overtime aren't likely to answer this question honestly, at any rate. Social skills matter. I understand that not everybody is super outgoing, but if we already feel awkward 1 hour into our acquaintance, why work together — to feel awkward for months to come? Just a few tips anyone can follow: Be energetic. You're tired, I'm tired, we're all tired of endless interviews. Are you just tired today, or generally always too tired to get anything done? I know it's easier said than done, but try and show me all the energy you have left. Show respect. People enjoy being respected. Very easy one: you have a great product. Sounds like you have a great engineering culture. This is one of the most interesting interviews I've ever seen. Like, I know you don't necessarily mean that, but subconsciously I'm very pleased: "oh yes, I'm very proud of my interview process, thanks for noticing" On a related note... Provide conversation opportunities. Q: Do you use TDD? Bad answer: "no". Good answer: "no, but I've heard of it. Interesting approach. Does your team use TDD?" Now you get to spend 5 minutes talking on your terms instead of being bombarded with random questions, and you come off as someone curious about stuff. On another related note... It's easy to hurt people. People normally ask you about stuff because they care about it. So, again, the interviewer askning "do you use TDD?", presumably, likes TDD and uses it. So, the worst answer: "no, TDD sucks, it's pure waste of time for idiots." A rare interviewer might appreciate you having a strong opinion on a topic, but to most this just paints you as a jerk, kinda like "Here's a photo of my children — "I hate children, and yours are especialy horrible". Not smart. Smart talk is not your friend. Saying stuff like "our front-end guild evaluated several cutting-edge approaches to testing universal applications" only makes you seem smart if you can elaborate on that topic: what these approaches were, the pros and cons you found, what tradeoffs you made for your final decision. If you can't answer a follow-up question beside "we settled on jest, not sure why", it was better to stay away from that topic altogether. Related: "in code reviews, I always consider the optimality of the algorithm selected" (proceeds to estimate the time complexity of comparison-based sorting as O(1). I never ask this unless the candidate boasts about her algo skillz). Admit your mistakes. Don't know an answer? Your code has a bug? It's always better to admit it and then try to come up with something at the spot than trying to talk your way out of it. Event loop? Sure thing, I'm an expert on loops. It's the way events are looped. Uses logarithmic weighing. Again, this makes you look like a candidate with big mouth and small hands. I have seen a couple of people who could talk their way out of any situation, but I honestly think with such skills you'd do better in a different line of work, like international relations, or selling financial services. Note that you really should give it your best shot — giving up at the first sign of trouble is not a good impression. If you genuinely have no idea — see conversation opportunity: "Event delegation? Tough luck, never heard of it. Would you tell me about it so that I learn something new today?" Make yourself memorable. It's hard to keep detailed profiles of 10 candidates in mind — after a good interview streak all I remember is the general impression (great / OK / horrible) and a few truly notable things. This guy worked for some crypto scam that went bust, that girl had a cute dog that was trying to eat the camera. The worst you can do is be a totally neutral candidate — we've had an interview, but I can't remember any details. So try and sneak some anecdote, or wear a silly scarf — something to remember. This point is especially important for intern / junior positions — online JS bootcamps do a good job of covering the basics, and it's really hard to differentiate these candidates. The memorable thing doesn't have to be professional, or even positive (even though it sure won't hurt) — your best bet would be some original personal project. Ask for feedback on the spot. Asking how you did at the end of the interview doesn't hurt. Yes, some interviewers will be hesitant to answer — at large companies, the feedback is normally sent through the recruiter, and you're never sure if sidestepping this process would get you into trouble. Besides, if the feedback is not complimentary, you're essentially asking for conflict at the spot, and people normally avoid conflict when possible. Still, it's a chance to adjust your expectations (if the interviewer says, looking you in the eyes, that you've done great, it's a good sign), and you might get actually useful tips that would probably get lost passing through the written report, and then through the non-technical recruiter.

a year ago 18 votes
Svelte stores: the curious parts

We've already learnt a lot about svelte's reactivity system — the primary way to work with state in svelte components. But not all state belongs in components — sometimes we want app-global state (think state manager), sometimes we just want to reuse logic between components. React has hooks, Vue has composables. For svelte, the problem is even harder — reactive state only works inside component files, so the rest is handled by a completely separate mechanism — stores. The tutorial does a decent job of covering the common use cases, but I still had questions: What's the relationship between the stores? Are they built on some common base? Is it safe to use { set } = store as a free function? How does get(store) receive the current value if it's not exposed on the object? Does set() trigger subscribers when setting the current value? What's the order of subscriber calls if you set() inside a subscriber? Does derived listen to the base stores when it's not observed? Will changing two dervied dependencies trigger one or two derived computations? Why does subscribe() have a second argument? What is $store sytax compiled to? In this article, I explore all these questions (and find a few svelte bugs in the process). writable is the mother store Svelte has 3 built-in store types: writable, readable, and derived. However, they are neatly implemented in terms of one another, taking only 236 lines, over half of which is TS types and comments. The implementation of readable is remarkably simple — it creates a writable, and only returns its subscribe method. Let me show it in its entirety: const readable = (value, start) => ({ subscribe: writable(value, start).subscribe }); Moreover, derived is just a special way of constructing readable: export function derived(stores, fn, initial_value) { // ...some normalization return readable(initial_value, /* some complex code */); } While we're at it, note that update method of a writable store is a very thin wrapper over set: fn => set(fn(value)). All in all: writable is the OG store, readable just removes set & update methods from a writable, derived is just a predefined readable setup, update is just a wrapper over set. This greatly simplifies our analysis — we can just investigate writable arguments, subscribe, and set — and our findings also hold for other store types. Well done, svelte! Store methods don't rely on this Writable (and, by extension, readable and derived) is implemented with objects and closures, and does not rely on this, so you can safely pass free methods around without dancing with bind: const { subscribe, set } = writable(false); const toggle = { subscribe, activate: () => set(true) }; However, arbitrary custom stores are not guaranteed to have this trait, so it's best to stay safe working with an unknown store-shaped argument — like svelte itself does with readonly: function readonly(store) { return { subscribe: store.subscribe.bind(store), }; } Subscriber is invoked immediately As svelte stores implement observable value pattern, you'd expect them to have a way to access current value via store.get() or store.value — but it's not there! Instead, you use the special get() helper function: import { get } from 'svelte/store' const value = get(store); But, if the store does not expose a value, how can get(store) synchronously access it? Normally, the subscribers are only called on change, which can occur whenever. Well, svelte subscribe is not your average subscribe — calling subscribe(fn) not only starts listening to changes, but also synchronously calls fn with the current value. get subscribes to the store, extracts the value from this immediate invocation, and immediately unsubscribes — like this: let value; const unsub = store.subscribe(v => value = v); unsub(); The official svelte tutorial section on custom stores says: as long as an object correctly implements the subscribe method, it's a store. This might bait you into writing "custom stores" with subscribe method, not based off of writable. The trick word here is correctly implements — even based on the tricky subscribe self-invocation it's not an easy feat, so please stick to manipulations with readable / writable / derived. set() is pure for primitives writable stores are pure in the same sense as svelte state — notifications are skipped when state is primitive, and the next value is equal to the current one: const s = writable(9); // logs 9 because immediate self-invocation s.subscribe(console.log); // does not log s.set(9); Object state disables this optimization — you can pass a shallow equal object, or the same (by reference) object, the subscribers will be called in any case: const s = writable({ value: 9 }); s.subscribe(console.log); // each one logs s.update(s => s); s.set(get(s)); s.set({ value: 9 }); On the bright side, you can mutate the state in update, and it works: s.update(s => { s.value += 1; return s }); Subscriber consistency Normally, store.set(value) synchronously calls all subscribers with value. However, a naive implementation will shoot you in the foot when updating a store from within a subscriber (if you think it's a wild corner case — it's not, it's how derived stores work): let currentValue = null; const store = naiveWritable(1); store.subscribe(v => { // let's try to avoid 0 if (v === 0) store.set(1); }) store.subscribe(v => currentValue = v); If we now call set(0), we intuitively expect both the store's internal value and currentValue to be 1 after all callbacks settle. But in practice it can fail: Store value becomes 0; First subscriber sees 0, calls set(1), then: Store value becomes 1; set(1) synchronously invokes all subscribers with 1; First subscriber sees 1, does nothing; Second subscriber is called with 1, sets currentValue to 1; First subscriber run for 0 is completed, continuing with the initial updates triggered by set(0) Second subscriber is called with 0, setting currentValue to 0; Bang, inconsistent state! This is very dangerous territory — you're bound to either skip some values, get out-of-order updates, or have subscribers called with different values. Rich Harris has taken a lot of effort to provide the following guarantees, regardless of where you set the value: Every subscriber always runs for every set() call (corrected for primitive purity). Subscribers for one set() run, uninterrupted, after one another (in insertion order, but I wouldn't rely on this too much). Subscribers are invoked globally (across all svelte stores) in the same order as set calls, even when set calls are nested (called from within a subscriber). All subscribers are called synchronously within the outermost set call (the one outside any subscriber). So, in our example, the actual callback order is: subscriber 1 sees 0, calls set(1) subscribers calls with 1 are enqueued subscriber 2 sets currentValue = 0 subscriber 1 runs with 1, does nothing subscriber 2 sets currentValue = 1 Since the callback queue is global, this holds even when updating store B from a subscriber to store A. One more reason to stick with svelte built-in stores instead of rolling your own. Derived is lazy derived looks simple on the surface — I thought it just subscribes to all the stores passed, and keeps an up-to-date result of the mapper function. In reality, it's smarter than that — subscription and unsubscription happens in the start / stop handler, which yields some nice properties: Subscriptions to base stores are automatically removed once you stop listening to the derived store, no leaks. Derived value and subscriptions are reused no matter how many times you subscribe to a derived store. When nobody is actively listening to a derived store, the mapper does not run. The value is automatically updated when someone first subscribes to the derived store (again, courtesy of subscribe self-invocation). Very, very tastefully done. Derived is not transactional While lazy, derived is not transactional, and not batched — synchronously changing 2 dependencies will trigger 2 derivations, and 2 subscriber calls — one after the first update, and one after the second one. In this code sample, we'd expect left + right to always be 200 (we synchronously move 10 from left to right), there's a glimpse of 190 (remember, the subscribers are synchronously called during set): const left = writable(100); const right = writable(100); const total = derived([left, right], ([x, y]) => { console.log('derive', x, y); return x + y; }); total.subscribe(t => console.log('total', t)); const update = () => { // try to preserve total = 200 left.update(l => l - 10); // ^^ derives, and logs "total 190" right.update(r => r + 10); // ^^ derives, and logs "total 200" }; This isn't a deal breaker, svelte won't render the intermediate state, but it's something to keep in mind, or you get hurt: const obj = writable({ me: { total: 0 } }); const key = writable('me'); const value = derived([obj, key], ([obj, key]) => obj[key].total); // throws, because { me: ... } has no 'order' field key.set('order'); obj.set({ order: { total: 100 } }); The mysteryous subscriber-invalidator Looking at subscribe() types, you may've noticed the mysterious second argument — invalidate callback. Unlike the subscriber, it's not queued, and is always called synchronously during set(). The only place I've seen an invalidator used in svelte codebase is inside derived — and, TBH, I don't understand its purpose. I expected it to stabilize derived chains, but it's not working. Also, the TS types are wrong — the value is never passed to invalidator as an argument. Verdict: avoid. $-dereference internals As you probably know, svelte components have a special syntax sugar for accessing stores — just prefix the store name with a $, and you can read and even assign it like a regular reactive variable — very convenient: import { writable } from 'svelte/store'; const value = writable(0); const add = () => $value += 1; <button on:click={add}> {$value} </button> I always thought that $value is compiled to get, $value = v to value.set(v), and so on, with a subscriber triggering a re-render in some smart way, but it's not the case. Instead, $value becomes a regular svelte reactive variable, synchronized to the store, and the rest is handled by the standard svelte update mechanism. Here's the compilation result: // the materialized $-variable let $value; // the store const value = writable(0); // auto-subscription const unsub = value.subscribe(value, value => { $$invalidate(0, $value = value) }); onDestroy(unsub); const add = () => { // assign to variable $value += 1; // update store value.set($value); }; In plain English: $store is a real actual svelte reactive variable. store.subscribe updates the variable and triggers re-render. The unsubscriber is stored and called onDestroy. AFAIK, store.update is never used by svelte. Assignments to $store simultaneously mutate $store variable without invalidating and triggering re-render and call store.set, which in turn enqueues the update via $$invalidate The last point puts us in a double-source-of-truth situation: the current store value lives both in the $store reactive variable, and inside store itself. I expected this to cause some havok in an edge case, and so it does — if you patch store.set method to skip some updates, the $-variable updates before your custom set runs, and the two values go out of sync as of svelte@3.59.1: const value = { ...writable(0), // prevent updates set: () => {} }; const add = () => $value += 1; let rerender = {}; $: total = $value + (rerender ? 0 : 1); {total} <button on:click={add}>increment</button> <button on:click={() => rerender = {}}> rerender </button> To summarize: Both readable and derived are built on top of writable — readable only picks subscribe method, derived is a readable with a smart start / stop notifier. Built-in stores don't rely on this, so you can safely use their methods as free functions. Calling subscribe(fn) immediately invokes fn with the current value — used in get(store) to get the current value. Calling set() with the current value of the store will skip notifying subscribers if the value is primitive. set() on object state always notifies, even if the object is same, by reference, as the current state. The subscribers for a single set() run after one another. If a subscriber calls set, this update will be processed once the first set() is fully flushed. derived only subscribes to the base stores and maps the value when someone's actively listening to it. When synchronously changing two dependencies of derived, the mapper will be called after the first change. There's no way to batch these updates. subscribe() has a second argument — a callback that's called synchronously during set(). I can't imagine a use case for it. $store syntax generates a regular svelte reactive variable called $store, and synchronizes it with the store in a subscriber. If you learn one thing from this article — svelte stores are thoughtfully done and help you with quite a few corner-cases. Please avoid excessive trickery, and build on top of the svelte primitives. In the next part of my svelte series, I'll show you some neat tricks with stores — stay tuned on twitter!

a year ago 16 votes

More in programming

Creating static map images with OpenStreetMap, Web Mercator, and Pillow

I’ve been working on a project where I need to plot points on a map. I don’t need an interactive or dynamic visualisation – just a static map with coloured dots for each coordinate. I’ve created maps on the web using Leaflet.js, which load map data from OpenStreetMap (OSM) and support zooming and panning – but for this project, I want a standalone image rather than something I embed in a web page. I want to put in coordinates, and get a PNG image back. This feels like it should be straightforward. There are lots of Python libraries for data visualisation, but it’s not an area I’ve ever explored in detail. I don’t know how to use these libraries, and despite trying I couldn’t work out how to accomplish this seemingly simple task. I made several attempts with libraries like matplotlib and plotly, but I felt like I was fighting the tools. Rather than persist, I wrote my own solution with “lower level” tools. The key was a page on the OpenStreetMap wiki explaining how to convert lat/lon coordinates into the pixel system used by OSM tiles. In particular, it allowed me to break the process into two steps: Get a “base map” image that covers the entire world Convert lat/lon coordinates into xy coordinates that can be overlaid on this image Let’s go through those steps. Get a “base map” image that covers the entire world Let’s talk about how OpenStreetMap works, and in particular their image tiles. If you start at the most zoomed-out level, OSM represents the entire world with a single 256×256 pixel square. This is the Web Mercator projection, and you don’t get much detail – just a rough outline of the world. We can zoom in, and this tile splits into four new tiles of the same size. There are twice as many pixels along each edge, and each tile has more detail. Notice that country boundaries are visible now, but we can’t see any names yet. We can zoom in even further, and each of these tiles split again. There still aren’t any text labels, but the map is getting more detailed and we can see small features that weren’t visible before. You get the idea – we could keep zooming, and we’d get more and more tiles, each with more detail. This tile system means you can get detailed information for a specific area, without loading the entire world. For example, if I’m looking at street information in Britain, I only need the detailed tiles for that part of the world. I don’t need the detailed tiles for Bolivia at the same time. OpenStreetMap will only give you 256×256 pixels at a time, but we can download every tile and stitch them together, one-by-one. Here’s a Python script that enumerates all the tiles at a particular zoom level, downloads them, and uses the Pillow library to combine them into a single large image: #!/usr/bin/env python3 """ Download all the map tiles for a particular zoom level from OpenStreetMap, and stitch them into a single image. """ import io import itertools import httpx from PIL import Image zoom_level = 2 width = 256 * 2**zoom_level height = 256 * (2**zoom_level) im = Image.new("RGB", (width, height)) for x, y in itertools.product(range(2**zoom_level), range(2**zoom_level)): resp = httpx.get(f"https://tile.openstreetmap.org/{zoom_level}/{x}/{y}.png", timeout=50) resp.raise_for_status() im_buffer = Image.open(io.BytesIO(resp.content)) im.paste(im_buffer, (x * 256, y * 256)) out_path = f"map_{zoom_level}.png" im.save(out_path) print(out_path) The higher the zoom level, the more tiles you need to download, and the larger the final image will be. I ran this script up to zoom level 6, and this is the data involved: Zoom level Number of tiles Pixels File size 0 1 256×256 17.1 kB 1 4 512×512 56.3 kB 2 16 1024×1024 155.2 kB 3 64 2048×2048 506.4 kB 4 256 4096×4096 2.7 MB 5 1,024 8192×8192 13.9 MB 6 4,096 16384×16384 46.1 MB I can just about open that zoom level 6 image on my computer, but it’s struggling. I didn’t try opening zoom level 7 – that includes 16,384 tiles, and I’d probably run out of memory. For most static images, zoom level 3 or 4 should be sufficient – I ended up a base map from zoom level 4 for my project. It takes a minute or so to download all the tiles from OpenStreetMap, but you only need to request it once, and then you have a static image you can use again and again. This is a particularly good approach if you want to draw a lot of maps. OpenStreetMap is provided for free, and we want to be a respectful user of the service. Downloading all the map tiles once is more efficient than making repeated requests for the same data. Overlay lat/lon coordinates on this base map Now we have an image with a map of the whole world, we need to overlay our lat/lon coordinates as points on this map. I found instructions on the OpenStreetMap wiki which explain how to convert GPS coordinates into a position on the unit square, which we can in turn add to our map. They outline a straightforward algorithm, which I implemented in Python: import math def convert_gps_coordinates_to_unit_xy( *, latitude: float, longitude: float ) -> tuple[float, float]: """ Convert GPS coordinates to positions on the unit square, which can be plotted on a Web Mercator projection of the world. This expects the coordinates to be specified in **degrees**. The result will be (x, y) coordinates: - x will fall in the range (0, 1). x=0 is the left (180° west) edge of the map. x=1 is the right (180° east) edge of the map. x=0.5 is the middle, the prime meridian. - y will fall in the range (0, 1). y=0 is the top (north) edge of the map, at 85.0511 °N. y=1 is the bottom (south) edge of the map, at 85.0511 °S. y=0.5 is the middle, the equator. """ # This is based on instructions from the OpenStreetMap Wiki: # https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Example:_Convert_a_GPS_coordinate_to_a_pixel_position_in_a_Web_Mercator_tile # (Retrieved 16 January 2025) # Convert the coordinate to the Web Mercator projection # (https://epsg.io/3857) # # x = longitude # y = arsinh(tan(latitude)) # x_webm = longitude y_webm = math.asinh(math.tan(math.radians(latitude))) # Transform the projected point onto the unit square # # x = 0.5 + x / 360 # y = 0.5 - y / 2π # x_unit = 0.5 + x_webm / 360 y_unit = 0.5 - y_webm / (2 * math.pi) return x_unit, y_unit Their documentation includes a worked example using the coordinates of the Hachiko Statue. We can run our code, and check we get the same results: >>> convert_gps_coordinates_to_unit_xy(latitude=35.6590699, longitude=139.7006793) (0.8880574425, 0.39385379958274735) Most users of OpenStreetMap tiles will use these unit positions to select the tiles they need, and then dowload those images – but we can also position these points directly on the global map. I wrote some more Pillow code that converts GPS coordinates to these unit positions, scales those unit positions to the size of the entire map, then draws a coloured circle at each point on the map. Here’s the code: from PIL import Image, ImageDraw gps_coordinates = [ # Hachiko Memorial Statue in Tokyo {"latitude": 35.6590699, "longitude": 139.7006793}, # Greyfriars Bobby in Edinburgh {"latitude": 55.9469224, "longitude": -3.1913043}, # Fido Statue in Tuscany {"latitude": 43.955101, "longitude": 11.388186}, ] im = Image.open("base_map.png") draw = ImageDraw.Draw(im) for coord in gps_coordinates: x, y = convert_gps_coordinates_to_unit_xy(**coord) radius = 32 draw.ellipse( [ x * im.width - radius, y * im.height - radius, x * im.width + radius, y * im.height + radius, ], fill="red", ) im.save("map_with_dots.png") and here’s the map it produces: The nice thing about writing this code in Pillow is that it’s a library I already know how to use, and so I can customise it if I need to. I can change the shape and colour of the points, or crop to specific regions, or add text to the image. I’m sure more sophisticated data visualisation libraries can do all this, and more – but I wouldn’t know how. The downside is that if I need more advanced features, I’ll have to write them myself. I’m okay with that – trading sophistication for simplicity. I didn’t need to learn a complex visualization library – I was able to write code I can read and understand. In a world full of AI-generating code, writing something I know I understand feels more important than ever. [If the formatting of this post looks odd in your feed reader, visit the original article]

19 hours ago 4 votes
AI: Where in the Loop Should Humans Go?

This is a re-publishing of a blog post I originally wrote for work, but wanted on my own blog as well. AI is everywhere, and its impressive claims are leading to rapid adoption. At this stage, I’d qualify it as charismatic technology—something that under-delivers on what it promises, but promises so much that the industry still leverages it because we believe it will eventually deliver on these claims. This is a known pattern. In this post, I’ll use the example of automation deployments to go over known patterns and risks in order to provide you with a list of questions to ask about potential AI solutions. I’ll first cover a short list of base assumptions, and then borrow from scholars of cognitive systems engineering and resilience engineering to list said criteria. At the core of it is the idea that when we say we want humans in the loop, it really matters where in the loop they are. My base assumptions The first thing I’m going to say is that we currently do not have Artificial General Intelligence (AGI). I don’t care whether we have it in 2 years or 40 years or never; if I’m looking to deploy a tool (or an agent) that is supposed to do stuff to my production environments, it has to be able to do it now. I am not looking to be impressed, I am looking to make my life and the system better. Another mechanism I want you to keep in mind is something called the context gap. In a nutshell, any model or automation is constructed from a narrow definition of a controlled environment, which can expand as it gains autonomy, but remains limited. By comparison, people in a system start from a broad situation and narrow definitions down and add constraints to make problem-solving tractable. One side starts from a narrow context, and one starts from a wide one—so in practice, with humans and machines, you end up seeing a type of teamwork where one constantly updates the other: The optimal solution of a model is not an optimal solution of a problem unless the model is a perfect representation of the problem, which it never is.  — Ackoff (1979, p. 97) Because of that mindset, I will disregard all arguments of “it’s coming soon” and “it’s getting better real fast” and instead frame what current LLM solutions are shaped like: tools and automation. As it turns out, there are lots of studies about ergonomics, tool design, collaborative design, where semi-autonomous components fit into sociotechnical systems, and how they tend to fail. Additionally, I’ll borrow from the framing used by people who study joint cognitive systems: rather than looking only at the abilities of what a single person or tool can do, we’re going to look at the overall performance of the joint system. This is important because if you have a tool that is built to be operated like an autonomous agent, you can get weird results in your integration. You’re essentially building an interface for the wrong kind of component—like using a joystick to ride a bicycle. This lens will assist us in establishing general criteria about where the problems will likely be without having to test for every single one and evaluate them on benchmarks against each other. Questions you'll want to ask The following list of questions is meant to act as reminders—abstracting away all the theory from research papers you’d need to read—to let you think through some of the important stuff your teams should track, whether they are engineers using code generation, SREs using AIOps, or managers and execs making the call to adopt new tooling. Are you better even after the tool is taken away? An interesting warning comes from studying how LLMs function as learning aides. The researchers found that people who trained using LLMs tended to fail tests more when the LLMs were taken away compared to people who never studied with them, except if the prompts were specifically (and successfully) designed to help people learn. Likewise, it’s been known for decades that when automation handles standard challenges, the operators expected to take over when they reach their limits end up worse off and generally require more training to keep the overall system performant. While people can feel like they’re getting better and more productive with tool assistance, it doesn’t necessarily follow that they are learning or improving. Over time, there’s a serious risk that your overall system’s performance will be limited to what the automation can do—because without proper design, people keeping the automation in check will gradually lose the skills they had developed prior. Are you augmenting the person or the computer? Traditionally successful tools tend to work on the principle that they improve the physical or mental abilities of their operator: search tools let you go through more data than you could on your own and shift demands to external memory, a bicycle more effectively transmits force for locomotion, a blind spot alert on your car can extend your ability to pay attention to your surroundings, and so on. Automation that augments users therefore tends to be easier to direct, and sort of extends the person’s abilities, rather than acting based on preset goals and framing. Automation that augments a machine tends to broaden the device’s scope and control by leveraging some known effects of their environment and successfully hiding them away. For software folks, an autoscaling controller is a good example of the latter. Neither is fundamentally better nor worse than the other—but you should figure out what kind of automation you’re getting, because they fail differently. Augmenting the user implies that they can tackle a broader variety of challenges effectively. Augmenting the computers tends to mean that when the component reaches its limits, the challenges are worse for the operator. Is it turning you into a monitor rather than helping build an understanding? If your job is to look at the tool go and then say whether it was doing a good or bad job (and maybe take over if it does a bad job), you’re going to have problems. It has long been known that people adapt to their tools, and automation can create complacency. Self-driving cars that generally self-drive themselves well but still require a monitor are not effectively monitored. Instead, having AI that supports people or adds perspectives to the work an operator is already doing tends to yield better long-term results than patterns where the human learns to mostly delegate and focus elsewhere. (As a side note, this is why I tend to dislike incident summarizers. Don’t make it so people stop trying to piece together what happened! Instead, I prefer seeing tools that look at your summaries to remind you of items you may have forgotten, or that look for linguistic cues that point to biases or reductive points of view.) Does it pigeonhole what you can look at? When evaluating a tool, you should ask questions about where the automation lands: Does it let you look at the world more effectively? Does it tell you where to look in the world? Does it force you to look somewhere specific? Does it tell you to do something specific? Does it force you to do something? This is a bit of a hybrid between “Does it extend you?” and “Is it turning you into a monitor?” The five questions above let you figure that out. As the tool becomes a source of assertions or constraints (rather than a source of information and options), the operator becomes someone who interacts with the world from inside the tool rather than someone who interacts with the world with the tool’s help. The tool stops being a tool and becomes a representation of the whole system, which means whatever limitations and internal constraints it has are then transmitted to your users. Is it a built-in distraction? People tend to do multiple tasks over many contexts. Some automated systems are built with alarms or alerts that require stealing someone’s focus, and unless they truly are the most critical thing their users could give attention to, they are going to be an annoyance that can lower the effectiveness of the overall system. What perspectives does it bake in? Tools tend to embody a given perspective. For example, AIOps tools that are built to find a root cause will likely carry the conceptual framework behind root causes in their design. More subtly, these perspectives are sometimes hidden in the type of data you get: if your AIOps agent can only see alerts, your telemetry data, and maybe your code, it will rarely be a source of suggestions on how to improve your workflows because that isn’t part of its world. In roles that are inherently about pulling context from many disconnected sources, how on earth is automation going to make the right decisions? And moreover, who’s accountable for when it makes a poor decision on incomplete data? Surely not the buyer who installed it! This is also one of the many ways in which automation can reinforce biases—not just based on what is in its training data, but also based on its own structure and what inputs were considered most important at design time. The tool can itself become a keyhole through which your conclusions are guided. Is it going to become a hero? A common trope in incident response is heroes—the few people who know everything inside and out, and who end up being necessary bottlenecks to all emergencies. They can’t go away for vacation, they’re too busy to train others, they develop blind spots that nobody can fix, and they can’t be replaced. To avoid this, you have to maintain a continuous awareness of who knows what, and crosstrain each other to always have enough redundancy. If you have a team of multiple engineers and you add AI to it, having it do all of the tasks of a specific kind means it becomes a de facto hero to your team. If that’s okay, be aware that any outages or dysfunction in the AI agent would likely have no practical workaround. You will essentially have offshored part of your ops. Do you need it to be perfect? What a thing promises to be is never what it is—otherwise AWS would be enough, and Kubernetes would be enough, and JIRA would be enough, and the software would work fine with no one needing to fix things. That just doesn’t happen. Ever. Even if it’s really, really good, it’s gonna have outages and surprises, and it’ll mess up here and there, no matter what it is. We aren’t building an omnipotent computer god, we’re building imperfect software. You’ll want to seriously consider whether the tradeoffs you’d make in terms of quality and cost are worth it, and this is going to be a case-by-case basis. Just be careful not to fix the problem by adding a human in the loop that acts as a monitor! Is it doing the whole job or a fraction of it? We don’t notice major parts of our own jobs because they feel natural. A classic pattern here is one of AIs getting better at diagnosing patients, except the benchmarks are usually run on a patient chart where most of the relevant observations have already been made by someone else. Similarly, we often see AI pass a test with flying colors while it still can’t be productive at the job the test represents. People in general have adopted a model of cognition based on information processing that’s very similar to how computers work (get data in, think, output stuff, rinse and repeat), but for decades, there have been multiple disciplines that looked harder at situated work and cognition, moving past that model. Key patterns of cognition are not just in the mind, but are also embedded in the environment and in the interactions we have with each other. Be wary of acquiring a solution that solves what you think the problem is rather than what it actually is. We routinely show we don’t accurately know the latter. What if we have more than one? You probably know how straightforward it can be to write a toy project on your own, with full control of every refactor. You probably also know how this stops being true as your team grows. As it stands today, a lot of AI agents are built within a snapshot of the current world: one or few AI tools added to teams that are mostly made up of people. By analogy, this would be like everyone selling you a computer assuming it were the first and only electronic device inside your household. Problems arise when you go beyond these assumptions: maybe AI that writes code has to go through a code review process, but what if that code review is done by another unrelated AI agent? What happens when you get to operations and common mode failures impact components from various teams that all have agents empowered to go fix things to the best of their ability with the available data? Are they going to clash with people, or even with each other? Humans also have that ability and tend to solve it via processes and procedures, explicit coordination, announcing what they’ll do before they do it, and calling upon each other when they need help. Will multiple agents require something equivalent, and if so, do you have it in place? How do they cope with limited context? Some changes that cause issues might be safe to roll back, some not (maybe they include database migrations, maybe it is better to be down than corrupting data), and some may contain changes that rolling back wouldn’t fix (maybe the workload is controlled by one or more feature flags). Knowing what to do in these situations can sometimes be understood from code or release notes, but some situations can require different workflows involving broader parts of the organization. A risk of automation without context is that if you have situations where waiting or doing little is the best option, then you’ll need to either have automation that requires input to act, or a set of actions to quickly disable multiple types of automation as fast as possible. Many of these may exist at the same time, and it becomes the operators’ jobs to not only maintain their own context, but also maintain a mental model of the context each of these pieces of automation has access to. The fancier your agents, the fancier your operators’ understanding and abilities must be to properly orchestrate them. The more surprising your landscape is, the harder it can become to manage with semi-autonomous elements roaming around. After an outage or incident, who does the learning and who does the fixing? One way to track accountability in a system is to figure out who ends up having to learn lessons and change how things are done. It’s not always the same people or teams, and generally, learning will happen whether you want it or not. This is more of a rhetorical question right now, because I expect that in most cases, when things go wrong, whoever is expected to monitor the AI tool is going to have to steer it in a better direction and fix it (if they can); if it can’t be fixed, then the expectation will be that the automation, as a tool, will be used more judiciously in the future. In a nutshell, if the expectation is that your engineers are going to be doing the learning and tweaking, your AI isn’t an independent agent—it’s a tool that cosplays as an independent agent. Do what you will—just be mindful All in all, none of the above questions flat out say you should not use AI, nor where exactly in the loop you should put people. The key point is that you should ask that question and be aware that just adding whatever to your system is not going to substitute workers away. It will, instead, transform work and create new patterns and weaknesses. Some of these patterns are known and well-studied. We don’t have to go rushing to rediscover them all through failures as if we were the first to ever automate something. If AI ever gets so good and so smart that it’s better than all your engineers, it won’t make a difference whether you adopt it only once it’s good. In the meanwhile, these things do matter and have real impacts, so please design your systems responsibly. If you’re interested to know more about the theoretical elements underpinning this post, the following references—on top of whatever was already linked in the text—might be of interest: Books: Joint Cognitive Systems: Foundations of Cognitive Systems Engineering by Erik Hollnagel Joint Cognitive Systems: Patterns in Cognitive Systems Engineering by David D. Woods Cognition in the Wild by Edwin Hutchins Behind Human Error by David D. Woods, Sydney Dekker, Richard Cook, Leila Johannesen, Nadine Sarter Papers: Ironies of Automation by Lisanne Bainbridge The French-Speaking Ergonomists’ Approach to Work Activity by Daniellou How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control by Nadine Sarter Can We Ever Escape from Data Overload? A Cognitive Systems Diagnosis by David D. Woods Ten Challenges for Making Automation a “Team Player” in Joint Human-Agent Activity by Gary Klein and David D. Woods MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination by Sidney Dekker Managing the Hidden Costs of Coordination by Laura Maguire Designing for Expertise by David D. Woods The Impact of Generative AI on Critical Thinking by Lee et al.

16 hours ago 4 votes
whippet lab notebook: untagged mallocs, bis

Earlier this weekGuileWhippet But now I do! Today’s note is about how we can support untagged allocations of a few different kinds in Whippet’s .mostly-marking collector Why bother supporting untagged allocations at all? Well, if I had my way, I wouldn’t; I would just slog through Guile and fix all uses to be tagged. There are only a finite number of use sites and I could get to them all in a month or so. The problem comes for uses of from outside itself, in C extensions and embedding programs. These users are loathe to adapt to any kind of change, and garbage-collection-related changes are the worst. So, somehow, we need to support these users if we are not to break the Guile community.scm_gc_malloclibguile The problem with , though, is that it is missing an expression of intent, notably as regards tagging. You can use it to allocate an object that has a tag and thus can be traced precisely, or you can use it to allocate, well, anything else. I think we will have to add an API for the tagged case and assume that anything that goes through is requesting an untagged, conservatively-scanned block of memory. Similarly for : you could be allocating a tagged object that happens to not contain pointers, or you could be allocating an untagged array of whatever. A new API is needed there too for pointerless untagged allocations.scm_gc_mallocscm_gc_mallocscm_gc_malloc_pointerless Recall that the mostly-marking collector can be built in a number of different ways: it can support conservative and/or precise roots, it can trace the heap precisely or conservatively, it can be generational or not, and the collector can use multiple threads during pauses or not. Consider a basic configuration with precise roots. You can make tagged pointerless allocations just fine: the trace function for that tag is just trivial. You would like to extend the collector with the ability to make pointerless allocations, for raw data. How to do this?untagged Consider first that when the collector goes to trace an object, it can’t use bits inside the object to discriminate between the tagged and untagged cases. Fortunately though . Of those 8 bits, 3 are used for the mark (five different states, allowing for future concurrent tracing), two for the , one to indicate whether the object is pinned or not, and one to indicate the end of the object, so that we can determine object bounds just by scanning the metadata byte array. That leaves 1 bit, and we can use it to indicate untagged pointerless allocations. Hooray!the main space of the mostly-marking collector has one metadata byte for each 16 bytes of payloadprecise field-logging write barrier However there is a wrinkle: when Whippet decides the it should evacuate an object, it tracks the evacuation state in the object itself; the embedder has to provide an implementation of a , allowing the collector to detect whether an object is forwarded or not, to claim an object for forwarding, to commit a forwarding pointer, and so on. We can’t do that for raw data, because all bit states belong to the object, not the collector or the embedder. So, we have to set the “pinned” bit on the object, indicating that these objects can’t move.little state machine We could in theory manage the forwarding state in the metadata byte, but we don’t have the bits to do that currently; maybe some day. For now, untagged pointerless allocations are pinned. You might also want to support untagged allocations that contain pointers to other GC-managed objects. In this case you would want these untagged allocations to be scanned conservatively. We can do this, but if we do, it will pin all objects. Thing is, conservative stack roots is a kind of a sweet spot in language run-time design. You get to avoid constraining your compiler, you avoid a class of bugs related to rooting, but you can still support compaction of the heap. How is this, you ask? Well, consider that you can move any object for which we can precisely enumerate the incoming references. This is trivially the case for precise roots and precise tracing. For conservative roots, we don’t know whether a given edge is really an object reference or not, so we have to conservatively avoid moving those objects. But once you are done tracing conservative edges, any live object that hasn’t yet been traced is fair game for evacuation, because none of its predecessors have yet been visited. But once you add conservatively-traced objects back into the mix, you don’t know when you are done tracing conservative edges; you could always discover another conservatively-traced object later in the trace, so you have to pin everything. The good news, though, is that we have gained an easier migration path. I can now shove Whippet into Guile and get it running even before I have removed untagged allocations. Once I have done so, I will be able to allow for compaction / evacuation; things only get better from here. Also as a side benefit, the mostly-marking collector’s heap-conservative configurations are now faster, because we have metadata attached to objects which allows tracing to skip known-pointerless objects. This regains an optimization that BDW has long had via its , used in Guile since time out of mind.GC_malloc_atomic With support for untagged allocations, I think I am finally ready to start getting Whippet into Guile itself. Happy hacking, and see you on the other side! inside and outside on intent on data on slop fin

18 hours ago 2 votes
AMD YOLO

AMD is sending us the two MI300X boxes we asked for. They are in the mail. It took a bit, but AMD passed my cultural test. I now believe they aren’t going to shoot themselves in the foot on software, and if that’s true, there’s absolutely no reason they should be worth 1/16th of NVIDIA. CUDA isn’t really the moat people think it is, it was just an early ecosystem. tiny corp has a fully sovereign AMD stack, and soon we’ll port it to the MI300X. You won’t even have to use tinygrad proper, tinygrad has a torch frontend now. Either NVIDIA is super overvalued or AMD is undervalued. If the petaflop gets commoditized (tiny corp’s mission), the current situation doesn’t make any sense. The hardware is similar, AMD even got the double throughput Tensor Cores on RDNA4 (NVIDIA artificially halves this on their cards, soon they won’t be able to). I’m betting on AMD being undervalued, and that the demand for AI has barely started. With good software, the MI300X should outperform the H100. In for a quarter million. Long term. It can always dip short term, but check back in 5 years.

16 hours ago 2 votes
Introducing the blogroll

This website has a new section: blogroll.opml! A blogroll is a list of blogs - a lightweight way of people recommending other people’s writing on the indieweb. What it includes The blogs that I included are just sampled from my many RSS subscriptions that I keep in my Feedbin reader. I’m subscribed to about 200 RSS feeds, the majority of which are dead or only publish once a year. I like that about blogs, that there’s no expectation of getting a post out every single day, like there is in more algorithmically-driven media. If someone who I interacted with on the internet years ago decides to restart their writing, that’s great! There’s no reason to prune all the quiet feeds. The picks are oriented toward what I’m into: niches, blogs that have a loose topic but don’t try to be general-interest, people with distinctive writing. If you import all of the feeds into your RSS reader, you’ll probably end up unsubscribing from some of them because some of the experimental electric guitar design or bonsai news is not what you’re into. Seems fine, or you’ll discover a new interest! How it works Ruben Schade figured out a brilliant way to show blogrolls and I copied him. Check out his post on styling OPML and RSS with XSLT to XHTML for how it works. My only additions to that scheme were making the blogroll page blend into the rest of the website by using an include tag with Jekyll to add the basic site skeleton, and adding a link with the download attribute to provide a simple way to download the OPML file. Oddly, if you try to save the OPML page using Save as… in Firefox, Firefox will save the transformed output via the XSLT, rather than the raw source code. XSLT is such an odd and rare part of the web ecosystem, I had to use it.

yesterday 1 votes