Queues are everywhere, and they follow mathematical rules. Learn a few of those rules! It'll go a long way to making you a stronger SRE.

Here’s a story from nearly 10 years ago. the bug I think it was my friend Richard Kettlewell who told me about a bug he encountered with Let’s Encrypt in its early days in autumn 2015: it was failing to validate mail domains correctly. the context At the time I had previously been responsible for Cambridge University’s email anti-spam system for about 10 years, and in 2014 I had been given responsibility for Cambridge University’s DNS. So I knew how Let’s Encrypt should validate mail domains. Let’s Encrypt was about one year old. Unusually, the code that runs their operations, Boulder, is free software and open to external contributors. Boulder is written in Golang, and I had not previously written any code in Golang. But its reputation is to be easy to get to grips with. So, in principle, the bug was straightforward for me to fix. How difficult would it be as a Golang newbie? And what would Let’s Encrypt’s contribution process be like? the hack I cloned the Boulder repository and had a look around the code. As is pretty typical, there are a couple of stages to fixing a bug in an unfamiliar codebase: work out where the problem is try to understand if the obvious fix could be better In this case, I remember discovering a relatively substantial TODO item that intersected with the bug. I can’t remember the details, but I think there were wider issues with DNS lookups in Boulder. I decided it made sense to fix the immediate problem without getting involved in things that would require discussion with Let’s Encrypt staff. I faffed around with the code and pushed something that looked like it might work. A fun thing about this hack is that I never got a working Boulder test setup on my workstation (or even Golang, I think!) – I just relied on the Let’s Encrypt cloud test setup. The feedback time was very slow, but it was tolerable for a simple one-off change. the fix My pull request was small, +48-14. After a couple of rounds of review and within a few days, it was merged and put into production! A pleasing result. the upshot I thought Golang (at least as it was used in the Boulder codebase) was as easy to get to grips with as promised. I did not touch it again until several years later, because there was no need to, but it seemed fine. I was very impressed by the Let’s Encrypt continuous integration and automated testing setup, and by their low-friction workflow for external contributors. One of my fastest drive-by patches to get into worldwide production. My fix was always going to be temporary, and all trace of it was overwritten years ago. It’s good when “temporary” turns out to be true! the point I was reminded of this story in the pub this evening, and I thought it was worth writing down. It demonstrated to me that Let’s Encrypt really were doing all the good stuff they said they were doing. So thank you to Let’s Encrypt for providing an exemplary service and for giving me a happy little anecdote.

It’s well known that Japan is experiencing a labor shortage that includes the tech industry. As a result, Japan needs more international developers. However, most Japanese companies aren’t interested in hiring new graduates from overseas. While it’s easier than in some countries, bringing a developer to Japan still requires some time and financial investment on the company’s side, which they’re unlikely to expend on junior developers. So if you’re a new grad with no experience, can you still find a job in Japan? The answer is yes, but it will take considerable effort, and the opportunities you find may not be what you’re looking for. If you’re determined, it’s best to start preparing while you’re still a student. If you’ve already graduated, then your road is tougher but not impossible, especially if you’re patient, persistent, and willing to go above and beyond. This article will cover: The new graduate hiring system in Japan How international students in Japan can take advantage of the system The steps students and new grads overseas can take to maximize their chances How new grad hiring works in Japan While the primary audience of this article is graduates outside of Japan, understanding how the domestic hiring process works provides necessary context. The Japanese system for hiring new graduates (新卒一括採用, shinsotsu ikkatsu saiyou) operates quite differently from practices in many other countries. It is, however, an effective method: in 2025, 98 percent of Japanese graduates were able to secure employment. What companies are looking for in new graduates To better understand Japan’s methods for hiring new graduates, it helps to know something about Japanese employment philosophy. Traditional Japanese business practices center on the idea of lifetime employment. This approach is currently changing—as of 2023, one in three Japanese employees were no longer committed to lifetime employment, but were open to changing jobs or working independently if the opportunity arose. Nonetheless, decades of conventional wisdom still shape how Japanese companies approach hiring new graduates. Most new graduate hires are considered members of the company on a “comprehensive career track” (総合職採用, sougoushoku saiyou) which means that they aren’t being hired for a specific job, but instead to work for the company in general. In exchange, the business will train them and identify their strengths by assigning them to various departments and roles. Japanese companies therefore typically aren’t as interested in a candidate’s specific experiences and academic achievements, since they’re not certain what position the candidate will eventually occupy in the company. Instead, they prize qualities like good communication skills, independence, and willingness. This is also why Japanese companies often employ Synthetic Personality Inventory (SPI) tests, to attempt to understand if a candidate will be an “objectively” good fit. SPI tests are generally half knowledge-based (often general knowledge), and half MBTI-style personality tests. The job-hunting schedule In Japan, both the school and fiscal year start in April, and so does the job hunt—the year before graduation. If you wait until you’ve graduated to start job-hunting, you’re already a year behind! Study in Japan, a government-approved information site, outlines the following typical schedule for a university student set to graduate the following year: March–May is the entry period. Go to job fairs, seek company introductions, and request entry submission sheets. If your initial application is accepted, begin taking company examinations. In June, the interviews begin. Expect at least three rounds of interviews, of several different types. From June–September is when you may receive a preliminary offer of employment. In October, if you’re successful, you’ll receive the official offer of employment. If you are an international student in Japan, you should change your status of residence. Some companies do continue recruiting in October, but the bulk of the new grad hiring is completed by then. What to expect when applying The new graduate application process also differs widely from standard hiring practices in other countries. Entry sheet If you’re interested in applying to a company, you must first request and complete an entry sheet, which resembles a university application essay more than your average resume. It frequently includes personality-based questions such as “If you had 300 million Japanese yen, what would you do to promote world peace?” and “Describe how you would be indispensable to our company by referring to what you did best during your school life.” Written Examinations If your entry sheet is approved, you can then move on to take the company’s written examinations. Most of these are aptitude tests that evaluate a candidate’s mathematics, language, and writing ability as well as general knowledge. These are typically produced by a third-party company. Interviews It’s customary for companies to conduct at least three rounds of interviews. Some of those interviews are standard single-person interviews. Others are group interviews, in which the interviewer or panel takes turns asking a group of applicants different questions. This method is typically adopted to save time and help winnow down a large applicant pool. You may also be asked to participate in a group discussion interview, in which a gathering of four to six candidates are asked to discuss a given theme, while an interviewer evaluates the group dynamic and individual performance of the applicants. In particular, interviewers are interested in seeing how participants interact in a team situation. Employment offers If a company is interested in hiring you, they may make a preliminary offer of employment (内々定, nai naitei).These types of offers are informal and are not considered binding under law. The more serious step is when a company issues a formal letter of intent to hire (内定, naitei). It is very common for Japanese companies to provide this to show their serious intent of hiring you while they do their due diligence and prepare to issue a formal contract. Under Japanese labor law, the company might be held liable if they subsequently withdraw this offer without cause. If you accept the letter of intent, you are signaling that you will sign a formal employment contract with the company; if you have received preliminary offers from any other companies, you should call them to politely decline. While it is possible to accept and then renege on a formal letter of intent to hire, you should be aware that it is a major faux pas, and you will likely be blacklisted within the company, depending on the circumstances. If this is unavoidable, you should communicate your intentions as quickly and clearly as possible to the company. Japanese companies generally follow the process described here, especially the formal letter of intent to hire. However, these steps are not required by law, so some companies—especially newer or international ones—may not follow this schedule at all! A company following this flow does not indicate that they are reputable or trustworthy, nor does a company not following this flow indicate that they are engaging in deceptive or illegal practices. If you’re an international student in Japan If you’re an international student in Japan, you can join the annual job hunt in April with the rest of your class. Because every step is typically conducted in Japanese, your success in the conventional process will largely depend on how fluent you are. Yet according to a 2024 survey by employment information firm Career-tasu, Inc., most foreign students started job search activities the year they graduated, rather than the year before as their Japanese peers did. This, the staff at Career-tasu believes, puts international students at a serious detriment. International students are also less likely to participate in internship programs or other job experience activities: only 46.1% of international graduates did so, compared to 88.7% of Japanese students surveyed. Nonetheless, The Japan News by The Yomiuri Shimbun reported that some companies are keen to hire international graduates, particularly those who are bilingual or trilingual. In 2022, the percentage of foreign-born graduates finding jobs in Japan exceeded 50% for the first time. Those numbers may improve still further as the labor shortage in Japan worsens. Only around 36% of companies in 2024 were able to meet their recruiting goals. In 2025, 80% of third-year students undergoing the new graduate hiring process already had job offers before June. Companies are already adapting by focusing on mid-career recruitment and introducing fast-tracked interviews, so they may also be more willing to accommodate international student candidates. Certain cities, prefectural governments, and universities have also committed to supporting international students’ job-hunting, and they can help match you with Japanese companies. Ritsumeikan University, for example, not only offers its own job search engine in Japanese, but also lists a number of English-language job boards. New graduates in Japan can take advantage of these school and government resources, as well as being already in the country and available for interviews, examinations, and more. In addition, changing from a Student visa to one of the work-related visas is relatively simple and doesn’t require getting a Certificate of Eligibility or leaving the country. If you’re a student or new grad overseas New graduates from other countries are at a significant disadvantage. First, you’re not available to interview or take examinations in person, which will disqualify you from most companies’ usual hiring procedures. Second, unlike in most countries, the Japanese school year ends in late March. Even if a Japanese company is willing to accept your application via its normal new-grad hiring program, the mismatch in school schedules means you’d need to wait months before you could start work. There are several ways to overcome these difficulties, though, which we’ll explore below. Japanese language skills It’s not absolutely necessary to learn Japanese to get a developer job in Japan. In fact, many Japanese tech companies are creating multinational, English-speaking development teams, or even adopting English as the company language. There’s a catch, however. As mentioned above, those companies who are willing to hire English-speaking developers from overseas are typically searching for senior developers. If you are a junior developer, or a new graduate without experience, you’re far less likely to find an English-speaking position in Japan. That’s the trade-off: you can get a job with work experience and no Japanese ability, but if you don’t have the work experience, you’d better speak Japanese! If the prospect of learning Japanese just to get a job seems daunting, be aware that not every position requires full Japanese fluency. If you’re studying hard and demonstrate enthusiasm in interviews, then conversational-level Japanese might be enough to persuade a company to take a chance on you. Training and recruitment programs Given the high demand in Japan for new tech talents that also speak Japanese, some recruiting companies are offering fast-tracked training programs for students overseas. FAST OFFER, for example, has relationships with over 50 universities worldwide, primarily in Asia. It provides free Japanese classes, which are worth college credits at some participating universities, for students majoring in select fields. These classes are specifically designed to help candidates pass interviews with Japanese companies, as well as acquire the general language skills needed to work in Japan. Candidates who reach a specified level of proficiency—JLPT N4 for most IT roles—will be assigned a mentor, coached on the interview process, and matched with Japanese companies. If the applicant passes the initial screening, they will even be flown to Japan for in-person interviews, all for free. Another example is xseeds Hub, a recruitment platform that developed its own curriculum to prepare students for employment in Japan. As of now, xseeds Hub runs these programs at four Vietnamese universities, as well as one in Indonesia and one in Malaysia. The curriculum includes lessons on different scripting languages, as well as Japanese classes intended to help students achieve at least a JLPT N3 level of speaking. Students gain practical experience via mock experiment lessons and internships in Japan, which run for a minimum of three months. University collaborations with Japanese companies While you’re still a student, you should also find out if your university offers events or programs with Japanese companies. Mercari and Rakuten have particularly targeted the Indian Institutes of Technology (IITs) for recruitment events. But interest in Indian graduates isn’t limited to those companies; when the Indian Embassy hosted a seminar on the subject in Tokyo, over 100 Japanese firms attended. I spoke with Victoria Astingo, who works in Senior Talent Acquisition for Mercari. Mercari has been visiting IITs since 2018, interviewing and hiring soon-to-be graduates on the spot, and they’ve recently started similar campus recruitment drives in Indonesia. These rapid-hire events only take place at universities in Asia for now, but Astingo is expanding Mercari’s internship program outreach to European universities. Her first event in Europe took place at three universities in France, including EPITA, a top-level French institution specializing in Computer Science and Software Engineering. “The reception was way better than we expected,” she said. “At first we were like, ‘Yeah, nobody [here] knows Mercari. Maybe there will be five people coming to our seminar, five anime fans that want to make it to Japan.’ But then actually for EPITA, we had more than 120 attendees.” Other Japanese companies have also started directly partnering with universities abroad. For example, I-Shou University in Taiwan is setting up internships and hiring channels for students in cooperation with Japanese company Nisso Kogyo Co, Ltd., to help supply new talent for the semiconductor industry in Kumamoto. In Vietnam, Ho Chi Minh Open University cooperated with 22 Japanese companies for its 2024 Japan Job Fair, and HUTECH University of Technology hosted 40 Japanese companies at its event. Even if your university doesn’t have preestablished relationships with Japanese companies, you should still check in with your university’s Japanese or East Asian Studies department. The professors there may have personal connections to Japan, or be able to suggest helpful programs, scholarships, or internships. You could also explore your alumni network, and see if you can make contact with any who are currently working in Japan. Internships at TokyoDev companies “Internship” in Japan doesn’t always mean what you think it means—as TokyoDev contributor Lai Huynh Binh Minh discovered, the average “internship” in Japan only lasts one day! That’s not an especially convenient or helpful timeframe for anyone who is overseas. Luckily, TokyoDev works with several companies that offer longer internships with visa support. Not only can you gain valuable work experience, but you also have the opportunity to live in Japan for a few months and discover if it’s really for you. Mercari An internship at Mercari is only available to current university students. It lasts for two months, is paid hourly, and comes with full visa and financial support. Job offers on completion of the internship are considered to be nearly guaranteed. “Most of the time,” said Victoria Astingo, “students apply while they’re second-year students and do their internships before or during their third year. Then, once they graduate, they join us as full-time employees.” This intern-to-FTE pipeline helps explain why Astingo’s new recruitment drive focuses on Europe rather than Singapore or the US, where new graduates are typically offered higher salaries or better compensation packages. Still, Astingo clarified, students from any country are welcome to apply. “I would say the people who we hire as interns usually have previous internship experience as a software engineer,” she said. “[And] if I’m hiring for frontend or backend, I will always check to see if the candidate shares our tech stack—for example, if the candidate has experience working with React or Golang. They don’t really need to master those, but should at least have a basic understanding or interest.” Team experience, such as participating in a hackathon or some other type of group project, is also something Astingo looks for. The name of the applicant’s university doesn’t carry much weight, but majoring in computer science or a related field does. However, even if a candidate doesn’t meet every requirement, she’ll most likely still send the initial skill assessment and give them the opportunity to prove their abilities. Japanese language skills, on the other hand, are not a requirement. “Most of the people I hired for Mercari Marketplace were not Japanese speakers,” Astingo affirmed. HENNGE Like Mercari’s internship program, HENNGE’s Global Internship Program accepts applicants year-round from all over the world. Also like Mercari, it is considered part of the company’s pre-hiring process, so those applicants who are interested in working at HENNGE after graduation will have an advantage. One difference is that the HENNGE internship is unpaid, though participants do receive a monthly stipend, round-trip airfare, and other benefits. Also, the HENNGE program is intended for third-year university students or those who have newly graduated, and so will accommodate slightly older/more advanced candidates. The internships fall into two distinct categories: the Full Stack Engineering Pathway, and the Frontend Engineering Pathway. Each pathway has its own tech stack and experience requirements. It’s possible to apply for both at the same time, but not to change between them mid-internship. As with Mercari, Japanese language ability is not required, but candidates must be fluent in English. HENNGE’s is one of the few global internship programs offered in Japan, so it’s quite competitive, and applicants should be prepared for a selective process with minimal communication. Working Holiday and J-Find visas If you’ve already graduated, then you still have other avenues to come to Japan to look for work: a Working Holiday visa or J-Find visa. These visas both permit the holder to travel, work, and job-hunt in Japan for up to a year. By giving you one year in Japan to network and interview, these visas significantly increase your odds of landing a permanent position, as many roles will not accept overseas applicants. In fact, that’s how TokyoDev founder Paul McMahon got his first job in Japan, and it’s also worked for other TokyoDev contributors as well. However, both visas have specific requirements, so be sure to check whether you qualify. International job fairs If you’re not eligible for a Working Holiday or J-Find visa, try visiting an international job fair instead. Participating businesses have already expressed their willingness to hire overseas candidates, and you’ll be able to personally meet with—and hopefully impress—company representatives. At least some Japanese ability will be required, however, to take advantage of these opportunities. One of the best-known Japanese career fair organizations is Career Forum, which hosts fairs in Boston, Los Angeles, London, Tokyo, and Osaka. The largest overseas event is in Boston, with over 100 companies slated to participate in 2025. Career Forum’s target audience is candidates who speak both Japanese and English to at least a beginner’s level. While their offerings are aimed at current students or recent graduates, they also are interested in candidates who have lived or studied abroad, even if that was not in Japan. On the other side of the globe, Mynavi is holding fairs in Taiwan, Korea, and Australia, as well as online. Though labeled as “career fairs” on their website, these events are actually described as group interviews, and Japanese language skills are a requirement. The Australian event is tightly restricted to current students and new graduates, but the Taiwan fair is open to all those seeking employment in Japanese companies. In Singapore, Asean Career Fair in Singapore will feature 20 Japanese companies in their 2026 event, though it is unclear at the time of writing which companies those will be. Conclusion If you’re a university student or new graduate who wants to find work in Japan, these are your best options: While you’re a student, study Japanese and apply for internships in Japan. If you’re already studying in Japan, you can use university and government resources to help you succeed in the local new graduate hiring process. If you’re overseas, check to see if your university hosts recruitment events or has other connections with Japanese companies and universities. Students in Asia, particularly, may qualify to join a recruiting company’s training program. Working adults may be able to use a Working Holiday or J-Find visa to come to Japan to job-hunt. Visiting an international job fair outside of Japan could get you an interview with an international-friendly company. Good luck and good hunting! Don’t forget to read TokyoDev’s other articles on software developer salaries in Japan, finding a developer job, passing the resume screening process, etc. You can also connect with other developers by joining the TokyoDev Discord, which has channels for resume review, relocating to Japan, and more.

I’ve long wanted the ability to create custom collections of icons from my icon gallery. Today I can browse collections of icons that share pre-defined metadata (e.g. “Show me all icons tagged as blue”) but I can’t create your own arbitrary collections of icons. That is, until now! I created a page at /lookup that allows you to specify however many id search params you want and it will pull all the matching icons into a single page. Here’s an example of macOS icons that follow the squircle shape but break out of it ever-so-slightly (something we’ll lose with macOS Tahoe). It requires a little know how to construct the URL, something I’ll address later, but it works for my own personal purposes at the moment. So how did I build it? Implementation So the sites are built with a static site generator, but this feature requires an ability to dynamically construct a page based on the icons specified in the URL, e.g. /lookup?id=foo&id=bar&id=baz How do I get that to work? I can’t statically pre-generate every possible combination[1] so what are my options? Create a “shell” page that uses JavaScript to read the search params, query a JSON API, and render whichever icons are specified in the URL. Send an HTML page with all icons over the wire, then use JavaScript to reach into the DOM and remove all icons whose IDs aren’t specified in the page URL. Render the page on the server with just the icons specified in the request URL. No. 1: this is fine, but I don’t have a JSON API for clients to query and I don’t want to create one. Plus I have to duplicate template logic, etc. I’m already rendering lists of icons in my static site generator, so can’t I just do that? Which leads me to: No. 2: this works, but I do have 2000+ icons so the resulting HTML page (I tried it) is almost 2MB if I render everything (whereas that same request for ~4 icons but filtered by the server would be like 11kb). There’s gotta be a way to make that smaller, which leads me to: No. 3: this is great, but it does require I have a “server” to construct pages at request time. Enter Netlify’s Edge Functions which allow you to easily transform an existing HTML page before it gets to the client. To get this working in my case, I: Create /lookup/index.html that has all 2000+ icons on it (trivial with my current static site generator). Create a lookup.ts edge function that intercepts the request to /lookup/index.html Read the search params for the request and get all specified icon IDs, e.g. /lookup?id=a&id=b&id=c turns into ['a','b','c'] Following Netlify’s example of transforming an HTML response, use HTMLRewriter to parse my HTML with all 2000+ icons in it then remove all icons that aren’t in my list of IDs, e.g. <a id='a'>…</a><a id='z'>…</a> might get pruned down to <a id='a'>…</a> Transform the parsed HTML back into a Response and return it to the client from the function. It took me a second to get all the Netlify-specific configurations right (put the function in ./netlify/edge-functions not ./netlify/functions, duh) but once I strictly followed all of Netlify’s rules it was working! (You gotta use their CLI tool to get things working on localhost and test it yourself.) Con-clusions I don’t particularly love that this ties me to a bespoke feature of Netlify’s platform — even though it works really well! But that said, if I ever switched hosts this wouldn’t be too difficult to change. If my new host provided control over the server, nothing changes about the URL for this page (/lookup?id=…). And if I had to move it all to the client, I could do that too. In that sense, I’m tying myself to Netlify from a developer point of view but not from an end-user point of view (everything still works at the URL-level) and I’m good with that trade-off. Just out of curiosity, I asked ChatGPT: if you have approximately 2,000 unique items, how many possible combinations of those IDs can be passed in a URL like /lookup?id=1&id=2? It said the number is 2^2000 which is “astronomically large” and “far more than atoms in the universe”. So statically pre-generating them is out of the question. ⏎ Email · Mastodon · Bluesky

In Chrome Dev Tools you can setup a mapping between the files web server sends to the browser and files on disk. This allows editing files in dev tools and having those changes saved to a file, which is handy. Early in 2025 they’ve added a way to automatically configure this mapping. It’s quite simple. Your server needs to serve /.well-known/appspecific/com.chrome.devtools.json file with content that looks like: { "workspace": { "root": "C:/Users/kjk/src/edna/src", "uuid": "8f6e3d9a-4b7c-4c1e-a2d5-7f9b1e3c1384" } } The uuid should be unique for each project. I got mine by asking Grok to generate random version 4 uuid. On Windows the path has to use Unix slash. It didn’t work when I sent Windows path. Shocking incompetence of the devs. This only works from localhost. Security. This file is read when you open dev tools in chrome. If you already have it open, you have to close and re-open. You still need to connect (annoying security theater). So the first thing you need to do: switch to sources tab in dev tools reveal left side panel (if hidden) switch to workspace tab there you should see the mapping is already configured but you need to click connect button Apparently bun sends this automatically. Here’s how I send it in my Go server: func serveChromeDevToolsJSON(w http.ResponseWriter, r *http.Request) { // your directory might be different than "src" srcDir, err := filepath.Abs("src") must(err) // stupid Chrome doesn't accept windows-style paths srcDir = filepath.ToSlash(srcDir) // uuid should be unique for each workspace uuid := "8f6e3d9a-4b7c-4c1e-a2d5-7f9b0e3c1284" s := `{ "workspace": { "root": "{{root}}", "uuid": "{{uuid}}" } }` s = strings.ReplaceAll(s, "{{root}}", srcDir) s = strings.ReplaceAll(s, "{{uuid}}", uuid) logf("serveChromeDevToolsJSON:\n\n%s\n\n", s) w.Header().Set("Content-Type", "application/json") w.Write(data) }