More from Paolo Amoroso's Journal
<![CDATA[DandeGUI now does graphics and this is what it looks like. Some text and graphics output windows created with DandeGUI on Medley Interlisp. In addition to the square root table text output demo, I created the other graphics windows with the newly implemented functionality. For example, this code draws the random circles of the top window: (DEFUN RANDOM-CIRCLES (&KEY (N 200) (MAX-R 50) (WIDTH 640) (HEIGHT 480)) (LET ((RANGE-X (- WIDTH ( 2 MAX-R))) (RANGE-Y (- HEIGHT ( 2 MAX-R))) (SHADES (LIST IL:BLACKSHADE IL:GRAYSHADE (RANDOM 65536)))) (DANDEGUI:WITH-GRAPHICS-WINDOW (STREAM :TITLE "Random Circles") (DOTIMES (I N) (DECLARE (IGNORE I)) (IL:FILLCIRCLE (+ MAX-R (RANDOM RANGE-X)) (+ MAX-R (RANDOM RANGE-Y)) (RANDOM MAX-R) (ELT SHADES (RANDOM 3)) STREAM))))) GUI:WITH-GRAPHICS-WINDOW, GUI:OPEN-GRAPHICS-STREAM, and GUI:WITH-GRAPHICS-STREAM are the main additions. These functions and macros are the equivalent for graphics of what GUI:WITH-OUTPUT-TO-WINDOW, GUI:OPEN-WINDOW-STREAM, and GUI:WITH-WINDOW-STREAM, respectively, do for text. The difference is the text facilities send output to TEXTSTREAM streams whereas the graphics facilities to IMAGESTREAM, a type of device-independent graphics streams. Under the hood DandeGUI text windows are customized TEdit windows with an associated TEXTSTREAM. TEdit is the rich text editor of Medley Interlisp. Similarly, the graphics windows of DandeGUI run the Sketch line drawing editor under the hood. Sketch windows have an IMAGESTREAM which Interlisp graphics primitives like IL:DRAWLINE and IL:DRAWPOINT accept as an output destination. DandeGUI creates and manages Sketch windows with the type of stream the graphics primitives require. In other words, IMAGESTREAM is to Sketch what TEXTSTREAM is to TEdit. The benefits of programmatically using Sketch for graphics are the same as TEdit windows for text: automatic window repainting, scrolling, and resizing. The downside is overhead. Scrolling more than a few thousand graphics elements is slow and adding even more may crash the system. However, this is an acceptable tradeoff. The new graphics functions and macros work similarly to the text ones, with a few differences. First, DandeGUI now depends on the SKETCH and SKETCH-STREAM library modules which it automatically loads. Since Sketch has no notion of a read-only drawing area GUI:OPEN-GRAPHICS-STREAM achieves the same effect by other means: (DEFUN OPEN-GRAPHICS-STREAM (&KEY (TITLE "Untitled")) "Open a new window and return the associated IMAGESTREAM to send graphics output to. Sets the window title to TITLE if supplied." (LET ((STREAM (IL:OPENIMAGESTREAM '|Untitled| 'IL:SKETCH '(IL:FONTS ,DEFAULT-FONT*))) (WINDOW (IL:\\SKSTRM.WINDOW.FROM.STREAM STREAM))) (IL:WINDOWPROP WINDOW 'IL:TITLE TITLE) ;; Disable left and middle-click title bar menu (IL:WINDOWPROP WINDOW 'IL:BUTTONEVENTFN NIL) ;; Disable sketch editing via right-click actions (IL:WINDOWPROP WINDOW 'IL:RIGHTBUTTONFN NIL) ;; Disable querying the user whether to save changes (IL:WINDOWPROP WINDOW 'IL:DONTQUERYCHANGES T) STREAM)) Only the mouse gestures and commands of the middle-click title bar menu and the right-click menu change the drawing area interactively. To disable these actions GUI:OPEN-GRAPHICS-STREAM removes their menu handlers by setting to NIL the window properties IL:BUTTONEVENTFN and IL:RIGHTBUTTONFN. This way only programmatic output can change the drawing area. The function also sets IL:DONTQUERYCHANGES to T to prevent querying whether to save the changes at window close. By design output to DandeGUI windows is not permanent, so saving isn't necessary. GUI:WITH-GRAPHICS-STREAM and GUI:WITH-GRAPHICS-WINDOW are straightforward: (DEFMACRO WITH-GRAPHICS-STREAM ((VAR STREAM) &BODY BODY) "Perform the operations in BODY with VAR bound to the graphics window STREAM. Evaluates the forms in BODY in a context in which VAR is bound to STREAM which must already exist, then returns the value of the last form of BODY." `(LET ((,VAR ,STREAM)) ,@BODY)) (DEFMACRO WITH-GRAPHICS-WINDOW ((VAR &KEY TITLE) &BODY BODY) "Perform the operations in BODY with VAR bound to a new graphics window stream. Creates a new window titled TITLE if supplied, binds VAR to the IMAGESTREAM associated with the window, and executes BODY in this context. Returns the value of the last form of BODY." `(WITH-GRAPHICS-STREAM (,VAR (OPEN-GRAPHICS-STREAM :TITLE (OR ,TITLE "Untitled"))) ,@BODY)) Unlike GUI:WITH-TEXT-STREAM and GUI:WITH-TEXT-WINDOW, which need to call GUI::WITH-WRITE-ENABLED to establish a read-only environment after every output operation, GUI:OPEN-GRAPHICS-STREAM can do this only once at window creation. GUI:CLEAR-WINDOW, GUI:WINDOW-TITLE, and GUI:PRINT-MESSAGE now work with graphics streams in addition to text streams. For IMAGESTREAM arguments GUI:PRINT-MESSAGE prints to the system prompt window as Sketch stream windows have no prompt area. The random circles and fractal triangles graphics demos round up the latest additions. #DandeGUI #CommonLisp #Interlisp #Lisp a href="https://remark.as/p/journal.paoloamoroso.com/adding-graphics-support-to-dandegui"Discuss.../a Email | Reply @amoroso@oldbytes.space !--emailsub--]]>
<![CDATA[Printing rich text to windows is one of the planned features of DandeGUI, the GUI library for Medley Interlisp I'm developing in Common Lisp. I finally got around to this and implemented the GUI:WITH-TEXT-STYLE macro which controls the attributes of text printed to a window, such as the font family and face. GUI:WITH-TEXT-STYLE establishes a context in which text printed to the stream associated with a TEdit window is rendered in the style specified by the arguments. The call to GUI:WITH-TEXT-STYLE here extends the square root table example by printing the heading in a 12-point bold sans serif font: (gui:with-output-to-window (stream :title "Table of square roots") (gui:with-text-style (stream :family :sans :size 12 :face :bold) (format stream "~&Number~40TSquare Root~2%")) (loop for n from 1 to 30 do (format stream "~&~4D~40T~8,4F~%" n (sqrt n)))) The code produces this window in which the styled column headings stand out: Medley Interlisp window of a square root table generated by the DandeGUI GUI library. The :FAMILY, :SIZE, and :FACE arguments determine the corresponding text attributes. :FAMILY may be a generic family such as :SERIF for an unspecified serif font; :SANS for a sans serif font; :FIX for a fixed width font; or a keyword denoting a specific family like :TIMESROMAN. At the heart of GUI:WITH-TEXT-STYLE is a pair of calls to the Interlisp function PRINTOUT that wrap the macro body, the first for setting the font of the stream to the specified style and the other for restoring the default: (DEFMACRO WITH-TEXT-STYLE ((STREAM &KEY FAMILY SIZE FACE) &BODY BODY) (ONCE-ONLY (STREAM) `(UNWIND-PROTECT (PROGN (IL:PRINTOUT ,STREAM IL:.FONT (TEXT-STYLE-TO-FD ,FAMILY ,SIZE ,FACE)) ,@BODY) (IL:PRINTOUT ,STREAM IL:.FONT DEFAULT-FONT)))) PRINTOUT is an Interlisp function for formatted output similar to Common Lisp's FORMAT but with additional font control via the .FONT directive. The symbols of PRINTOUT, i.e. its directives and arguments, are in the Interlisp package. In turn GUI:WITH-TEXT-STYLE calls GUI::TEXT-STYLE-TO-FD, an internal DandeGUI function which passes to .FONT a font descriptor matching the required text attributes. GUI::TEXT-STYLE-TO-FD calls IL:FONTCOPY to build a descriptor that merges the specified attributes with any unspecified ones copied from the default font. The font descriptor is an Interlisp data structure that represents a font on the Medley environment. #DandeGUI #CommonLisp #Interlisp #Lisp a href="https://remark.as/p/journal.paoloamoroso.com/changing-text-style-for-dandegui-window-output"Discuss.../a Email | Reply @amoroso@oldbytes.space !--emailsub--]]>
<![CDATA[I continued working on DandeGUI, a GUI library for Medley Interlisp I'm writing in Common Lisp. I added two new short public functions, GUI:CLEAR-WINDOW and GUI:PRINT-MESSAGE, and fixed a bug in some internal code. GUI:CLEAR-WINDOW deletes the text of the window associated with the Interlisp TEXTSTREAM passed as the argument: (DEFUN CLEAR-WINDOW (STREAM) "Delete all the text of the window associated with STREAM. Returns STREAM" (WITH-WRITE-ENABLED (STR STREAM) (IL:TEDIT.DELETE STR 1 (IL:TEDIT.NCHARS STR))) STREAM) It's little more than a call to the TEdit API function IL:TEDIT.DELETE for deleting text in the editor buffer, wrapped in the internal macro GUI::WITH-WRITE-ENABLED that establishes a context for write access to a window. I also wrote GUI:PRINT-MESSAGE. This function prints a message to the prompt area of the window associated with the TEXTSTREAM passed as an argument, optionally clearing the area prior to printing. The prompt area is a one-line Interlisp prompt window attached above the title bar of the TEdit window where the editor displays errors and status messages. (DEFUN PRINT-MESSAGE (STREAM MESSAGE &OPTIONAL DONT-CLEAR-P) "Print MESSAGE to the prompt area of the window associated with STREAM. If DONT-CLEAR-P is non NIL the area will be cleared first. Returns STREAM." (IL:TEDIT.PROMPTPRINT STREAM MESSAGE (NOT DONT-CLEAR-P)) STREAM) GUI:PRINT-MESSAGE just passes the appropriate arguments to the TEdit API function IL:TEDIT.PROMPTPRINT which does the actual printing. The documentation of both functions is in the API reference on the project repo. Testing DandeGUI revealed that sometimes text wasn't appended to the end but inserted at the beginning of windows. To address the issue I changed GUI::WITH-WRITE-ENABLED to ensure the file pointer of the stream is set to the end of the file (i.e -1) prior to passing control to output functions. The fix was to add a call to the Interlisp function IL:SETFILEPTR: (IL:SETFILEPTR ,STREAM -1) #DandeGUI #CommonLisp #Interlisp #Lisp a href="https://remark.as/p/journal.paoloamoroso.com/adding-window-clearing-and-message-printing-to-dandegui"Discuss.../a Email | Reply @amoroso@oldbytes.space !--emailsub--]]>
<![CDATA[I'm working on DandeGUI, a Common Lisp GUI library for simple text and graphics output on Medley Interlisp. The name, pronounced "dandy guy", is a nod to the Dandelion workstation, one of the Xerox D-machines Interlisp-D ran on in the 1980s. DandeGUI allows the creation and management of windows for stream-based text and graphics output. It captures typical GUI patterns of the Medley environment such as printing text to a window instead of the standard output. The main window of this screenshot was created by the code shown above it. A text output window created with DandeGUI on Medley Interlisp and the Lisp code that generated it. The library is written in Common Lisp and exposes its functionality as an API callable from Common Lisp and Interlisp code. Motivations In most of my prior Lisp projects I wrote programs that print text to windows. In general these windows are actually not bare Medley windows but running instances of the TEdit rich-text editor. Driving a full editor instead of directly creating windows may be overkill, but I get for free content scrolling as well as window resizing and repainting which TEdit handles automatically. Moreover, TEdit windows have an associated TEXTSTREAM, an Interlisp data structure for text stream I/O. A TEXTSTREAM can be passed to any Common Lisp or Interlisp output function that takes a stream as an argument such as PRINC, FORMAT, and PRIN1. For example, if S is the TEXTSTREAM associated with a TEdit window, (FORMAT S "~&Hello, Medley!~%") inserts the text "Hello, Medley!" in the window at the position of the cursor. Simple and versatile. As I wrote more GUI code, recurring patterns and boilerplate emerged. These programs usually create a new TEdit window; set up the title and other options; fetch the associated text stream; and return it for further use. The rest of the program prints application specific text to the stream and hence to the window. These patterns were ripe for abstracting and packaging in a library that other programs can call. This work is also good experience with API design. Usage An example best illustrates what DandeGUI can do and how to use it. Suppose you want to display in a window some text such as a table of square roots. This code creates the table in the screenshot above: (gui:with-output-to-window (stream :title "Table of square roots") (format stream "~&Number~40TSquare Root~2%") (loop for n from 1 to 30 do (format stream "~&~4D~40T~8,4F~%" n (sqrt n)))) DandeGUI exports all the public symbols from the DANDEGUI package with nickname GUI. The macro GUI:WITH-OUTPUT-TO-WINDOW creates a new TEdit window with title specified by :TITLE, and establishes a context in which the variable STREAM is bound to the stream associated with the window. The rest of the code prints the table by repeatedly calling the Common Lisp function FORMAT with the stream. GUI:WITH-OUTPUT-TO-WINDOW is best suited for one-off output as the stream is no longer accessible outside of its scope. To retain the stream and send output in a series of steps, or from different parts of the program, you need a combination of GUI:OPEN-WINDOW-STREAM and GUI:WITH-WINDOW-STREAM. The former opens and returns a new window stream which may later be used by FORMAT and other stream output functions. These functions must be wrapped in calls to the macro GUI:WITH-WINDOW-STREAM to establish a context in which a variable is bound to the appropriate stream. The DandeGUI documentation on the project repository provides more details, sample code, and the API reference. Design DandeGUI is a thin wrapper around the Interlisp system facilities that provide the underlying functionality. The main reason for a thin wrapper is to have a simple API that covers the most common user interface patterns. Despite the simplicity, the library takes care of a lot of the complexity of managing Medley GUIs such as content scrolling and window repainting and resizing. A thin wrapper doesn't hide much the data structures ubiquitous in Medley GUIs such as menus and font descriptors. This is a plus as the programmer leverages prior knowledge of these facilities. So far I have no clear idea how DandeGUI may evolve. One more reason not to deepen the wrapper too much without a clear direction. The user needs not know whether DandeGUI packs TEdit or ordinary windows under the hood. Therefore, another design goal is to hide this implementation detail. DandeGUI, for example, disables the main command menu of TEdit and sets the editor buffer to read-only so that typing in the window doesn't change the text accidentally. Using Medley Common Lisp DandeGUI relies on basic Common Lisp features. Although the Medley Common Lisp implementation is not ANSI compliant it provides all I need, with one exception. The function DANDEGUI:WINDOW-TITLE returns the title of a window and allows to set it with a SETF function. However, the SEdit structure editor and the File Manager of Medley don't support or track function names that are lists such as (SETF WINDOW-TITLE). A good workaround is to define SETF functions with DEFSETF which Medley does support along with the CLtL macro DEFINE-SETF-METHOD. Next steps At present DandeGUI doesn't do much more than what described here. To enhance this foundation I'll likely allow to clear existing text and give control over where to insert text in windows, such as at the beginning or end. DandeGUI will also have rich text facilities like printing in bold or changing fonts. The windows of some of my programs have an attached menu of commands and a status area for displaying errors and other messages. I will eventually implement such menu-ed windows. To support programs that do graphics output I plan to leverage the functionality of Sketch for graphics in a way similar to how I build upon TEdit for text. Sketch is the line drawing editor of Medley. The Interlisp graphics primitives require as an argument a DISPLAYSTREAM, a data stracture that represents an output sink for graphics. It is possible to use the Sketch drawing area as an output destination by associating a DISPLAYSTREAM with the editor's window. Like TEdit, Sketch takes care of repainting content as well as window scrolling and resizing. In other words, DISPLAYSTREAM is to Sketch what TEXTSTREAM is to TEdit. DandeGUI will create and manage Sketch windows with associated streams suitable for use as the DISPLAYSTREAM the graphics primitives require. #DandeGUI #CommonLisp #Interlisp #Lisp a href="https://remark.as/p/journal.paoloamoroso.com/dandegui-a-gui-library-for-medley-interlisp"Discuss.../a Email | Reply @amoroso@fosstodon.org !--emailsub--]]>
<![CDATA[I spoke too soon when I said I was enjoying the stability of Linux. I have been using Linux Mint Cinnamon on a System76 Merkaat PC with no major issues since July of 2024. But a few days ago a routine system update of Mint 22 dumped me to the text console. A fresh install of Mint 22.1, the latest release, brought the system back online. I had backups and the mishap luckily turned out as just an annoyance that consumed several hours of unplanned maintenance. It all started when the Mint Update Manager listed several packages for update, including the System76 driver and tools. Oddly, the Update Manager also marked for removal several packages including core ones such as Xorg, Celluloid, and more. The smooth running of Mint made my paranoid side fall asleep and I applied the recommend changes. At the next reboot the graphics session didn't start and landed me at the text console with no clue what happened. I don't use Timeshift for system snapshots as I prefer a fresh install and restore of data backups if the system breaks. Therefore, to fix such an issue apparently related to Mint 22 the obvious route was to install Mint 22.1. Besides, this was the right occasion to try the new release. On my Raspberry Pi 400 I ran dd to flash a bootable USB stick with Mint 22.1. I had no alternatives as GNOME Disks didn't work. The Merkaat failed to boot off the stick, possibly because I messed with the arguments of dd. I still had around a USB stick with Mint 22 and I used it to freshly install it on the Merkaat. Then I immediately ran the upgrade to Mint 22.1 which completed successfully unlike a prior upgrade attempt. Next, I tried to install the System76 driver with sudo apt install system76-driver but got a package not found error. At that point I had already added the System76 package repository to the APT sources and refreshing the Mint Update Manager yielded this error: Could not refresh the list of updates Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages Aside from the errors the system was up and running on the Merkaat, so with Nemo I reflashed the Mint 22.1 stick. This time the PC did boot off the stick and let me successfully install Mint 22.1. Restoring the data completed the system recovery. I left out the System76 driver as it's the primary suspect, possibly due to package conflicts. Mint detects and supports all hardware of the Merkaat anyway and it's only prudent to skip the package for the time being. Besides improvements under the hood, Mint 22.1 features a redesigned default Cinnamon theme. No major changes, I feel at home. The main takeaway of this adventure is that it's better to have a bootable USB stick ready with the latest Mint release, even if I don't plan to upgrade immediately. Another takeaway is the Pi 400 makes for a viable backup computer that can support my major tasks, should it take longer to recover the Merkaat. However, using the device for making bootable media is problematic as little flashing software is available and some is unreliable. Finally, over decades of Linux experience I honed my emergency installation skills so much I can now confidently address most broken system situations. #linux #pi400 a href="https://remark.as/p/journal.paoloamoroso.com/an-unplanned-upgrade-to-linux-mint-22-1-cinnamon"Discuss.../a Email | Reply @amoroso@fosstodon.org !--emailsub--]]>
More in programming
Queues are everywhere, and they follow mathematical rules. Learn a few of those rules! It'll go a long way to making you a stronger SRE.
I’ve long wanted the ability to create custom collections of icons from my icon gallery. Today I can browse collections of icons that share pre-defined metadata (e.g. “Show me all icons tagged as blue”) but I can’t create your own arbitrary collections of icons. That is, until now! I created a page at /lookup that allows you to specify however many id search params you want and it will pull all the matching icons into a single page. Here’s an example of macOS icons that follow the squircle shape but break out of it ever-so-slightly (something we’ll lose with macOS Tahoe). It requires a little know how to construct the URL, something I’ll address later, but it works for my own personal purposes at the moment. So how did I build it? Implementation So the sites are built with a static site generator, but this feature requires an ability to dynamically construct a page based on the icons specified in the URL, e.g. /lookup?id=foo&id=bar&id=baz How do I get that to work? I can’t statically pre-generate every possible combination[1] so what are my options? Create a “shell” page that uses JavaScript to read the search params, query a JSON API, and render whichever icons are specified in the URL. Send an HTML page with all icons over the wire, then use JavaScript to reach into the DOM and remove all icons whose IDs aren’t specified in the page URL. Render the page on the server with just the icons specified in the request URL. No. 1: this is fine, but I don’t have a JSON API for clients to query and I don’t want to create one. Plus I have to duplicate template logic, etc. I’m already rendering lists of icons in my static site generator, so can’t I just do that? Which leads me to: No. 2: this works, but I do have 2000+ icons so the resulting HTML page (I tried it) is almost 2MB if I render everything (whereas that same request for ~4 icons but filtered by the server would be like 11kb). There’s gotta be a way to make that smaller, which leads me to: No. 3: this is great, but it does require I have a “server” to construct pages at request time. Enter Netlify’s Edge Functions which allow you to easily transform an existing HTML page before it gets to the client. To get this working in my case, I: Create /lookup/index.html that has all 2000+ icons on it (trivial with my current static site generator). Create a lookup.ts edge function that intercepts the request to /lookup/index.html Read the search params for the request and get all specified icon IDs, e.g. /lookup?id=a&id=b&id=c turns into ['a','b','c'] Following Netlify’s example of transforming an HTML response, use HTMLRewriter to parse my HTML with all 2000+ icons in it then remove all icons that aren’t in my list of IDs, e.g. <a id='a'>…</a><a id='z'>…</a> might get pruned down to <a id='a'>…</a> Transform the parsed HTML back into a Response and return it to the client from the function. It took me a second to get all the Netlify-specific configurations right (put the function in ./netlify/edge-functions not ./netlify/functions, duh) but once I strictly followed all of Netlify’s rules it was working! (You gotta use their CLI tool to get things working on localhost and test it yourself.) Con-clusions I don’t particularly love that this ties me to a bespoke feature of Netlify’s platform — even though it works really well! But that said, if I ever switched hosts this wouldn’t be too difficult to change. If my new host provided control over the server, nothing changes about the URL for this page (/lookup?id=…). And if I had to move it all to the client, I could do that too. In that sense, I’m tying myself to Netlify from a developer point of view but not from an end-user point of view (everything still works at the URL-level) and I’m good with that trade-off. Just out of curiosity, I asked ChatGPT: if you have approximately 2,000 unique items, how many possible combinations of those IDs can be passed in a URL like /lookup?id=1&id=2? It said the number is 2^2000 which is “astronomically large” and “far more than atoms in the universe”. So statically pre-generating them is out of the question. ⏎ Email · Mastodon · Bluesky
Here’s a story from nearly 10 years ago. the bug I think it was my friend Richard Kettlewell who told me about a bug he encountered with Let’s Encrypt in its early days in autumn 2015: it was failing to validate mail domains correctly. the context At the time I had previously been responsible for Cambridge University’s email anti-spam system for about 10 years, and in 2014 I had been given responsibility for Cambridge University’s DNS. So I knew how Let’s Encrypt should validate mail domains. Let’s Encrypt was about one year old. Unusually, the code that runs their operations, Boulder, is free software and open to external contributors. Boulder is written in Golang, and I had not previously written any code in Golang. But its reputation is to be easy to get to grips with. So, in principle, the bug was straightforward for me to fix. How difficult would it be as a Golang newbie? And what would Let’s Encrypt’s contribution process be like? the hack I cloned the Boulder repository and had a look around the code. As is pretty typical, there are a couple of stages to fixing a bug in an unfamiliar codebase: work out where the problem is try to understand if the obvious fix could be better In this case, I remember discovering a relatively substantial TODO item that intersected with the bug. I can’t remember the details, but I think there were wider issues with DNS lookups in Boulder. I decided it made sense to fix the immediate problem without getting involved in things that would require discussion with Let’s Encrypt staff. I faffed around with the code and pushed something that looked like it might work. A fun thing about this hack is that I never got a working Boulder test setup on my workstation (or even Golang, I think!) – I just relied on the Let’s Encrypt cloud test setup. The feedback time was very slow, but it was tolerable for a simple one-off change. the fix My pull request was small, +48-14. After a couple of rounds of review and within a few days, it was merged and put into production! A pleasing result. the upshot I thought Golang (at least as it was used in the Boulder codebase) was as easy to get to grips with as promised. I did not touch it again until several years later, because there was no need to, but it seemed fine. I was very impressed by the Let’s Encrypt continuous integration and automated testing setup, and by their low-friction workflow for external contributors. One of my fastest drive-by patches to get into worldwide production. My fix was always going to be temporary, and all trace of it was overwritten years ago. It’s good when “temporary” turns out to be true! the point I was reminded of this story in the pub this evening, and I thought it was worth writing down. It demonstrated to me that Let’s Encrypt really were doing all the good stuff they said they were doing. So thank you to Let’s Encrypt for providing an exemplary service and for giving me a happy little anecdote.
It’s well known that Japan is experiencing a labor shortage that includes the tech industry. As a result, Japan needs more international developers. However, most Japanese companies aren’t interested in hiring new graduates from overseas. While it’s easier than in some countries, bringing a developer to Japan still requires some time and financial investment on the company’s side, which they’re unlikely to expend on junior developers. So if you’re a new grad with no experience, can you still find a job in Japan? The answer is yes, but it will take considerable effort, and the opportunities you find may not be what you’re looking for. If you’re determined, it’s best to start preparing while you’re still a student. If you’ve already graduated, then your road is tougher but not impossible, especially if you’re patient, persistent, and willing to go above and beyond. This article will cover: The new graduate hiring system in Japan How international students in Japan can take advantage of the system The steps students and new grads overseas can take to maximize their chances How new grad hiring works in Japan While the primary audience of this article is graduates outside of Japan, understanding how the domestic hiring process works provides necessary context. The Japanese system for hiring new graduates (新卒一括採用, shinsotsu ikkatsu saiyou) operates quite differently from practices in many other countries. It is, however, an effective method: in 2025, 98 percent of Japanese graduates were able to secure employment. What companies are looking for in new graduates To better understand Japan’s methods for hiring new graduates, it helps to know something about Japanese employment philosophy. Traditional Japanese business practices center on the idea of lifetime employment. This approach is currently changing—as of 2023, one in three Japanese employees were no longer committed to lifetime employment, but were open to changing jobs or working independently if the opportunity arose. Nonetheless, decades of conventional wisdom still shape how Japanese companies approach hiring new graduates. Most new graduate hires are considered members of the company on a “comprehensive career track” (総合職採用, sougoushoku saiyou) which means that they aren’t being hired for a specific job, but instead to work for the company in general. In exchange, the business will train them and identify their strengths by assigning them to various departments and roles. Japanese companies therefore typically aren’t as interested in a candidate’s specific experiences and academic achievements, since they’re not certain what position the candidate will eventually occupy in the company. Instead, they prize qualities like good communication skills, independence, and willingness. This is also why Japanese companies often employ Synthetic Personality Inventory (SPI) tests, to attempt to understand if a candidate will be an “objectively” good fit. SPI tests are generally half knowledge-based (often general knowledge), and half MBTI-style personality tests. The job-hunting schedule In Japan, both the school and fiscal year start in April, and so does the job hunt—the year before graduation. If you wait until you’ve graduated to start job-hunting, you’re already a year behind! Study in Japan, a government-approved information site, outlines the following typical schedule for a university student set to graduate the following year: March–May is the entry period. Go to job fairs, seek company introductions, and request entry submission sheets. If your initial application is accepted, begin taking company examinations. In June, the interviews begin. Expect at least three rounds of interviews, of several different types. From June–September is when you may receive a preliminary offer of employment. In October, if you’re successful, you’ll receive the official offer of employment. If you are an international student in Japan, you should change your status of residence. Some companies do continue recruiting in October, but the bulk of the new grad hiring is completed by then. What to expect when applying The new graduate application process also differs widely from standard hiring practices in other countries. Entry sheet If you’re interested in applying to a company, you must first request and complete an entry sheet, which resembles a university application essay more than your average resume. It frequently includes personality-based questions such as “If you had 300 million Japanese yen, what would you do to promote world peace?” and “Describe how you would be indispensable to our company by referring to what you did best during your school life.” Written Examinations If your entry sheet is approved, you can then move on to take the company’s written examinations. Most of these are aptitude tests that evaluate a candidate’s mathematics, language, and writing ability as well as general knowledge. These are typically produced by a third-party company. Interviews It’s customary for companies to conduct at least three rounds of interviews. Some of those interviews are standard single-person interviews. Others are group interviews, in which the interviewer or panel takes turns asking a group of applicants different questions. This method is typically adopted to save time and help winnow down a large applicant pool. You may also be asked to participate in a group discussion interview, in which a gathering of four to six candidates are asked to discuss a given theme, while an interviewer evaluates the group dynamic and individual performance of the applicants. In particular, interviewers are interested in seeing how participants interact in a team situation. Employment offers If a company is interested in hiring you, they may make a preliminary offer of employment (内々定, nai naitei).These types of offers are informal and are not considered binding under law. The more serious step is when a company issues a formal letter of intent to hire (内定, naitei). It is very common for Japanese companies to provide this to show their serious intent of hiring you while they do their due diligence and prepare to issue a formal contract. Under Japanese labor law, the company might be held liable if they subsequently withdraw this offer without cause. If you accept the letter of intent, you are signaling that you will sign a formal employment contract with the company; if you have received preliminary offers from any other companies, you should call them to politely decline. While it is possible to accept and then renege on a formal letter of intent to hire, you should be aware that it is a major faux pas, and you will likely be blacklisted within the company, depending on the circumstances. If this is unavoidable, you should communicate your intentions as quickly and clearly as possible to the company. Japanese companies generally follow the process described here, especially the formal letter of intent to hire. However, these steps are not required by law, so some companies—especially newer or international ones—may not follow this schedule at all! A company following this flow does not indicate that they are reputable or trustworthy, nor does a company not following this flow indicate that they are engaging in deceptive or illegal practices. If you’re an international student in Japan If you’re an international student in Japan, you can join the annual job hunt in April with the rest of your class. Because every step is typically conducted in Japanese, your success in the conventional process will largely depend on how fluent you are. Yet according to a 2024 survey by employment information firm Career-tasu, Inc., most foreign students started job search activities the year they graduated, rather than the year before as their Japanese peers did. This, the staff at Career-tasu believes, puts international students at a serious detriment. International students are also less likely to participate in internship programs or other job experience activities: only 46.1% of international graduates did so, compared to 88.7% of Japanese students surveyed. Nonetheless, The Japan News by The Yomiuri Shimbun reported that some companies are keen to hire international graduates, particularly those who are bilingual or trilingual. In 2022, the percentage of foreign-born graduates finding jobs in Japan exceeded 50% for the first time. Those numbers may improve still further as the labor shortage in Japan worsens. Only around 36% of companies in 2024 were able to meet their recruiting goals. In 2025, 80% of third-year students undergoing the new graduate hiring process already had job offers before June. Companies are already adapting by focusing on mid-career recruitment and introducing fast-tracked interviews, so they may also be more willing to accommodate international student candidates. Certain cities, prefectural governments, and universities have also committed to supporting international students’ job-hunting, and they can help match you with Japanese companies. Ritsumeikan University, for example, not only offers its own job search engine in Japanese, but also lists a number of English-language job boards. New graduates in Japan can take advantage of these school and government resources, as well as being already in the country and available for interviews, examinations, and more. In addition, changing from a Student visa to one of the work-related visas is relatively simple and doesn’t require getting a Certificate of Eligibility or leaving the country. If you’re a student or new grad overseas New graduates from other countries are at a significant disadvantage. First, you’re not available to interview or take examinations in person, which will disqualify you from most companies’ usual hiring procedures. Second, unlike in most countries, the Japanese school year ends in late March. Even if a Japanese company is willing to accept your application via its normal new-grad hiring program, the mismatch in school schedules means you’d need to wait months before you could start work. There are several ways to overcome these difficulties, though, which we’ll explore below. Japanese language skills It’s not absolutely necessary to learn Japanese to get a developer job in Japan. In fact, many Japanese tech companies are creating multinational, English-speaking development teams, or even adopting English as the company language. There’s a catch, however. As mentioned above, those companies who are willing to hire English-speaking developers from overseas are typically searching for senior developers. If you are a junior developer, or a new graduate without experience, you’re far less likely to find an English-speaking position in Japan. That’s the trade-off: you can get a job with work experience and no Japanese ability, but if you don’t have the work experience, you’d better speak Japanese! If the prospect of learning Japanese just to get a job seems daunting, be aware that not every position requires full Japanese fluency. If you’re studying hard and demonstrate enthusiasm in interviews, then conversational-level Japanese might be enough to persuade a company to take a chance on you. Training and recruitment programs Given the high demand in Japan for new tech talents that also speak Japanese, some recruiting companies are offering fast-tracked training programs for students overseas. FAST OFFER, for example, has relationships with over 50 universities worldwide, primarily in Asia. It provides free Japanese classes, which are worth college credits at some participating universities, for students majoring in select fields. These classes are specifically designed to help candidates pass interviews with Japanese companies, as well as acquire the general language skills needed to work in Japan. Candidates who reach a specified level of proficiency—JLPT N4 for most IT roles—will be assigned a mentor, coached on the interview process, and matched with Japanese companies. If the applicant passes the initial screening, they will even be flown to Japan for in-person interviews, all for free. Another example is xseeds Hub, a recruitment platform that developed its own curriculum to prepare students for employment in Japan. As of now, xseeds Hub runs these programs at four Vietnamese universities, as well as one in Indonesia and one in Malaysia. The curriculum includes lessons on different scripting languages, as well as Japanese classes intended to help students achieve at least a JLPT N3 level of speaking. Students gain practical experience via mock experiment lessons and internships in Japan, which run for a minimum of three months. University collaborations with Japanese companies While you’re still a student, you should also find out if your university offers events or programs with Japanese companies. Mercari and Rakuten have particularly targeted the Indian Institutes of Technology (IITs) for recruitment events. But interest in Indian graduates isn’t limited to those companies; when the Indian Embassy hosted a seminar on the subject in Tokyo, over 100 Japanese firms attended. I spoke with Victoria Astingo, who works in Senior Talent Acquisition for Mercari. Mercari has been visiting IITs since 2018, interviewing and hiring soon-to-be graduates on the spot, and they’ve recently started similar campus recruitment drives in Indonesia. These rapid-hire events only take place at universities in Asia for now, but Astingo is expanding Mercari’s internship program outreach to European universities. Her first event in Europe took place at three universities in France, including EPITA, a top-level French institution specializing in Computer Science and Software Engineering. “The reception was way better than we expected,” she said. “At first we were like, ‘Yeah, nobody [here] knows Mercari. Maybe there will be five people coming to our seminar, five anime fans that want to make it to Japan.’ But then actually for EPITA, we had more than 120 attendees.” Other Japanese companies have also started directly partnering with universities abroad. For example, I-Shou University in Taiwan is setting up internships and hiring channels for students in cooperation with Japanese company Nisso Kogyo Co, Ltd., to help supply new talent for the semiconductor industry in Kumamoto. In Vietnam, Ho Chi Minh Open University cooperated with 22 Japanese companies for its 2024 Japan Job Fair, and HUTECH University of Technology hosted 40 Japanese companies at its event. Even if your university doesn’t have preestablished relationships with Japanese companies, you should still check in with your university’s Japanese or East Asian Studies department. The professors there may have personal connections to Japan, or be able to suggest helpful programs, scholarships, or internships. You could also explore your alumni network, and see if you can make contact with any who are currently working in Japan. Internships at TokyoDev companies “Internship” in Japan doesn’t always mean what you think it means—as TokyoDev contributor Lai Huynh Binh Minh discovered, the average “internship” in Japan only lasts one day! That’s not an especially convenient or helpful timeframe for anyone who is overseas. Luckily, TokyoDev works with several companies that offer longer internships with visa support. Not only can you gain valuable work experience, but you also have the opportunity to live in Japan for a few months and discover if it’s really for you. Mercari An internship at Mercari is only available to current university students. It lasts for two months, is paid hourly, and comes with full visa and financial support. Job offers on completion of the internship are considered to be nearly guaranteed. “Most of the time,” said Victoria Astingo, “students apply while they’re second-year students and do their internships before or during their third year. Then, once they graduate, they join us as full-time employees.” This intern-to-FTE pipeline helps explain why Astingo’s new recruitment drive focuses on Europe rather than Singapore or the US, where new graduates are typically offered higher salaries or better compensation packages. Still, Astingo clarified, students from any country are welcome to apply. “I would say the people who we hire as interns usually have previous internship experience as a software engineer,” she said. “[And] if I’m hiring for frontend or backend, I will always check to see if the candidate shares our tech stack—for example, if the candidate has experience working with React or Golang. They don’t really need to master those, but should at least have a basic understanding or interest.” Team experience, such as participating in a hackathon or some other type of group project, is also something Astingo looks for. The name of the applicant’s university doesn’t carry much weight, but majoring in computer science or a related field does. However, even if a candidate doesn’t meet every requirement, she’ll most likely still send the initial skill assessment and give them the opportunity to prove their abilities. Japanese language skills, on the other hand, are not a requirement. “Most of the people I hired for Mercari Marketplace were not Japanese speakers,” Astingo affirmed. HENNGE Like Mercari’s internship program, HENNGE’s Global Internship Program accepts applicants year-round from all over the world. Also like Mercari, it is considered part of the company’s pre-hiring process, so those applicants who are interested in working at HENNGE after graduation will have an advantage. One difference is that the HENNGE internship is unpaid, though participants do receive a monthly stipend, round-trip airfare, and other benefits. Also, the HENNGE program is intended for third-year university students or those who have newly graduated, and so will accommodate slightly older/more advanced candidates. The internships fall into two distinct categories: the Full Stack Engineering Pathway, and the Frontend Engineering Pathway. Each pathway has its own tech stack and experience requirements. It’s possible to apply for both at the same time, but not to change between them mid-internship. As with Mercari, Japanese language ability is not required, but candidates must be fluent in English. HENNGE’s is one of the few global internship programs offered in Japan, so it’s quite competitive, and applicants should be prepared for a selective process with minimal communication. Working Holiday and J-Find visas If you’ve already graduated, then you still have other avenues to come to Japan to look for work: a Working Holiday visa or J-Find visa. These visas both permit the holder to travel, work, and job-hunt in Japan for up to a year. By giving you one year in Japan to network and interview, these visas significantly increase your odds of landing a permanent position, as many roles will not accept overseas applicants. In fact, that’s how TokyoDev founder Paul McMahon got his first job in Japan, and it’s also worked for other TokyoDev contributors as well. However, both visas have specific requirements, so be sure to check whether you qualify. International job fairs If you’re not eligible for a Working Holiday or J-Find visa, try visiting an international job fair instead. Participating businesses have already expressed their willingness to hire overseas candidates, and you’ll be able to personally meet with—and hopefully impress—company representatives. At least some Japanese ability will be required, however, to take advantage of these opportunities. One of the best-known Japanese career fair organizations is Career Forum, which hosts fairs in Boston, Los Angeles, London, Tokyo, and Osaka. The largest overseas event is in Boston, with over 100 companies slated to participate in 2025. Career Forum’s target audience is candidates who speak both Japanese and English to at least a beginner’s level. While their offerings are aimed at current students or recent graduates, they also are interested in candidates who have lived or studied abroad, even if that was not in Japan. On the other side of the globe, Mynavi is holding fairs in Taiwan, Korea, and Australia, as well as online. Though labeled as “career fairs” on their website, these events are actually described as group interviews, and Japanese language skills are a requirement. The Australian event is tightly restricted to current students and new graduates, but the Taiwan fair is open to all those seeking employment in Japanese companies. In Singapore, Asean Career Fair in Singapore will feature 20 Japanese companies in their 2026 event, though it is unclear at the time of writing which companies those will be. Conclusion If you’re a university student or new graduate who wants to find work in Japan, these are your best options: While you’re a student, study Japanese and apply for internships in Japan. If you’re already studying in Japan, you can use university and government resources to help you succeed in the local new graduate hiring process. If you’re overseas, check to see if your university hosts recruitment events or has other connections with Japanese companies and universities. Students in Asia, particularly, may qualify to join a recruiting company’s training program. Working adults may be able to use a Working Holiday or J-Find visa to come to Japan to job-hunt. Visiting an international job fair outside of Japan could get you an interview with an international-friendly company. Good luck and good hunting! Don’t forget to read TokyoDev’s other articles on software developer salaries in Japan, finding a developer job, passing the resume screening process, etc. You can also connect with other developers by joining the TokyoDev Discord, which has channels for resume review, relocating to Japan, and more.
In Chrome Dev Tools you can setup a mapping between the files web server sends to the browser and files on disk. This allows editing files in dev tools and having those changes saved to a file, which is handy. Early in 2025 they’ve added a way to automatically configure this mapping. It’s quite simple. Your server needs to serve /.well-known/appspecific/com.chrome.devtools.json file with content that looks like: { "workspace": { "root": "C:/Users/kjk/src/edna/src", "uuid": "8f6e3d9a-4b7c-4c1e-a2d5-7f9b1e3c1384" } } The uuid should be unique for each project. I got mine by asking Grok to generate random version 4 uuid. On Windows the path has to use Unix slash. It didn’t work when I sent Windows path. Shocking incompetence of the devs. This only works from localhost. Security. This file is read when you open dev tools in chrome. If you already have it open, you have to close and re-open. You still need to connect (annoying security theater). So the first thing you need to do: switch to sources tab in dev tools reveal left side panel (if hidden) switch to workspace tab there you should see the mapping is already configured but you need to click connect button Apparently bun sends this automatically. Here’s how I send it in my Go server: func serveChromeDevToolsJSON(w http.ResponseWriter, r *http.Request) { // your directory might be different than "src" srcDir, err := filepath.Abs("src") must(err) // stupid Chrome doesn't accept windows-style paths srcDir = filepath.ToSlash(srcDir) // uuid should be unique for each workspace uuid := "8f6e3d9a-4b7c-4c1e-a2d5-7f9b0e3c1284" s := `{ "workspace": { "root": "{{root}}", "uuid": "{{uuid}}" } }` s = strings.ReplaceAll(s, "{{root}}", srcDir) s = strings.ReplaceAll(s, "{{uuid}}", uuid) logf("serveChromeDevToolsJSON:\n\n%s\n\n", s) w.Header().Set("Content-Type", "application/json") w.Write(data) }