Full Width [alt+shift+f] Shortcuts [alt+shift+k]
Sign Up [alt+shift+s] Log In [alt+shift+l]

New here?

Welcome! BoredReading is a fresh way to read high quality articles (updated every hour). Our goal is to curate (with your help) Michelin star quality articles (stuff that's really worth reading). We currently have articles in 0 categories from architecture, history, design, technology, and more. Grab a cup of freshly brewed coffee and start reading. This is the best way to increase your attention span, grow as a person, and get a better understanding of the world (or atleast that's why we built it).

17
Using gomarkdown/markdown library This article describes an advanced markdown processing in Go using gomarkdown/markdown library. All the code examples are available at https://github.com/gomarkdown/markdown/tree/master/examples Basics first Here’s a good start for markdown => HTML conversion: func mdToHTML(md []byte) []byte { // create markdown parser with extensions extensions := parser.CommonExtensions | parser.AutoHeadingIDs | parser.NoEmptyLineBeforeBlock p := parser.NewWithExtensions(extensions) doc := p.Parse(md) // create HTML renderer with extensions htmlFlags := html.CommonFlags | html.HrefTargetBlank opts := html.RendererOptions{Flags: htmlFlags} renderer := html.NewRenderer(opts) return markdown.Render(doc, renderer) } Try it online Basic markdown syntax is very limited and there are many extensions that provide additional feature, like tables. HTML render is customizable as well. Here we create markdown parser and HTML renderer with...
over a year ago

Improve your reading experience

Logged in users get linked directly to articles resulting in a better reading experience. Please login for free, it takes less than 1 minute.

More from Krzysztof Kowalczyk blog

Man vs. AI: optimizing JavaScript (Claude, Cursor)

How AI beat me at code optimization game. When I started writing this article I did not expect AI to beat me at optimizing JavaScript code. But it did. I’m really passionate about optimizing JavaScript. Some say it’s a mental illness but I like my code to go balls to the wall fast. I feel the need. The need for speed. Optimizing code often requires tedious refactoring. Can we delegate the tedious parts to AI? Can I just have ideas and get AI to be my programming slave? Let’s find out. Optimizing Unicode range lookup with AI In my experiment I used Cursor with Claude 3.5 Sonnet model. I assume it could be done with other tools / models. I was browsing pdf.js code and saw this function: const UnicodeRanges = [ [0x0000, 0x007f], // 0 - Basic Latin ... omited [0x0250, 0x02af, 0x1d00, 0x1d7f, 0x1d80, 0x1dbf], // 4 - IPA Extensions - Phonetic Extensions - Phonetic Extensions Supplement ... omited ]; function getUnicodeRangeFor(value, lastPosition = -1) { // TODO: create a map range => position, sort the ranges and cache it. // Then we can make a binary search for finding a range for a given unicode. if (lastPosition !== -1) { const range = UnicodeRanges[lastPosition]; for (let i = 0, ii = range.length; i < ii; i += 2) { if (value >= range[i] && value <= range[i + 1]) { return lastPosition; } } } for (let i = 0, ii = UnicodeRanges.length; i < ii; i++) { const range = UnicodeRanges[i]; for (let j = 0, jj = range.length; j < jj; j += 2) { if (value >= range[j] && value <= range[j + 1]) { return i; } } } return -1; } This is a simple function: given Unicode character it returns Unicode range (number 0 ... UnicodeRanges.length - 1). The number is cleverly encoded implicitly as array index. The code already has a TODO about a potential optimization: sort UnicodeRanges and do binary search. There’s a minor difficulty: some Unicode ranges have multiple intervals, so we have to change the UnicodeRanges to explicitly include the range as 3rd element: const UnicodeRanges = [ [0x0000, 0x007f, 0], // 0 - Basic Latin [0x0250, 0x02af, 4], // 4 - IPA Extensions - Phonetic Extensions - Phonetic Extensions Supplement [0x1d00, 0x1d7f, 4], // 4 - IPA Extensions - Phonetic Extensions - Phonetic Extensions Supplement ... omitted ]; Then sort it and change the implementation of getUnicodeRangeFor(). Re-writing UnicodeRanges is simple but tedious and I don’t feel like doing it. Can AI do such change for us? Changing representation I’ve selected the whole of UnicodeRanges, added to chat with Ctrl + Shift + L and commanded: Change representation of UnicodeRanges array where each entry is 3 elements: [start, end, rangeNo]; preserve comments; if rangeNo has multiple ranges, split into separate entries Cursor responded with: const UnicodeRanges = [ [0x0000, 0x007f, 0], // 0 - Basic Latin [0x0080, 0x00ff, 1], // 1 - Latin-1 Supplement ... omitted ]; There was an issue: it only did part of the array. So I commanded: don’t be lazy, finish the whole UnicodeRanges array It did but the result was too long so I had to click Continue button. It worked and the generated code seems to be correct so I applied it. For binary search, the array must be sorted, so I commanded: Sort the array by first element It did, the result looked correct so I accepted the change. Hmm, I can get used to having a programming slave. Updating the code to implement binary search I copied getUnicodeRangeFor to chat and commanded: change the implementation to use binary search over UnicodeRanges array It gave me: function getUnicodeRangeFor(value, lastPosition = -1) { // Check the last position first if (lastPosition !== -1) { const range = UnicodeRanges[lastPosition]; if (value >= range[0] && value <= range[1]) { return lastPosition; } } // Binary search let left = 0; let right = UnicodeRanges.length - 1; while (left <= right) { const mid = Math.floor((left + right) / 2); const range = UnicodeRanges[mid]; if (value < range[0]) { right = mid - 1; } else if (value > range[1]) { left = mid + 1; } else { return range[2]; // Return the range index } } return -1; // Not found } It looks correct so I accepted it. We achieved the goal: replaced linear search with binary search using cheap AI labor. But I want more. More speed. Changing array of arrays to flat array A more efficient representation of the ranges would be a flat array: const UnicodeRanges = [ 0x0000, 0x007f, 0, // 0 - Basic Latin 0x0080, 0x00ff, 1, // 1 - Latin-1 Supplement ]; Let’s see if our AI slave can do it: change the whole UnicodeRanges array to flat array, preserve comments, don’t be lazy and do all of it I pre-emptively command the AI to convert the whole array. It did it. Now is the time to update getUnicodeRangeFor(). It’s a simple change I could do myself: now we just multiply index * 3 to get range definition. But I kind of like commanding the AI so I did it again: update getUnicodeRangeFor for new flat array representations of UnicodeRanges It gave me: function getUnicodeRangeFor(value, lastPosition = -1) { // Check the last position first if (lastPosition !== -1) { const index = lastPosition * 3; if (value >= UnicodeRanges[index] && value <= UnicodeRanges[index + 1]) { return UnicodeRanges[index + 2]; } } // Binary search let left = 0; let right = UnicodeRanges.length / 3 - 1; while (left <= right) { const mid = Math.floor((left + right) / 2); const index = mid * 3; const rangeStart = UnicodeRanges[index]; const rangeEnd = UnicodeRanges[index + 1]; if (value < rangeStart) { right = mid - 1; } else if (value > rangeEnd) { left = mid + 1; } else { return UnicodeRanges[index + 2]; // Return the range index } } return -1; // Not found } It does seem correct, but let’s do one more tweak: move binary search code to its own function And it worked: function binarySearchUnicodeRange(value) { let left = 0; let right = UnicodeRanges.length / 3 - 1; while (left <= right) { const mid = Math.floor((left + right) / 2); const index = mid * 3; const rangeStart = UnicodeRanges[index]; const rangeEnd = UnicodeRanges[index + 1]; if (value < rangeStart) { right = mid - 1; } else if (value > rangeEnd) { left = mid + 1; } else { return UnicodeRanges[index + 2]; // Return the range index } } return -1; // Not found } function getUnicodeRangeFor(value, lastPosition = -1) { // Check the last position first if (lastPosition !== -1) { const index = lastPosition * 3; if (value >= UnicodeRanges[index] && value <= UnicodeRanges[index + 1]) { return UnicodeRanges[index + 2]; } } // Perform binary search return binarySearchUnicodeRange(value); } Inlining some of the binary search It’s good but we haven’t yet reached the levels of insanity I’ve hinted at. UnicodeRanges is a fixed array. We could “unroll” the binary search into code. It would be very painful to do manually. I would have to write custom code generator to ensure it’s correct but maybe AI can do it for us? unroll first 3 levels of binarySearchUnicodeRange To my surprise, it worked. AI understood what I wanted and gave me this: function getUnicodeRangeFor(value, lastPosition = -1) { ... omited const length = UnicodeRanges.length / 3; let mid = Math.floor(length / 2) * 3; if (value < UnicodeRanges[mid]) { mid = Math.floor(mid / 2); if (value < UnicodeRanges[mid]) { mid = Math.floor(mid / 2); } else if (value > UnicodeRanges[mid + 1]) { mid = mid + Math.floor(mid / 2); } else { return UnicodeRanges[mid + 2]; } } else if (value > UnicodeRanges[mid + 1]) { mid = mid + Math.floor(mid / 2); if (value < UnicodeRanges[mid]) { mid = mid - Math.floor(mid / 4); } else if (value > UnicodeRanges[mid + 1]) { mid = mid + Math.floor(mid / 4); } else { return UnicodeRanges[mid + 2]; } } else { return UnicodeRanges[mid + 2]; } ... omited } Can we do even better? unroll the values of UnicodeRanges[mid] etc. and replace with constants Holly cow, it worked: function getUnicodeRangeFor(value, lastPosition = -1) { // Check the last position first if (lastPosition !== -1) { const index = lastPosition * 3; if (value >= UnicodeRanges[index] && value <= UnicodeRanges[index + 1]) { return UnicodeRanges[index + 2]; } } // Unrolled binary search with constants if (value < 0x4E00) { // CJK Unified Ideographs start if (value < 0x0530) { // Armenian start if (value < 0x0250) { // IPA Extensions start if (value < 0x0100) { // Latin Extended-A start if (value < 0x0080) { // Latin-1 Supplement start return 0; // Basic Latin } return 1; // Latin-1 Supplement } if (value < 0x0180) { return 2; // Latin Extended-A } return 3; // Latin Extended-B } if (value < 0x0300) { // Combining Diacritical Marks start if (value < 0x02B0) { // Spacing Modifier Letters start return 4; // IPA Extensions } return 5; // Spacing Modifier Letters } if (value < 0x0370) { // Greek and Coptic start return 6; // Combining Diacritical Marks } if (value < 0x0400) { // Cyrillic start return 7; // Greek and Coptic } return 9; // Cyrillic } // Continue with binary search for remaining ranges let left = 10; // Index of Armenian in UnicodeRanges let right = 102; // Index of CJK Unified Ideographs in UnicodeRanges while (left <= right) { const mid = Math.floor((left + right) / 2) * 3; if (value < UnicodeRanges[mid]) { right = Math.floor(mid / 3) - 1; } else if (value > UnicodeRanges[mid + 1]) { left = Math.floor(mid / 3) + 1; } else { return UnicodeRanges[mid + 2]; } } } else { // Binary search for ranges after CJK Unified Ideographs let left = 103; // Index after CJK Unified Ideographs in UnicodeRanges let right = UnicodeRanges.length / 3 - 1; while (left <= right) { const mid = Math.floor((left + right) / 2) * 3; if (value < UnicodeRanges[mid]) { right = Math.floor(mid / 3) - 1; } else if (value > UnicodeRanges[mid + 1]) { left = Math.floor(mid / 3) + 1; } else { return UnicodeRanges[mid + 2]; } } } return -1; // Not found } Is it correct? I have no idea. Would be easy to write a test that uses UnicodeRanges as source of truth and tests getUnicodeRangeFor() for every possible value. Can I just ask AI to do it faster? So far I micro-managed the AI coder. I was telling it exactly what kind of change I wanted and commanded AI until it did what I wanted. A good boss delegates. Good boss tells the worker the high level goal and leaves implementation detail to the worker. Let’s start from scratch and command AI thusly: could you make a faster implementation of getUnicodeRangeFor()? You can change the representation of UnicodeRanges to achieve that goal Holly cow again. The AI figured out the binary search: const UnicodeRanges = [ { start: 0x0000, end: 0x007f, index: 0 }, // Basic Latin { start: 0x0080, end: 0x00ff, index: 1 }, // Latin-1 Supplement { start: 0x0100, end: 0x017f, index: 2 }, // Latin Extended-A // ... (other ranges) { start: 0x1f000, end: 0x1f09f, index: 122 }, // Domino Tiles - Mahjong Tiles ].sort((a, b) => a.start - b.start); function getUnicodeRangeFor(value, lastPosition = -1) { if (lastPosition !== -1) { const range = UnicodeRanges[lastPosition]; if (value >= range.start && value <= range.end) { return range.index; } } let left = 0; let right = UnicodeRanges.length - 1; while (left <= right) { const mid = Math.floor((left + right) / 2); const range = UnicodeRanges[mid]; if (value >= range.start && value <= range.end) { return range.index; } if (value < range.start) { right = mid - 1; } else { left = mid + 1; } } return -1; } The object {start, end, index} is less efficient than my flat array representation and it didn’t unroll the binary search but the core idea of binary search is there. But just for kicks, maybe there’s more: can you make it even more efficient? It did something slightly different, but really the same. I’m afraid no bonus this quarter. Ok, one last try: but could you go all out and use every possible implementation trick you know to optkmize getUnicodeRangeFor and UnicodeRanges representation? Fuck me, I have no idea if the generated code is correct but it does seem it would be faster than the non-unrolled binary search I came up with earlier: // Preprocess UnicodeRanges into a more efficient structure const BUCKET_SIZE = 256; const MAX_UNICODE = 0x10FFFF; const buckets = new Uint16Array(Math.ceil((MAX_UNICODE + 1) / BUCKET_SIZE)); const ranges = []; UnicodeRanges.forEach((range, index) => { for (let i = 0; i < range.length; i += 2) { const start = range[i]; const end = range[i + 1]; ranges.push({ start, end, index }); const startBucket = start >>> 8; const endBucket = end >>> 8; for (let j = startBucket; j <= endBucket; j++) { buckets[j] |= (1 << (index & 15)); } } }); ranges.sort((a, b) => a.start - b.start); // Create a compact representation of ranges const compactRanges = new Uint32Array(ranges.length * 3); ranges.forEach((range, i) => { compactRanges[i * 3] = range.start; compactRanges[i * 3 + 1] = range.end; compactRanges[i * 3 + 2] = range.index; }); function getUnicodeRangeFor(value, lastPosition = -1) { if (value > MAX_UNICODE) return -1; // Check last position first if (lastPosition !== -1) { const start = compactRanges[lastPosition * 3]; const end = compactRanges[lastPosition * 3 + 1]; if (value >= start && value <= end) { return compactRanges[lastPosition * 3 + 2]; } } // Quick check using buckets const bucket = buckets[value >>> 8]; if (bucket === 0) return -1; // Binary search within the relevant ranges let left = 0; let right = compactRanges.length / 3 - 1; while (left <= right) { const mid = (left + right) >>> 1; const start = compactRanges[mid * 3]; const end = compactRanges[mid * 3 + 1]; if (value < start) { right = mid - 1; } else if (value > end) { left = mid + 1; } else { return compactRanges[mid * 3 + 2]; } } return -1; } In retrospect, using Uint32Array for UnicodeRanges array was a no-brainer. It’s more compact in memory, the values will probably be compared as integers, not floats. Should be faster to scan than array of floats. The buckets are not necessary. They seems to be for quick “doesn’t exist check” which is not important. I believe most lookups are for values that exist. I’m humbled that just asking for super duper optimization made AI produce something I didn’t think of. More optimization ideas I can’t help myself. Those are ideas I didn’t ask AI to implement. UnicodeRanges is small. A linear search of compact Uint32Array representation where we just have (start, end) values for each range would be faster than binary search due to cache lines. We could start the search in the middle of array and scan half the data going forward or backwards. We could also store ranges smaller than 0x10000 in Uint16Array and larger in Uint32Array. And do linear search starting in the middle. Since the values are smaller than 256, we could encode the first 0xffff values in 64kB as Uint8Array and the rest as Uint32Array. That would probably be the fastest on average, because I believe most lookups are for Unicode chars smaller than 0xffff. Finally, we could calculate the the frequency of each range in representative sample of PDF documents, check the ranges based on that frequency, fully unrolled into code, without any tables. Conclusions AI is a promising way to do tedious code refactoring. If I didn’t have the AI, I would have to write a program to e.g. convert UnicodeRanges to a flat representation. It’s simple and therefore doable but certainly would take longer than few minutes it took me to command AI. The final unrolling of getUnicodeRangeFor() would probably never happen. It would require writing a sophisticated code generator which would be a big project by itself. AI can generate buggy code so it needs to be carefully reviewed. The unrolled binary search could not be verified by review, it would need a test. But hey, I could command my AI sidekick to write the test for me. There was this idea of organizing programming teams into master programmer and coding grunts. The job of master programmer, the thinking was, to generate high level ideas and having coding grunts implement them. Turns out that we can’t organize people that way but now we can use AI to be our coding grunt. Prompt engineering is a thing. I wasted a bunch of time doing incremental improvements. I should have started by asking for super-duper optimization. Productivity gains is real. The whole thing took me about an hour. For this particular task easily 2x compared to not using cheap AI labor. Imagine you’re running a software business and instead of spending 2 months on a task, you only spend 1 month. I’ll be using more AI for coding in the future.

7 months ago 63 votes
Implementing Notion-like table of contents in JavaScript

Notion-like table of contents in JavaScript Long web pages benefit from having a table of contents. Especially technical, reference documentation. As a reader you want a way to quickly navigate to a specific part of the documentation. This article describes how I implemented table of contents for documentation page for my Edna note taking application. Took only few hours. Here’s full code. A good toc A good table of contents is: always available unobtrusive Table of contents cannot be always visible. Space is always at premium and should be used for the core functionality of a web page. For a documentation page the core is documentation text so space should be used to show documentation. But it should always be available in some compact form. I noticed that Notion implemented toc in a nice way. Since great artists steal, I decided to implement similar behavior for Edna’s documentation When hidden, we show mini toc i.e. for each toc entry we have a gray rectangle. A block rectangle indicates current position in the document: It’s small and unobtrusive. When you hover mouse over that area we show the actual toc: Clicking on a title goes to that part of the page. Implementing table of contents My implementation can be added to any page. Grabbing toc elements I assume h1 to h6 elements mark table of contents entries. I use their text as text of the entry. After page loads I build the HTML for the toc. I grab all headers elements: function getAllHeaders() { return Array.from(document.querySelectorAll("h1, h2, h3, h4, h5, h6")); } Each toc entry is represented by: class TocItem { text = ""; hLevel = 0; nesting = 0; element; } text we show to the user. hLevel is 1 … 6 for h1 … h6. nesting is like hLevel but sanitized. We use it to indent text in toc, to show tree structure of the content. element is the actual HTML element. We remember it so that we can scroll to that element with JavaScript. I create array of TocItem for each header element on the page: function buildTocItems() { let allHdrs = getAllHeaders(); let res = []; for (let el of allHdrs) { /** @type {string} */ let text = el.innerText.trim(); text = removeHash(text); text = text.trim(); let hLevel = parseInt(el.tagName[1]); let h = new TocItem(); h.text = text; h.hLevel = hLevel; h.nesting = 0; h.element = el; res.push(h); } return res; } function removeHash(str) { return str.replace(/#$/, ""); } Generate toc HTML Toc wrapper Here’s the high-level structure: .toc-wrapper has 2 children: .toc-mini, always visible, shows overview of the toc .toc-list hidden by default, shown on hover over .toc-wrapper Wrapper is always shown on the right upper corner using fixed position: .toc-wrapper { position: fixed; top: 1rem; right: 1rem; z-index: 50; } You can adjust top and right for your needs. When toc is too long to fully shown on screen, we must make it scrollable. But also default scrollbars in Chrome are large so we make them smaller and less intrusive: .toc-wrapper { position: fixed; top: 1rem; right: 1rem; z-index: 50; } When user hovers over .toc-wrapper, we switch display from .toc-mini to .toc-list: .toc-wrapper:hover > .toc-mini { display: none; } .toc-wrapper:hover > .toc-list { display: flex; } Generate mini toc We want to generate the following HTML: <div class="toc-mini"> <div class="toc-item-mini toc-light">▃</div> ... repeat for every TocItem </div> ▃ is a Unicode characters that is a filled rectangle of the bottom 30% of the character. We use a very small font becuase it’s only a compact navigation heler. .toc-light is gray color. By removing this class we make it default black which marks current position in the document. .toc-mini { display: flex; flex-direction: column; font-size: 6pt; cursor: pointer; } .toc-light { color: lightgray; } Generating HTML in vanilla JavaScript is not great, but it works for small things: function genTocMini(items) { let tmp = ""; let t = `<div class="toc-item-mini toc-light">▃</div>`; for (let i = 0; i < items.length; i++) { tmp += t; } return `<div class="toc-mini">` + tmp + `</div>`; } items is an array of TocItem we get from buildTocItems(). We mark the items with toc-item-mini class so that we can query them all easily. Generate toc list Table of contents list is only slightly more complicated: <div class="toc-list"> <div title="{title}" class="toc-item toc-trunc {ind}" onclick=tocGoTo({n})>{text}</div> ... repeat for every TocItem </div> {ind} is name of the indent class, like: .toc-ind-1 { padding-left: 4px; } tocGoTo(n) is a function that scroll the page to show n-th TocItem.element at the top. function genTocList(items) { let tmp = ""; let t = `<div title="{title}" class="toc-item toc-trunc {ind}" onclick=tocGoTo({n})>{text}</div>`; let n = 0; for (let h of items) { let s = t; s = s.replace("{n}", n); let ind = "toc-ind-" + h.nesting; s = s.replace("{ind}", ind); s = s.replace("{text}", h.text); s = s.replace("{title}", h.text); tmp += s; n++; } return `<div class="toc-list">` + tmp + `</div>`; } .toc-trunc is for limiting the width of toc and gracefully truncating it: .toc-trunc { max-width: 32ch; min-width: 12ch; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } Putting it all together Here’s the code that runs at page load, generates HTML and appends it to the page: function genToc() { tocItems = buildTocItems(); fixNesting(tocItems); const container = document.createElement("div"); container.className = "toc-wrapper"; let s = genTocMini(tocItems); let s2 = genTocList(tocItems); container.innerHTML = s + s2; document.body.appendChild(container); } Navigating Showing / hiding toc list is done in CSS. When user clicks toc item, we need to show it at the top of page: let tocItems = []; function tocGoTo(n) { let el = tocItems[n].element; let y = el.getBoundingClientRect().top + window.scrollY; let offY = 12; y -= offY; window.scrollTo({ top: y, }); } We remembered HTML element in TocItem.element so all we need to is to scroll to it to show it at the top of page. You can adjust offY e.g. if you show a navigation bar at the top that overlays the content, you want to make offY at least the height of navigation bar. Updating toc mini to reflect current position When user scrolls the page we want to reflect that in toc mini by changing the color of corresponding rectangle from gray to black. On scroll event we calculate which visible TocItem.element is closest to the top of window. function updateClosestToc() { let closestIdx = -1; let closestDistance = Infinity; for (let i = 0; i < tocItems.length; i++) { let tocItem = tocItems[i]; const rect = tocItem.element.getBoundingClientRect(); const distanceFromTop = Math.abs(rect.top); if ( distanceFromTop < closestDistance && rect.bottom > 0 && rect.top < window.innerHeight ) { closestDistance = distanceFromTop; closestIdx = i; } } if (closestIdx >= 0) { console.log("Closest element:", closestIdx); let els = document.querySelectorAll(".toc-item-mini"); let cls = "toc-light"; for (let i = 0; i < els.length; i++) { let el = els[i]; if (i == closestIdx) { el.classList.remove(cls); } else { el.classList.add(cls); } } } } window.addEventListener("scroll", updateClosestToc); All together now After page loads I run: genToc(); updateClosestToc(); Which I achieve by including this in HTML: <script src="/help.js" defer></script> </body> Possible improvements Software is never finished. Software can always be improved. I have 2 ideas for further improvements. Always visible when enough space Most of the time my browser window uses half of 13 to 15 inch screen. I’m aggravated when websites don’t work well at that size. At that size there’s not enough space to always show toc. But if the user chooses a wider browser window, it makes sense to use available space and always show table of contents. Keyboard navigation It would be nice to navigate table of contents with keyboard, in addition to mouse. For example: t would show table of contents Esc would dismiss it up / down arrows would navigate toc tree Enter would navigate to selected part and dismiss toc

7 months ago 68 votes
Porting a medium-sized Vue application to Svelte 5

Porting a medium-sized Vue application to Svelte 5 The short version: porting from Vue to Svelte is pretty straightforward and Svelte 5 is nice upgrade to Svelte 4. Why port? I’m working on Edna, a note taking application for developers. It started as a fork of Heynote. I’ve added a bunch of features, most notably managing multiple notes. Heynote is written in Vue. Vue is similar enough to Svelte that I was able to add features without really knowing Vue but Svelte is what I use for all my other projects. At some point I invested enough effort (over 350 commits) into Edna that I decided to port from Vue to Svelte. That way I can write future code faster (I know Svelte much better than Vue) and re-use code from my other Svelte projects. Since Svelte 5 is about to be released, I decided to try it out. There were 10 .vue components. It took me about 3 days to port everything. Adding Svelte 5 to build pipeline I started by adding Svelte 5 and converting the simplest component. In the above commit: I’ve installed Svelte 5 and it’s vite plugin by adding it to package.json updated tailwind.config.cjs to also scan .svelte files added Svelte plugin to vite.config.js to run Svelte compiler on .svelte and .svelte.js files during build deleted Help.vue, which is not related to porting, I just wasn’t using it anymore started converting smallest component AskFSPermissions.vue as AskFSPermissions.svelte In the next commit: I finished porting AskFSPermissions.vue I tweaked tsconfig.json so that VSCode type-checks .svelte files I replaced AskFSPermissions.vue with Svelte 5 version Here replacing was easy because the component was a stand-alone component. All I had to do was to replace Vue’s: app = createApp(AskFSPermissions); app.mount("#app"); with Svelte 5: const args = { target: document.getElementById("app"), }; appSvelte = mount(AskFSPermissions, args); Overall porting strategy Next part was harder. Edna’s structure is: App.vue is the main component which shows / hides other components depending on state and desired operations. My preferred way of porting would be to start with leaf components and port them to Svelte one by one. However, I haven’t found an easy way of using .svelte components from .vue components. It’s possible: Svelte 5 component can be imported and mounted into arbitrary html element and I could pass props down to it. If the project was bigger (say weeks of porting) I would try to make it work so that I have a working app at all times. Given I estimated I can port it quickly, I went with a different strategy: I created mostly empty App.svelte and started porting components, starting with the simplest leaf components. I didn’t have a working app but I could see and test the components I’ve ported so far. This strategy had it’s challenges. Namely: most of the state is not there so I had to fake it for a while. For example the first component I ported was TopNav.vue, which displays name of the current note in the top upper part of the screen. The problem was: I didn’t port the logic to load the file yet. For a while I had to fake the state i.e. I created noteName variable with a dummy value. With each ported component I would port App.vue parts needed by the component Replacing third-party components Most of the code in Edna is written by me (or comes from the original Heynote project) and doesn’t use third-party Vue libraries. There are 2 exceptions: I wanted to show notification messages and have a context menu. Showing notifications messages isn’t hard: for another project I wrote a Svelte component for that in a few hours. But since I didn’t know Vue well, it would have taken me much longer, possibly days. For that reason I’ve opted to use a third-party toast notifications Vue library. The same goes menu component. Even more so: implementing menu component is complicated. At least few days of effort. When porting to Svelte I replaced third-party vue-toastification library with my own code. At under 100 loc it was trivial to write. For context menu I re-used context menu I wrote for my notepad2 project. It’s a more complicated component so it took longer to port it. Vue => Svelte 5 porting Vue and Svelte have very similar structure so porting is straightforward and mostly mechanical. The big picture: <template> become Svelte templates. Remove <template> and replace Vue control flow directives with Svelte equivalent. For example <div v-if="foo"> becomes {#if foo}<div>{/if} setup() can be done either at top-level, when component is imported, or in $effect( () => { ... } ) when component is mounted data() become variables. Some of them are regular JavaScript variables and some of them become reactive $state() props becomes $props() mounted() becomes $effect( () => { ... } ) methods() become regular JavaScript functions computed() become $derived.by( () => { ... } ) ref() becomes $state() $emit('foo') becomes onfoo callback prop. Could also be an event but Svelte 5 recommends callback props over events @click becomes onclick v-model="foo" becomes bind:value={foo} {{ foo }} in HTML template becomes { foo } ref="foo" becomes bind:this={foo} :disabled="!isEnabled" becomes disabled={!isEnabled} CSS was scoped so didn’t need any changes Svelte 5 At the time of this writing Svelte 5 is Release Candidates and the creators tell you not use it in production. Guess what, I’m using it in production. It works and it’s stable. I think Svelte 5 devs operate from the mindset of “abundance of caution”. All software has bugs, including Svelte 4. If Svelte 5 doesn’t work, you’ll know it. Coming from Svelte 4, Svelte 5 is a nice upgrade. One small project is too early to have deep thoughts but I like it so far. It’s easy to learn new ways of doing things. It’s easy to convert Svelte 4 to Svelte 5, even without any tools. Things are even more compact and more convenient than in Svelte 4. {#snippet} adds functionality that I was missing from Svelte 4.

9 months ago 64 votes
Changing font size in Windows dialog in C++

How to dynamically change font size in a Windows dialog Windows’s win32 API is old and crufty. Many things that are trivial to do in HTML are difficult in win32. One of those things is changing size of font used by your native, desktop app. I encountered this in SumatraPDF. A user asked for a way to increase the font size. I introduced UIFontSize option but implementing that was difficult and time consuming. One of the issues was changing the font size used in dialogs. This article describes how I did it. The method is based on https://stackoverflow.com/questions/14370238/can-i-dynamically-change-the-font-size-of-a-dialog-window-created-with-c-in-vi How dialogs work SumatraPDF defines a bunch of dialogs in SumatraPDF.rc. Here’s a find dialog: IDD_DIALOG_FIND DIALOGEX 0, 0, 247, 52 STYLE DS_SETFONT | DS_MODALFRAME | DS_FIXEDSYS | WS_POPUP | WS_CAPTION | WS_SYSMENU CAPTION "Find" FONT 8, "MS Shell Dlg", 400, 0, 0x1 BEGIN LTEXT "&Find what:",IDC_STATIC,6,8,60,9 EDITTEXT IDC_FIND_EDIT,66,6,120,13,ES_AUTOHSCROLL CONTROL "&Match case",IDC_MATCH_CASE,"Button",BS_AUTOCHECKBOX | WS_TABSTOP,6,24,180,9 LTEXT "Hint: Use the F3 key for finding again",IDC_FIND_NEXT_HINT,6,37,180,9,WS_DISABLED DEFPUSHBUTTON "Find",IDOK,191,6,50,14 PUSHBUTTON "Cancel",IDCANCEL,191,24,50,14 END .rc is compiled by a resource compiler rc.exe and embedded in resources section of a PE .exe file. Compiled version is a binary blob that has a stable format. At runtime we can get that binary blob from resources and pass it to DialogBoxIndirectParam() function to create a dialog. How to change font size of a dialog at runtime DIALOGEX tell us it’s an extended dialog, which has different binary layout than non-extended DIALOG. As you can see part of dialog definition is a font definition: FONT 8, "MS Shell Dlg", 400, 0, 0x1 To provide a FONT you also need to specify DS_SETFONT or DS_FIXEDSYS flag. We’re asking for MS Shell Dlg font with size of 8 points (12 pixels). 400 specifies standard weight (800 would be bold font). Unfortunately the binary blob is generated at compilation time and we want to change font size when application runs. The simplest way to achieve that is to patch the binary blob in memory. The code for changing dialog font size at runtime You can find the full code at https://github.com/sumatrapdfreader/sumatrapdf/blob/b6aed9e7d257510ff82fee915506ce2e75481c64/src/SumatraDialogs.cpp#L20 It uses small number of SumatraPDF base code so you’ll need to lightly massage it to use it in your own code. The layout of binary blob is documented at http://msdn.microsoft.com/en-us/library/ms645398(v=VS.85).aspx In C++ this is represented by the following struct: #pragma pack(push, 1) struct DLGTEMPLATEEX { WORD dlgVer; // 0x0001 WORD signature; // 0xFFFF DWORD helpID; DWORD exStyle; DWORD style; WORD cDlgItems; short x, y, cx, cy; /* sz_Or_Ord menu; sz_Or_Ord windowClass; WCHAR title[titleLen]; WORD fontPointSize; WORD fontWWeight; BYTE fontIsItalic; BYTE fontCharset; WCHAR typeface[stringLen]; */ }; #pragma pack(pop) #pragma pack(push, 1) tells C++ compiler to not do padding between struct members. That part after x, y, cx, cy is commented out because sz_or_Ord and WCHAR [] are variable length, which can’t be represented in C++ struct. fontPointSize is the value we need to patch. But first we need to get a copy binary blob. DLGTEMPLATE* DupTemplate(int dlgId) { HRSRC dialogRC = FindResourceW(nullptr, MAKEINTRESOURCE(dlgId), RT_DIALOG); CrashIf(!dialogRC); HGLOBAL dlgTemplate = LoadResource(nullptr, dialogRC); CrashIf(!dlgTemplate); void* orig = LockResource(dlgTemplate); size_t size = SizeofResource(nullptr, dialogRC); CrashIf(size == 0); DLGTEMPLATE* ret = (DLGTEMPLATE*)memdup(orig, size); UnlockResource(orig); return ret; } dlgId is from .rc file (e.g. IDD_DIALOG_FIND for our find dialog). Most of it is win32 APIs, memdup() makes a copy of memory block. Here’s the code to patch the font size: static void SetDlgTemplateExFont(DLGTEMPLATE* tmp, int fontSize) { CrashIf(!IsDlgTemplateEx(tmp)); DLGTEMPLATEEX* tpl = (DLGTEMPLATEEX*)tmp; CrashIf(!HasDlgTemplateExFont(tpl)); u8* d = (u8*)tpl; d += sizeof(DLGTEMPLATEEX); // sz_Or_Ord menu d = SkipSzOrOrd(d); // sz_Or_Ord windowClass; d = SkipSzOrOrd(d); // WCHAR[] title d = SkipSz(d); // WCHAR pointSize; WORD* wd = (WORD*)d; fontSize = ToFontPointSize(fontSize); *wd = fontSize; } We start at the end of fixed-size portion of the blob () d += sizeof(DLGTEMPLATEEX). We then skip variable-length fields menu, windowClass and title and patch the font size in points. SumatraPDF code operates in pixels so has to convert that to Windows points: static int ToFontPointSize(int fontSize) { int res = (fontSize * 72) / 96; return res; } Here’s how we skip past sz_or_Ord fields: /* Type: sz_Or_Ord A variable-length array of 16-bit elements that identifies a menu resource for the dialog box. If the first element of this array is 0x0000, the dialog box has no menu and the array has no other elements. If the first element is 0xFFFF, the array has one additional element that specifies the ordinal value of a menu resource in an executable file. If the first element has any other value, the system treats the array as a null-terminated Unicode string that specifies the name of a menu resource in an executable file. */ static u8* SkipSzOrOrd(u8* d) { WORD* pw = (WORD*)d; WORD w = *pw++; if (w == 0x0000) { // no menu } else if (w == 0xffff) { // menu id followed by another WORD item pw++; } else { // anything else: zero-terminated WCHAR* WCHAR* s = (WCHAR*)pw; while (*s) { s++; } s++; pw = (WORD*)s; } return (u8*)pw; } Strings are zero-terminated utf-16: static u8* SkipSz(u8* d) { WCHAR* s = (WCHAR*)d; while (*s) { s++; } s++; // skip terminating zero return (u8*)s; } To make the code more robust, we check the dialog is extended and has font information to patch: static bool IsDlgTemplateEx(DLGTEMPLATE* tpl) { return tpl->style == MAKELONG(0x0001, 0xFFFF); } static bool HasDlgTemplateExFont(DLGTEMPLATEEX* tpl) { DWORD style = tpl->style & (DS_SETFONT | DS_FIXEDSYS); return style != 0; } Changing font name It’s also possible to change font name but it’s slightly harder (which is why I didn’t implement it). WCHAR typeface[] is inline null-terminated string that is name of the font. To change it we would also have to move the data that follows it. The roads not taken There are other ways to achieve that. Dialog is just a HWND. In WM_INITDIALOG message we could iterate over all controls, change their font with WM_SETFONT message and then resize the controls and the window. That’s much more work than our solution. We just patch the font size and let Windows do the font setting and resizing. Another option would be to generate binary blog representing dialogs at runtime. It would require writing more code but then we could define new dialogs in C++ code that wouldn’t be that much different than .rc syntax. I want to explore that solution because this would also allow adding simple layout system to simplify definition the dialogs. In .rc files everything must be absolutely positioned. The visual dialog editor helps a bit but is unreliable and I need resizing logic anyway because after translating strings absolute positioning doesn’t work.

12 months ago 64 votes
How I implemented wc in the browser in 3 days

Building wc in the browser From time to time I like to run wc -l on my source code to see how much code I wrote. For those not in the know: wc -l shows number of lines in files. Actually, what I have to do is more like find -name "*.go" | xargs wc -l because wc isn’t a particularly good at handling directories. I just want to see number of lines in all my source files, man. I don’t want to google the syntax of find and xargs for a hundredth time. After learning about File System API I decided to write a tool that does just that as a web app. No need to install software. I did just that and you can use it yourself. Here’s how it sees itself: The rest of this article describes how I would have done it if I did it. Building software quickly It only took me 3 days, which is a testament to how productive the web platform can be. My weapons of choice are: Svelte for frontend Tailwind CSS for CSS JSDoc for static typing of JavaScript File System API to access files and directories on your computer vite for a bundler and dev server render to deploy For a small project Svelte and Tailwind CSS are arguably an overkill. I used them because I standardized on that toolset. Standardization allows me to re-use prior experience and sometimes even code. Why those technologies? Svelte is React without the bloat. Try it and you’ll love it. Tailwind CSS is CSS but more productive. You have to try it to believe it. JSDoc is happy medium between no types at all and TypeScript. I have great internal resistance to switching to TypeScript. Maybe 5 years from now. And none of that would be possible without browser APIs that allow access to files on your computer. Which FireFox doesn’t implement because they are happy to loose market share to browser that implement useful features. Clearly $3 million a year is not enough to buy yourself a CEO with understanding of the obvious. Implementation tidbits Getting list of files To get a recursive listing of files in a directory use showDirectoryPicker to get a FileSystemDirectoryHandle. Call dirHandle.values() to get a list of directory entries. Recurse if an entry is a directory. Not all browsers support that API. To detect if it works: /** * @returns {boolean} */ export function isIFrame() { let isIFrame = false; try { // in iframe, those are different isIFrame = window.self !== window.top; } catch { // do nothing } return isIFrame; } /** * @returns {boolean} */ export function supportsFileSystem() { return "showDirectoryPicker" in window && !isIFrame(); } Because people on Hacker News always complain about slow, bloated software I took pains to make my code fast. One of those pains was using an array instead of an object to represent a file system entry. Wait, now HN people will complain that I’m optimizing prematurely. Listen buddy, Steve Wozniak wrote assembly in hex and he liked it. In comparison, optimizing memory layout of most frequently used object in JavaScript is like drinking champagne on Jeff Bezos’ yacht. Here’s a JavaScript trick to optimizing memory layout of objects with fixed number of fields: derive your class from an Array. Deriving a class from an Array Little known thing about JavaScript is that an Array is just an object and you can derive your class from it and add methods, getters and setters. You get a compact layout of an array and convenience of accessors. Here’s the sketch of how I implemented FsEntry object: // a directory tree. each element is either a file: // [file, dirHandle, name, path, size, null] // or directory: // [[entries], dirHandle, name, path, size, null] // extra null value is for the caller to stick additional data // without the need to re-allocate the array // if you need more than 1, use an object // handle (file or dir), parentHandle (dir), size, path, dirEntries, meta const handleIdx = 0; const parentHandleIdx = 1; const sizeIdx = 2; const pathIdx = 3; const dirEntriesIdx = 4; const metaIdx = 5; export class FsEntry extends Array { get size() { return this[sizeIdx]; } // ... rest of the accessors } We have 6 slots in the array and we can access them as e.g. entry[sizeIdx]. We can hide this implementation detail by writing a getter as FsEntry.size() shown above. Reading a directory recursively Once you get FileSystemDirectoryHandle by using window.showDirectoryPicker() you can read the content of the directory. Here’s one way to implement recursive read of directory: /** * @param {FileSystemDirectoryHandle} dirHandle * @param {Function} skipEntryFn * @param {string} dir * @returns {Promise<FsEntry>} */ export async function readDirRecur( dirHandle, skipEntryFn = dontSkip, dir = dirHandle.name ) { /** @type {FsEntry[]} */ let entries = []; // @ts-ignore for await (const handle of dirHandle.values()) { if (skipEntryFn(handle, dir)) { continue; } const path = dir == "" ? handle.name : `${dir}/${handle.name}`; if (handle.kind === "file") { let e = await FsEntry.fromHandle(handle, dirHandle, path); entries.push(e); } else if (handle.kind === "directory") { let e = await readDirRecur(handle, skipEntryFn, path); e.path = path; entries.push(e); } } let res = new FsEntry(dirHandle, null, dir); res.dirEntries = entries; return res; } Function skipEntryFn is called for every entry and allows the caller to decide to not include a given entry. You can, for example, skip a directory like .git. It can also be used to show progress of reading the directory to the user, as it happens asynchronously. Showing the files I use tables and I’m not ashamed. It’s still the best technology to display, well, a table of values where cells are sized to content and columns are aligned. Flexbox doesn’t remember anything across rows so it can’t align columns. Grid can layout things properly but I haven’t found a way to easily highlight the whole row when mouse is over it. With CSS you can only target individual cells in a grid, not rows. With table I just style <tr class="hover:bg-gray-100">. That’s Tailwind speak for: on mouse hover set background color to light gray. Folder can contain other folders so we need recursive components to implement it. Svelte supports that with <svelte:self>. I implemented it as a tree view where you can expand folders to see their content. It’s one big table for everything but I needed to indent each expanded folder to make it look like a tree. It was a bit tricky. I went with indent property in my Folder component. Starts with 0 and goes +1 for each level of nesting. Then I style the first file name column as <td class="ind-{indent}">...</td> and use those CSS styles: <style> :global(.ind-1) { padding-left: 0.5rem; } :global(.ind-2) { padding-left: 1rem; } /* ... up to .ind-17 */ Except it goes to .ind-17. Yes, if you have deeper nesting, it won’t show correctly. I’ll wait for a bug report before increasing it further. Calculating line count You can get the size of the file from FileSystemFileEntry. For source code I want to see number of lines. It’s quite trivial to calculate: /** * @param {Blob} f * @returns {Promise<number>} */ export async function lineCount(f) { if (f.size === 0) { // empty files have no lines return 0; } let ab = await f.arrayBuffer(); let a = new Uint8Array(ab); let nLines = 0; // if last character is not newline, we must add +1 to line count let toAdd = 0; for (let b of a) { // line endings are: // CR (13) LF (10) : windows // LF (10) : unix // CR (13) : mac // mac is very rare so we just count 10 as they count // windows and unix lines if (b === 10) { toAdd = 0; nLines++; } else { toAdd = 1; } } return nLines + toAdd; } It doesn’t handle Mac files that use CR for newlines. It’s ok to write buggy code as long as you document it. I also skip known binary files (.png, .exe etc.) and known “not mine” directories like .git and node_modules. Small considerations like that matter. Remembering opened directories I typically use it many times on the same directories and it’s a pain to pick the same directory over and over again. FileSystemDirectoryHandle can be stored in IndexedDB so I implemented a history of opened directories using a persisted store using IndexedDB. Asking for permissions When it comes to accessing files and directories on disk you can’t ask for forgiveness, you have to ask for permission. User grants permissions in window.showDirectoryPicker() and browser remembers them for a while, but they expire quite quickly. You need to re-check and re-ask for permission to FileSystemFileHandle and FileSystemDirectoryHandle before each access: export async function verifyHandlePermission(fileHandle, readWrite) { const options = {}; if (readWrite) { options.mode = "readwrite"; } // Check if permission was already granted. If so, return true. if ((await fileHandle.queryPermission(options)) === "granted") { return true; } // Request permission. If the user grants permission, return true. if ((await fileHandle.requestPermission(options)) === "granted") { return true; } // The user didn't grant permission, so return false. return false; } If permissions are still valid from before, it’s a no-op. If not, the browser will show a dialog asking for permissions. If you ask for write permissions, Chrome will show 2 confirmations dialogs vs. 1 for read-only access. I start with read-only access and, if needed, ask again to get a write (or delete) permissions. Deleting files and directories Deleting files has nothing to do with showing line counts but it was easy to implement, it was useful so I added it. You need to remember FileSystemDirectoryHandle for the parent directory. To delete a file: parentDirHandle.removeEntry("foo.txt") To delete a directory: parentDirHandle.removeEntry("node_modules", {recursive: true}) Getting bit by a multi-threading bug JavaScript doesn’t have multiple threads and you can’t have all those nasty bugs? Right? Right? Yes and no. Async is not multi-threading but it does create non-obvious execution flows. I had a bug: I noticed that some .txt files were showing line count of 0 even though they clearly did have lines. I went bug hunting. I checked the lineCount function. Seems ok. I added console.log(), I stepped through the code. Time went by and my frustration level was reaching DEFCON 1. Thankfully before I reached cocked pistol I had an epiphany. You see, JavaScript has async where some code can interleave with some other code. The browser can splice those async “threads” with UI code. No threads means there are no data races i.e. writing memory values that other thread is in the middle of reading. But we do have non-obvious execution flows. Here’s how my code worked: get a list of files (async) show the files in UI calculate line counts for all files (async) update UI to show line counts after we get them all Async is great for users: calculating line counts could take a long time as we need to read all those files. If this process wasn’t async it would block the UI. Thanks to async there’s enough checkpoints for the browser to process UI events in between processing files. The issue was that function to calculate line counts was using an array I got from reading a directory. I passed the same array to Folder component to show the files. And I sorted the array to show files in human friendly order. In JavaScript sorting mutates an array and that array was partially processed by line counting function. As a result if series of events was unfortunate enough, I would skip some files in line counting. They would be resorted to a position that line counting thought it already counted. Result: no lines for you! A happy ending and an easy fix: Folder makes a copy of an array so sorting doesn’t affect line counting process. The future No software is ever finished but I arrived at a point where it does the majority of the job I wanted so I shipped it. There is a feature I would find useful: statistics for each extensions. How many lines in .go files vs. .js files etc.? But I’m holding off implementing it until: I really, really want it I get feature requests from people who really, really want it You can look at the source code. It’s source visible but not open source.

over a year ago 19 votes

More in programming

1995 Was the Most Important Year for the Web

The world changed a lot in 1995. And for the web, it was a transformational year. The post 1995 Was the Most Important Year for the Web appeared first on The History of the Web.

18 hours ago 3 votes
Solving a "Layton Puzzle" with Prolog

I have a lot in the works for the this month's Logic for Programmers release. Among other things, I'm completely rewriting the chapter on Logic Programming Languages. I originally showcased the paradigm with puzzle solvers, like eight queens or four-coloring. Lots of other demos do this too! It takes creativity and insight for humans to solve them, so a program doing it feels magical. But I'm trying to write a book about practical techniques and I want everything I talk about to be useful. So in v0.9 I'll be replacing these examples with a couple of new programs that might get people thinking that Prolog could help them in their day-to-day work. On the other hand, for a newsletter, showcasing a puzzle solver is pretty cool. And recently I stumbled into this post by my friend Pablo Meier, where he solves a videogame puzzle with Prolog:1 Summary for the text-only readers: We have a test with 10 true/false questions (denoted a/b) and four student attempts. Given the scores of the first three students, we have to figure out the fourth student's score. bbababbabb = 7 baaababaaa = 5 baaabbbaba = 3 bbaaabbaaa = ??? You can see Pablo's solution here, and try it in SWI-prolog here. Pretty cool! But after way too long studying Prolog just to write this dang book chapter, I wanted to see if I could do it more elegantly than him. Code and puzzle spoilers to follow. (Normally here's where I'd link to a gentler introduction I wrote but I think this is my first time writing about Prolog online? Uh here's a Picat intro instead) The Program You can try this all online at SWISH or just jump to my final version here. :- use_module(library(dif)). % Sound inequality :- use_module(library(clpfd)). % Finite domain constraints First some imports. dif lets us write dif(A, B), which is true if A and B are not equal. clpfd lets us write A #= B + 1 to say "A is 1 more than B".2 We'll say both the student submission and the key will be lists, where each value is a or b. In Prolog, lowercase identifiers are atoms (like symbols in other languages) and identifiers that start with a capital are variables. Prolog finds values for variables that match equations (unification). The pattern matching is real real good. % ?- means query ?- L = [a,B,c], [Y|X] = [1,2|L], B + 1 #= 7. B = 6, L = [a, 6, c], X = [2, a, 6, c], Y = 1 Next, we define score/33 recursively. % The student's test score % score(student answers, answer key, score) score([], [], 0). score([A|As], [A|Ks], N) :- N #= M + 1, score(As, Ks, M). score([A|As], [K|Ks], N) :- dif(A, K), score(As, Ks, N). First key is the student's answers, second is the answer key, third is the final score. The base case is the empty test, which has score 0. Otherwise, we take the head values of each list and compare them. If they're the same, we add one to the score, otherwise we keep the same score. Notice we couldn't write if x then y else z, we instead used pattern matching to effectively express (x && y) || (!x && z). Prolog does have a conditional operator, but it prevents backtracking so what's the point??? A quick break about bidirectionality One of the coolest things about Prolog: all purely logical predicates are bidirectional. We can use score to check if our expected score is correct: ?- score([a, b, b], [b, b, b], 2). true But we can also give it answers and a key and ask it for the score: ?- score([a, b, b], [b, b, b], X). X = 2 Or we could give it a key and a score and ask "what test answers would have this score?" ?- score(X, [b, b, b], 2). X = [b, b, _A], dif(_A,b) X = [b, _A, b], dif(_A,b) X = [_A, b, b], dif(_A,b) The different value is written _A because we never told Prolog that the array can only contain a and b. We'll fix this later. Okay back to the program Now that we have a way of computing scores, we want to find a possible answer key that matches all of our observations, ie gives everybody the correct scores. key(Key) :- % Figure it out score([b, b, a, b, a, b, b, a, b, b], Key, 7), score([b, a, a, a, b, a, b, a, a, a], Key, 5), score([b, a, a, a, b, b, b, a, b, a], Key, 3). So far we haven't explicitly said that the Key length matches the student answer lengths. This is implicitly verified by score (both lists need to be empty at the same time) but it's a good idea to explicitly add length(Key, 10) as a clause of key/1. We should also explicitly say that every element of Key is either a or b.4 Now we could write a second predicate saying Key had the right 'type': keytype([]). keytype([K|Ks]) :- member(K, [a, b]), keytype(Ks). But "generating lists that match a constraint" is a thing that comes up often enough that we don't want to write a separate predicate for each constraint! So after some digging, I found a more elegant solution: maplist. Let L=[l1, l2]. Then maplist(p, L) is equivalent to the clause p(l1), p(l2). It also accepts partial predicates: maplist(p(x), L) is equivalent to p(x, l1), p(x, l2). So we could write5 contains(L, X) :- member(X, L). key(Key) :- length(Key, 10), maplist(contains([a,b]), L), % the score stuff Now, let's query for the Key: ?- key(Key) Key = [a, b, a, b, a, a, b, a, a, b] Key = [b, b, a, b, a, a, a, a, a, b] Key = [b, b, a, b, a, a, b, b, a, b] Key = [b, b, b, b, a, a, b, a, a, b] So there are actually four different keys that all explain our data. Does this mean the puzzle is broken and has multiple different answers? Nope The puzzle wasn't to find out what the answer key was, the point was to find the fourth student's score. And if we query for it, we see all four solutions give him the same score: ?- key(Key), score([b, b, a, a, a, b, b, a, a, a], Key, X). X = 6 X = 6 X = 6 X = 6 Huh! I really like it when puzzles look like they're broken, but every "alternate" solution still gives the same puzzle answer. Total program length: 15 lines of code, compared to the original's 80 lines. Suck it, Pablo. (Incidentally, you can get all of the answer at once by writing findall(X, (key(Key), score($answer-array, Key, X)), L).) I still don't like puzzles for teaching The actual examples I'm using in the book are "analyzing a version control commit graph" and "planning a sequence of infrastructure changes", which are somewhat more likely to occur at work than needing to solve a puzzle. You'll see them in the next release! I found it because he wrote Gamer Games for Lite Gamers as a response to my Gamer Games for Non-Gamers. ↩ These are better versions of the core Prolog expressions \+ (A = B) and A is B + 1, because they can defer unification. ↩ Prolog-descendants have a convention of writing the arity of the function after its name, so score/3 means "score has three parameters". I think they do this because you can overload predicates with multiple different arities. Also Joe Armstrong used Prolog for prototyping, so Erlang and Elixir follow the same convention. ↩ It still gets the right answers without this type restriction, but I had no idea it did until I checked for myself. Probably better not to rely on this! ↩ We could make this even more compact by using a lambda function. First import module yall, then write maplist([X]>>member(X, [a,b]), Key). But (1) it's not a shorter program because you replace the extra definition with an extra module import, and (2) yall is SWI-Prolog specific and not an ISO-standard prolog module. Using contains is more portable. ↩

10 hours ago 3 votes
Market Ending Moves

Startup CEOs should ask themselves what crazy ideas can turn into a move that just ends a market's competitive dynamic

10 hours ago 2 votes
Why we won't hire a junior with five years of experience

We just opened a search for a new junior programmer at 37signals. It's been years since we last hired a junior, but the real reason the listing is turning heads is because we're open about the yearly salary: $145,849*. That's high enough that programmers with lots of experience are asking whether they could apply, even if they aren't technically "junior". The answer is no. The reason we're willing to pay a junior more than most is because we're looking for a junior who's better than most. Not better in "what do they already know", but in "how far could they go". We're hiring for peak promise — and such promise only remains until it's revealed. Maybe it sounds a little harsh, but a programmer who's been working professionally for five years has likely already revealed their potential. What you're going to get is roughly what you see. That doesn't mean that people can't get better after that, but it means that the trajectory by which they improve has already been plotted. Whereas a programmer who's either straight out of school or fresh off their first internship or short-stint job is essentially all potential. So you draw their line on the basis of just a few early dots, but the line can be steep. It's not that different from something like the NFL scouting combine. Teams fight to find the promise of The Next All-Star. These rookies won't have the experience that someone who's already played in the league for years would have, but they have the potential to be the best. Someone who's already played for several seasons will have shown what they have and be weighed accordingly. This is not easy to do! Plenty of rookies, in sports and programming, may show some early potential, then fail to elevate their game to where the buyer is betting it could be. But that's the chance you take to land someone extraordinary. So if you know a junior programmer with less than three years of industry experience who is sparkling with potential, do let them know of our listing. And if you know someone awesome who's already a senior programmer, we also have an opening for them. *It's a funnily precise number because it's pulled directly from the Radford salary database, which we query for the top 10% of San Francisco salaries for junior programmers.

18 hours ago 2 votes
A Data Engineering Perspective of LLMs

Data engineering is a field I would categorize as a subspecialty of software engineering. It shares the same concerns as software engineering—scalability, maintainability, and other “-ilities”—but its primary focus is on data. It’s a unique discipline because data is inherently messy, and as a result, no standard enterprise framework has emerged to dominate the space—and […]

4 hours ago 2 votes