Full Width 
Shortcuts 
Sign Up 
Log In 
Articles 
Reading List More from John Salvatier
Ray Bradbury once explained with a poem why he writes science fiction and why space travel is so important to him. It is perhaps my favorite poem. Text version: The fence we walked between the years We ached and almost touched that stuff; O, Thomas, will a Race one day stand really tall Short man, Large dream I love this poem deeply for how dumb his reason is. He’s very straightforward about why he dreams of space travel ‘Twould teach us, not to, never to, be dead Advancing space travel is a silly way of try to escape death, but when the mind really really wants something, it clings to the best plan it can find for achieving it. Even if that plan is very very dumb. I see a lot of honor in this poem because while many hope to escape death, few are willing to admit to themselves, much less talk openly about their dumb plan for it. Their friends and family would think they were foolish, naive and a bit suspect – only villains want to cheat death. But Bradbury was willing to dream openly anyway. His courage makes it easier for us to have courage too. I have a similar awed respect for the child who whispers quietly to herself when she’s alone ‘I wish mom wouldn’t hit me’. They both have the virtue of looking directly at a terrible darkness they are powerless in front of and whispering ‘I wish that weren’t there’. The poem is also tragic now that he’s dead. It reminds me of all those who have gone before me, longing futilely and silently, and now he has joined them. My mother and the many billions of other humans who preceeded me. And hoped by stretching tall that they might keep their land I, too, long to keep my soul.
Followup to: Words as Mental Paintbrush Handles, Guessing The Teacher’s Password Jessica Taylor recently wrote a description of Paul Christiano’s and MIRI’s differing driving intuitions for thinking about the AI alignment problem. Jacob Steinhardt observes that the “do cognitive reductions” intuition seems to be at the heart of MIRI’s thought and the “search for solutions and fundamental obstructions” intuition at the heart of Paul’s thought. As I read his comment, I noticed myself make an error I’ve made before: thinking I get the intuitions by mere virtue of not thinking they’re crazy. I call this The “I Already Get It” Slide, and I suspect this error happens to people all the time but passes unnoticed. This is unfortunate because the error prevents you from actually absorbing other’s intutions, and absorbing other’s intuitions is important for doing anything hard. Jessica describes Search For Solutions And Fundamental Obstructions like this: Almost all technical problems are either tractable to solve or are intractable/impossible for a good reason. […] If the previous intuition is true, we should Search For Solutions And Fundamental Obstructions. If there is either a solution or a fundamental obstruction to a problem, then an obvious way to make progress on the problem is to alternate between generating obvious solutions and finding good reasons why a class of solutions (or all solutions) won’t work. In the case of AI alignment, we should try getting a very good solution (e.g. one that allows the aligned AI to be competitive with unprincipled AI systems such as ones based on deep learning by exploiting the same techniques) until we have a fundamental obstruction to this. Such a fundamental obstruction would tell us which relaxations to the “full problem” we should consider, and be useful for convincing others that coordination is required to ensure that aligned AI can prevail even if it is not competitive with unaligned AI. As I thought about Paul’s Search For Solutions And Fundamental Obstructions intuition, a justification easily came to mind — a non-verbal feeling that it looked like other well-accepted problem solving strategies. This justification was easy, familiar and wrong. There is no way that “it looks like other accepted strategies” is actually the reason Paul thinks finding fundamental obstructions is central. And yet it was very easy for me to mentally slide from getting the conclusion and not immediately thinking it’s crazy, into thinking I also got the intuitive argument that generated it. If I had to guess at Paul’s actual intuitive reasons, I would guess something like this In Computer Science Theory, whenever there have been these kind of hard and confusing problems and people have tried to solve them, they’ve always turned out to either be possible or have some very revealing fundamental problem. For example, here are 4 clear examples. Furthermore, this makes intuitive sense because X. Also, this is also the case in these 3 other fields. And AI alignment looks a lot like these fields because it has Y and Z in common.“ But I also bet that not only will Paul have a more detailed argument, but also he will use a different ontology in a way that makes the argument meaningfully different. The argument is not yet compelling to me. Now, perhaps his arguments sound weak or just boring to you. How could a useful intuition be consistent with weak sounding arguments? To answer, put yourself in Paul’s shoes, and ask yourself what could explain weak or boring sounding arguments? Maybe you have a strong but difficult to articulate intution – maybe a mental picture of how different parts of the research process move against each other. Or maybe you can articulate your intuition, but when you do people quickly offer counterarguments that are — sigh — totally off topic. They nod along as if understanding, but then go right back to what they were doing before. You can probably imagine your conclusion being wrong, but not your insight being irrelevant. If Paul is at least as sensible as you are and his arguments sound weak or boring, you probably haven’t grokked his real internal reasons. Your intuitive mental picture of how parts of the research process moves is shaped differently than his. Maybe you’re even using different piece. If so, then it is not surprising that you come to different conclusions. You don’t even have the machinery to come to his conclusion. Maybe instead you think that getting his intuitive reasons from him doesn’t matter. After all, now that I know what Search For Solutions And Fundamental Obstructions means, I can just check that it should be a central strategy myself. But without an intuitive model of why it should be a central strategy, to check I would probably have to do computer science theory for at least a few months. Without my own intuitive model pulled from Paul’s intuitive model, there’s little to distinguish Search For Solutions And Fundamental Obstructions from a near-infinite variety of nearby strategies like “search for solutions and obstructions on complexity problems” or “search directly for fundamental obstructions”. Intuitive models let us cut down our uncertainty in great swaths by concentrating our probability on simple hypotheses. With my own intuitive model, checking often just requires seeing a few well chosen examples, or even just thinking back on past problems. All this is to say that Paul almost certainly has a valuable intuitive reason for his position. If I don’t catch my slide from understanding the conclusion to thinking I understand the argument, I’ll never notice that there’s something more to absorb. There’s a world of difference between understanding what Search For Solutions And Fundamental Obstructions means, and understanding the intuition that generates it. A difference, in other words, between understanding the conclusion and understanding the argument for it. If you mistake the conclusion for the argument, you will never get the argument. This reasoning doesn’t just apply to Paul and his intuitions, it applies to anyone who you think is about as reasonable as you. If they avoid errors about as well as you, then it would be silly to think that their intutions don’t point to real insight about the world. This also applies nicely to MIRI’s intuition that doing Cognitive Reductions is the main thing that can push AI alignment research ahead. Jessica describes Do Cognitive Reductions like this: Cognitive Reductions are great. When we feel confused about something, there is often a way out of this confusion, by figuring out which algorithm would have generated that confusion. Often, this works even when the original problem seemed “messy” or “subjective”; something that looks messy can have simple principles behind it that haven’t been discovered yet. Again, it is tempting to gloss over the fact that cognitive reductions are useful but not central, since we do already agree to some extent. But consider: if I were in their position, what kind of intuitions would actually lead me to think that Cognitive Reduction is so central? It couldn’t be just a stronger version of the belief that I already have, that would just make me think its somewhat more useful, rather than something to base my whole strategy around. Only a new argument could make sense of that. If I go argue with MIRI without noticing that there’s an argument I’m missing, we’ll just go around in circles. I suspect that The “I Already Get It” Slide happens all the time and passes unnoticed. That people mistake a person’s conclusions with their intuitive reasons and don’t end up absorbing their real arguments, even when they have insight. That would explain why peoples opinions converge so slowly.
More in science
One reason it's been so hard to directly detect the dark matter is that we're not sure what it is. […]
Back in the ancient mists of time, scientists and mathematicians would circulate preprints of their articles among friends and colleagues via the postal service, as a courtesy, to get feedback and to try to make sure people in the community were aware of their forthcoming work. As the wikipedia entry says, with the advent of widespread LaTeX and the development of the web, Paul Ginsparg (then at LANL) put together an html-based site for electronic sharing of preprints, initially at xxx.lanl.gov (back before "xxx" in URLs was the kind of thing filtered and blocked by employers). In 2001 he moved to Cornell, and by then the lab was perhaps relieved to see the site, rebranded as the arXiv, shift to Cornell's library/repository infrastructure. PIs my age remember (fondly? maybe?) the old days of the arXiv, when Prof. Ginsparg's rather dry sense of humor pervaded the site. The skull-and-crossbones logo. The "help"/FAQ pages that basically said, "if you can't figure out how to .tar.gz all of your necessary LaTeX files, and you can't figure out how to make your .eps figures small, maybe you should reconsider whether you're smart enough to be sharing your ideas here". The arXiv was for preprints, without peer review (though interesting follow-on sites like Scirate now exist for organized commenting on the articles). From these modest beginnings, the arXiv has grown enormously, including imitators/spin-offs such as chemrxiv and biorXiv and socarXiv. The arXiv has recently become an independent nonprofit, hired a CEO, secured multiyear philanthropic support, and hosts over 3 million articles. It's been interesting seeing some level of complaints online about some of these steps, but when the audience is so large, not everyone is going to be happy. Rapid growth has been a major issue - see here for a graph of monthly submissions: The exponential rise (except for a slight pandemic-correlated shoulder) has been problematic, especially recently. One hallmark of the arXiv over the years has been its ability to function with comparatively minimal need for "moderation". Early on, one consequence of the minimalistic help and moderate technical entry barrier was that it was unusual for fringe/pseudoscience to make its way onto the server. (Hence the establishment of viXra.) With the ease of cranking out properly formatted readable manuscripts using AI, clearly the arXiv has been struggling. If 10-15% of submissions need some kind of human intervention or review, the support needs are rapidly outpacing the limited count of support staff. There can be substantial backlogs. To help deal with this, the arXiv recently updated its policies regarding AI-generated content (and AI cannot be a co-author, because the AI tools cannot take responsibility for content), and most recently has had to limit submission rates to two papers per month per submitting author. These moves, too, have drawn some criticism (e.g. here). Personally, I think the operators of the arXiv face an incredibly challenging environment and are doing the best they can - the idea that they are making moves because they are establishment sticks in the mud who don't understand the New Way of Doing Science is just wrong-headed. It's completely unclear where all this is heading. Exponential growth in nature signals instability and does not continue forever. If proponents of very heavily AI-driven research want to establish a repository specifically for that work, that's up to them. [It is very on brand for the hard core AI advocates to argue that the arXiv is somehow morally obligated to host everything (regardless of hardware or personnel costs) so that future AI tools can read everything (a repository growing too quickly for human researchers to keep up) and summarize it.] One overarching point that should come up in any arXiv discussion: The arXiv has become a global repository for an enormous amount of human knowledge, without charging anyone publication fees. This should make interested parties think reallllllly hard about economic models of for-profit publishers.
Is philosophy real? We sent our correspondent to find out.
Stephen J Gould (still my favorite science essayist) wrote an excellent article in 1985 (Red Wings in the Sunset, later published in his book, Bully for Brontosaurus) about artist and naturalist Abbott Handerson Thayer. Thayer wrote about how animals use coloration as camouflage – what he called “cryptic coloration”. His ideas were solid, but he made a classic mistake that scientists sometimes make, overapplying their key discovery. Thayer argued that all animal coloration is cryptic. For example, he argued that flamingos are pink because it hides them in the setting sun (hence the title of the essay). This is a transparently absurd argument, and it shows how Thayer tried to shoehorn all evidence into his preferred and absolute narrative. It is better to assume that nature is complex, and all explanations are at best partial (unless proven otherwise). Animal coloration, in fact, can serve many different purposes, only one of which is camouflage. Thayer also struggled with the male peacock, for example. Butterflies appear to be another example. Actually, many butterflies are camouflaged on the underside of their wings, so that when they are at rest with their wings up they tend to blend into their surroundings. But the top side of their wings are often very colorful and not camouflaged at all. One assumption is that the brightly colored part of their wings is to attract mates. This may be true, but that does not mean the coloration does not serve another function. Often animals use visual cues when choosing their mates that are markers for health and success. As evidence that butterfly wing color may be serving a survival benefit, if you look at birds that feed on insects during flight, they target dully-colored moths much more than brightly colored butterflies, even though the butterflies should be easier to see. A recent study tests the hypothesis that the brightly colored and patterned top side of butterfly wings may have evolved to produce an optical illusion to confuse predators. The idea of using optical illusions as visual protection in animals is not new. For example, zebra stripes allow zebras to hide in the herd, confusing predators as to where one zebra ends and another begins. Stripes on zebras and snakes may also serve to confuses predators about their direction of motion, but this hypothesis has not been tested previously. The researchers started by filming butterflies taking off using high speed cameras. They found that the wing patterns created a powerful “barber pole” illusion. The stripes on a barber pole look like they are moving up or town even when the pole is just spinning. Similarly, the wing patterns combined with the way butterflies move their wings and their flight dynamics combine to create a similar barber pole illusion, making the butterfly look like it is moving down when it is in fact moving up. They also showed that this strategy is phylogenetically widespread. They then did modeling in silico and showed digital creatures converge on butterfly-like patterns. To understand how effective this strategy can be it’s important to understand how catching a butterfly in midflight works. Butterflies have a very jumpy pattern of flight. In order to grab them in flight, a bird will have to zero in on their exact location with a few hundred millisecond and millimeter precision. If the butterfly suddenly zigs while the bird perceives that they zagged, the birdy will miss. Alternatively they may make only a glancing blow or grab an edge of a wing rather than their body. Either way, the butterfly lives another day and the bird goes hungry. In zebras this effect has been referred to as the “visual dazzle” strategy. Now there is some empiric evidence that this works not just by confusing predators, but by creating a specific optical illusion. Zebras will also zig-zag to evade predators, and misjudging that last second movement can cause a pouncing lioness to miss. There are two specific illusion effects at work – the aperture effect and spatiotemporal aliasing. The aperture effect refers to the brain’s processing of visual information through a limited field of view. The visual system has a hard time processing many moving stripes, and specifically will confuse the direction of movement (this is the barber pole effect). So a predator may miss a zebra’s vertical movement, for example, and perceive all movement as perpendicular to the stripes. They may also misinterpret the angle of movement and only perceive the perpendicular motion. Spatiotemporal aliasing has to do with ratio of the movement with the “refresh” speed of the brain’s visual processing. You have likely seen this with spinning wheels that have spoke-like features. As the wheel slows down, at one point the spinning will appear to stop completely, and then will appear to spin backwards. This is simply an artifact of your brain’s visual processing speed. Now imagine being surrounded by a field of rapidly moving and zig-zagging stripes, and your brain trying to make sense of all this information, while trying to compensate for these powerful optical illusions. Butterflies don’t have a herd to hide in, but they do have the added element of their flapping wings. Not only are they moving in a way to maximize these optical illusions, their wings are also doing this, while alternating top-side and bottom-side. Some butterflies have bright spots on their colorful upper wings, that will flash as they flap their wings, causing another type of dazzling disorientation. I will end by returning to my original point – do not be limited in the types of explanations that you reach for when trying to understand nature. Nature is not so limited. Animals do not just use coloration for camouflage and attracting mates. They can also use their coloring for thermoregulation, for mimicking other animals, for producing a danger-signal to would-be predators, and to communicate with other members of their species. It can communicate mood, danger, or social status. Now we have to add optical illusions to the list. There may be other strategies yet to be discovered or imagined. The post Butterflies Are Masters of Illusion first appeared on NeuroLogica Blog.
