Vidar Hokstad

I wrote one, though it kinda got away from me (I started very simple, but then started turning it into a Ruby compiler): https://hokstad.com/compiler On that note, while there's no new part, after finally revisiting it for the first time in a couple of years, it now compiles itself (from Ruby into 32bit x86). A few very critical bugs that I'm working on, and a lot of big omissions (no Regexp yet; the Ruby Regexp engine source is bigger than my entire compiler, nor any Floats; syntax lacks a lot of newer features) so it's very much not a usable Ruby at this point. Maybe one day if I get more time. If you find it interesting, I'd recommend looking at the earlier parts rather than the current source. Lots of the recent changes were very aggressively working around bugs to move it towards self hosting, and it's not pretty.

I was speaking metaphorically of Pascal being the LLVM of its day, more that it was a known base, portable, easy to explain and the first tool someone would reach for. Templated IR code that has been LZ compressed sounds excellent, Thumb++ or uop buffer engine. It sounds like from the post below, that the instructions could even be pathologically the uop or VLIW instruction form and the SDE will effectively discover the compact CISC for your specific workload. Why not push the decode engine into the instruction decoder? I found this awesome post that covers SDE, http://hokstad.com/semantic-dictionary-encoding I think one way to explore this would be to implement compressed binary loaders, like done for demos etc, but target wasm, wrap it in a decompressor and have the wasm program decompress itself as it executed perhaps on a method by method basis.

Do you have a link to your Ruby compiler? EDIT found it in your profile http://www.hokstad.com/compiler and submitted to HN. Great series of posts. About Python, it's a little older than Ruby (its design started in 1989) and Guido worked on the ABC language at the beginning of the 80s. IMHO that difference shows in many places, for example in having to explicitly pass self as argument in method definitions. Newer languages do without it and programmers don't get confused at all. All those double underscore methods and the general explicitness of the language smell of 80s and of C. Not necessarily a bad thing but reading some Python code is like getting on a time machine.

> If anyone knows of a better path to get from here (can't design my own language) to there (clarity and understanding) I would love to hear your ideas. http://www.hokstad.com/compiler was what made it make sense to me. I found the dragon book approach (where you start with this 4-stage design and you're supposed to implement a parser and lexer and all this because I said so) hopelessly confusing and unmotivated. Hokstad takes you through writing a compiler in a way that makes sense, the same way you'd write any other program.

I followed this in the early 90s and had a lovely time. It helped that Turbo Pascal was my language of choice though and might not be quite so helpful now although Pascal is a pretty good pseudocode.. It's not the same but Vidar Hokstad has been writing a series for several years now in Ruby: http://www.hokstad.com/compiler/ .. and other resources aplenty: http://stackoverflow.com/a/1672/3951

Another one I recommend, because it starts with codegen and works its way back: Writing a compiler in Ruby, bottom up: http://www.hokstad.com/compiler

The reason I love Ruby is because I do consider it to be 'an acceptable Lisp.' For extra fun, you can write really nice sexps in Ruby by just using arrays with a symbol as the first element... [:puts, [:+, 2, 3]] I wrote a tiny compiler with an IR that did that. Stole the idea from http://www.hokstad.com/compiler

I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth! Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropriate. The last "rant" was here, about meta-languages only being suitable for toys: https://news.ycombinator.com/item?id=13040682 vidarh also responds with his problems parsing Ruby (a "real" language): http://hokstad.com/compiler The blog is based on my experience in writing a parser for bash. I ported the POSIX shell grammar to ANTLR, but it fell ridiculously short of being a production quality parser. I believe it's essentially impossible to write a bash parser with ANTLR. But I don't hear about any alternatives. All I heard is yacc/bison for bottom up parsers, and ANTLR for top-down. Shell needs a top-down parser because it's an interactive language (the PS2 prompt) and for completion. But it doesn't appear there is any other "serious" meta-language for top-down parsers other than ANTLR? It certainly is the best documented and longest-lived, but I am surprised how far short it falls for many tasks. Part of the rant will be a survey of "real" language parsers (i.e. take the top 20 TIOBE language implementations) and see that they almost all use…

Also cool to play around with are the various options that let GCC (and presumably LLVM) show you the various compilation stages of your C code. You can even spit out the C and resulting assembly side-by-side. I haven't had the time to play with this, but Part 4 of the excellent "Unix as IDE" series [1] goes into it and I'm sure there's more around the web. Another really fun way to get into the underlying assembler that the C compiler generates is Vidar Hokstad's "Writing a compiler in Ruby, bottom up" [2]. This series involves writing little C functions, compiling them to the simplest assembly you can get, then writing a ruby compiler (in Ruby!) that emits that assembly. Some people have objections to the approach, but it's really quite nifty. I especially like it because it's really refreshing to see a compiler tutorial that doesn't start with the lexer and parser. [1] http://blog.sanctum.geek.nz/unix-as-ide-compiling/ [2] http://www.hokstad.com/compiler

This is not really accurate. When using java, you will not use JSP, JSTL, JSF, Play, Wicket, STripes, Tapestry, etc, all at the same time. Unless you are crazy. The same way, in Ruby, you will not use ERB, Haml, Erector, Markaby, or the other (at least) 19 template engines out there (http://www.hokstad.com/mini-reviews-of-19-ruby-template-engines.html) But this comparison is not actually fair, because you mixxed template engines with frameworks. To be fair, you'd have to add Sinatra, Camping, Rails, Merb, Ramaze, Padrino and Hobo to the Ruby list, and CakePhp, Zend and Symfony (at least) to the Php list. It is not which language has the most options of framework/template engines, but which has the best optins. JSP is one of the worst template engines on Java land, and this is why it is not used as much (well, there is also the fact that it has a very bad reputation, and JSF is considered its successor, but it is worse nonetheless).

The article mentions: > Slim binaries were invented by Michael Franz in the early 1990s. They were motivated and opposed to the fat binaries invented by Apple during the transition from 68k to PowerPC architectures. OMI provided portable code based on a compressed version of the abstract syntax tree. If you are interested in pursuing that topic, Vidar Hokstad wrote a nice blog post about it a couple of years back: https://hokstad.com/semantic-dictionary-encoding

I'm a bit sad to see that it's the only article tagged: http://hokstad.com/semantic-dictionary-encoding Did you ever get around to exploring SDE's in a Ruby context?

Thanks for the pointer. It led me to a few useful links. Michael Franz has a personal web page at http://www.michaelfranz.com. His thesis is at http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1424. I think http://hokstad.com/semantic-dictionary-encoding is your work, so I assume it is a good description of what I can expect from reading that thesis.

I like this book a lot - especially making it available to read free in HTML. It's good to see practical information about parsing and grammar rather than plainly theoretical - although there's room and a for need both. Another similar read is Vidar Hokstad's blog series - Writing a compiler in Ruby [1] which was first submitted here 9 years ago! [2] [1] http://hokstad.com/compiler [2] https://news.ycombinator.com/item?id=319337

I don't know Smalltalk well enough to say for sure, it's just my rather vague impression that Smalltalk is more compiler-friendly, likely based mostly on what I've read over the years about this, for example looking at: http://lambda-the-ultimate.org/node/2606 Ruby permits adding methods to individual objects; in Smalltalk, all methods reside in classes. In Ruby, it is practical and somewhat useful to add methods dynamically; in Smalltalk, the practice is generally to treat the methods and classes as static. Have also a look at: http://www.hokstad.com/the-problem-with-compiling-ruby I don't think all those 5 problems apply equally well to Smalltalk, do they?

Here's something similar, more tutorial-like in Ruby: http://www.hokstad.com/compiler

https://jrock.us/ https://jgc.org/ http://kaledavis.com/ https://aegir.sexy/ https://www.gwern.net/ https://jacquesmattheij.com/ https://danwin.com/ https://www.kalzumeus.com/ https://www.magic.do/ https://edweissman.com/ https://jakeseliger.com/ http://www.fogus.me/ https://www.stavros.io/ https://jasonlefkowitz.net/ https://minimaxir.com/ https://www.robinwhittleton.com/ https://ciexinc.com/ https://rongarret.info/ https://williampietri.com/ https://bpier.re/ https://joshtriplett.org/ https://ooer.com/ https://feross.org/resume/ https://www.brandons.me/ https://benjaminpbreen.com/ http://hokstad.com/ https://www.scott-a-s.com/ https://www.bermita.com/ http://tlrobinson.net/ https://ldpreload.com/ https://dotat.at/ http://xavvy.com/ https://correa.in/ https://adriann.github.io/ https://leclan.ch/ https://joeyespo.com/ https://pius.me/ https://adamierymenko.com/ https://drewdevault.com/ https://battlepenguin.com/ https://chris.improbable.org/ https://www.snellman.net/ https://ocdtrekkie.com/ https://www.cryptolotus.com/ https://www.gregkogan.com/ https://david.kitchen/ In no particular order. From: https://jessimekirk.com/blog/hn_users_links/. I went through about half of the list looking for personal websites to use as inspiration for my own.

Is this it? https://hokstad.com/semantic-dictionary-encoding

Also, a bunch of resources I found useful: http://treetop.rubyforge.org/ http://www.complang.org/ragel/ http://en.wikipedia.org/wiki/Abstract_syntax_tree http://en.wikipedia.org/wiki/Compiler http://en.wikipedia.org/wiki/Parsing http://en.wikipedia.org/wiki/Parsing_expression_grammar http://www.hokstad.com/writing-a-compiler-in-ruby-bottom-up-step-1.htm…