Marginalia

> Maybe someone runs a search engine that only indexes crawled pages that have a X-Web-Classic header or whatever. Well, not exactly, but this is the next best thing: https://www.marginalia.nu/ The author is a HN regular.

NLnet is a great initiative. Among the numerous projects they have supported is Marginalia [1] search engine. 1. https://www.marginalia.nu/

https://www.marginalia.nu/ I should probably get around to making an update or two, but alas it's the dark part of the year and my inspiration is running on fumes.

Viktor Lofgren's blog is probably worth a read — he is developing the rather delightful marginalia search engine. https://www.marginalia.nu/log/

https://1mb.club has a lot of performance-focused websites, which generally implies not too much. If you use Chrome, there's also reading mode (accessible on right-clicking), which I don't use very often but which just gets you down to links and text. (No images, though.) https://marginalia.nu also has 'indie' sites which are pretty good with that, but for the big ones...good luck.

I think the major search engines guide us towards the dead parts of the Internet. It's unsurprising that Google (an ad agency) sends traffic to the ad-laden, dead parts of the web. Living parts of the web tend not to like ads (they are user hostile) while the dead parts are just trying to drive clicks. But the web is still alive, see: * https://www.marginalia.nu/ * https://www.ribbonfarm.com/2024/04/16/the-dark-forest-anthology-of-the… * https://alexsci.com/rss-blogroll-network/

Have you heard of https://www.marginalia.nu/ in general, and especially the https://search.marginalia.nu/ from there?

Safari doesn't like https://marginalia.nu. Probably because *.marginalia.nu is not valid for the base domain. Add it as a Subject Alt Name

“So it’s a search engine. It’s perhaps not the greatest at finding what you already knew was there. Instead it is designed to help you find some things you didn’t even know you were looking for.”[0] [0] https://www.marginalia.nu/marginalia-search/about/

Why should contribution to “the world”, the gargantua of global culture, history, and economy, be the yardstick of worth? I believe almost everyone has something valuable to contribute to their friends, family, neighborhood, or community. We can step away from the chaotic, nebulous global village and focus on the tangible human connections of actual village systems, where “the defining trait of a village is that it’s group of people where the average interaction over time is with people you’ve seen before.” https://www.marginalia.nu/log/82_killing_community/ https://news.ycombinator.com/item?id=36296882

I'd love to support, but Patreon link [0] on "supporting" page [1] is 404. Is another support option in progress to replace? [0] https://www.marginalia.nu/marginalia-search/supporting/patreon [1] https://www.marginalia.nu/marginalia-search/supporting/

Kudos. [0] is the best URL I have come across in the past few years [0] https://www.marginalia.nu/junk/just-a-fleshwound.webp

Bookmarked. Will revisit. Anyone else notice everything old is new again? Neocities[0], Marginalia Search[1], Project Gemini[2], etc There's many others I'm forgetting, and new ones popup on Hackernews each week. Is this just basic nostalgia, people wanting to recreate the dial-up days or even BBS days? [0] https://neocities.org/ [1] https://www.marginalia.nu/ [2] https://gemini.circumlunar.space/

I misunderstood what URL we were talking about. What I am seeing: https://marginalia.nu = https://crt.sh/?id=6046506678 (includes wildcard but NOT apex) https://search.marginalia.nu = https://crt.sh/?id=6125359537 (includes both wildcard and apex)

That's not the certificate I get served though (this one is from 5 february, the one served is from 25 january) do you actually get this certificate on https://marginalia.nu?

If the website is targeted towards international audience then its nice to have the first page links to content in english. All the four links in the main page https://www.marginalia.nu/ have links to non-english content which is not useful. Disclaimer: I am not a native english speaker. English is my second language.

Is your site/code on GitHub? I would be happy to give performance tips/tweaks. Also Fyi, https://marginalia.nu/ gives a certificate error (I know that's not the search site)

And of course there is always https://marginalia.nu

> It would only be feasible to build on top of existing libraries, which is the whole point here. I don't think you've made a single point that substantiates that claim and neither has anyone else, really. The person you're arguing with has built (as per their own estimation) more from scratch when it comes to the actual search engine core than anything else in their own project (https://marginalia.nu/). Honestly, it seems more like the people arguing a search engine is somehow "off limits" from scratch are doing so because they imagine it's less feasible, probably because they simply don't know or are in general pretty bad at implementing things from scratch, period. To a large degree there is also a wide difference in actual project scope. Remember that we are comparing this to implementing Space Invaders, one of the simplest games you can make, and one that is doable in an evening even if you write your own renderer. To talk about "webscale" (a term that is so deliciously dissonant with most of the web world which runs at a scale several 100 times slower and less efficient than things should be) is like suddenly talking about implementing deferred shading for the Space Invaders game. Even if this was something you'd want to explore later, it's just not something you're doing right now because this is something you're using to learn as you go.

https://www.marginalia.nu/weird-ai-crap/hn/ The data seems to suggest it. Anecdotally, I'm seeing a lot of green accounts posting nonsense. They generally do get flagged or moderated quickly though, so I wouldn't say they have a large effect overall, at least yet.

This isn't true, Marginalia Search has a crawler. The guy who runs the site writes a lot about the trials and tribulations of running a web crawler on his blog if you're interested. https://www.marginalia.nu/log/ I just checked and my site is listed despite me never manually submitting it. I guess if nobody links to your site, you would have to manually submit it due to it not showing up in crawls, but that's true about any search engine.

seen the win32 aesthetic used for higher-order reasons: separating "work mode" from playtime. An extra user to do so on the same machine helps. Connect the dots reading https://www.marginalia.nu/log/99_context/ and seeing the ui change in old vs new screencasts https://www.youtube.com/@ViktorLofgren/videos

just a sampling of my rss reader this morning: marginalia's blog for interesting tech problems building a search engine "specifically" for indie/small/old-web sites. (the search engine itself is a gold mine for exactly what you're looking for). https://www.marginalia.nu/log/ feuilleton for thoughtful posts on recent niche art history - http://www.johncoulthart.com/feuilleton/feed/ the spatial heritage review for advances in 3D models from an educational context https://nebulousflynn.substack.com/

I recently wrote an essay about this search engine, and its ranking algorithms. Initially Marginalia used an interesting variant of PageRank discussed in the original paper, called Personal Pagerank.[1] Currently pages are ranked with BM25. I think Personalized PageRank is still used for a new feature of Marginalia which is ranking pages based on similarity. I think this is already integrated into the website but there used to only be this testing page: https://explore2.marginalia.nu/ In any case I have a lot of respect for the creator. Marginalia has seen a lot of growth and it's been interesting reading the blogposts.[2] [1]: https://www.marginalia.nu/log/26-personalized-pagerank/ [2]: https://www.marginalia.nu/log/

I don't know if something has changed since then, but in 2023 the author did quit their job to start working on the search engine full time [0]. So yes, it's a passion project, but it's also not just a side project. [0] https://www.marginalia.nu/log/83_full_time/

There are already many project about search: - https://www.marginalia.nu/ - https://searchmysite.net/ - https://lucene.apache.org/ - elastic search - https://presearch.com/ - https://stract.com/ - https://wiby.me/ I think that all project are fun. I would like to see one succeeding at reaching mainstream level of attention. I have also been gathering links meta data for some time. Maybe I will use them to feed any eventual self hosted search engine, or language model, if I decide to experiment with that. - domains for seed https://github.com/rumca-js/Internet-Places-Database - bookmarks seed https://github.com/rumca-js/RSS-Link-Database - links for year https://github.com/rumca-js/RSS-Link-Database-2024

See the same sentiment from Marginalia : https://www.marginalia.nu/log/79-ikea-offramp/

Having an RSS indicator in the URL bar used to be the first step to subscribing a feed. How do you get it if its not there. When a feature is removed, slowly everyone will forget that it was even a thing once. Opera (my main browser back then) use to have so much stuff built in (RSS, email, IRC, note taking, mouse gestures). Chrome use to have a resizable extension bar recently. OneNote use to have a more colorful and compact UI. Start menu and taskbar used to be useful before windows 11/10/8. Reminds of an article posted here about removing less used keys/letters from keyboard. https://www.marginalia.nu/log/48-i-have-no-capslock/

You seem to be missing that that's already how it's implemented. https://www.marginalia.nu/log/69-creepy-website-similarity/

There's details in the first linked See Also -post: https://www.marginalia.nu/log/69-creepy-website-similarity/