IPFS Blog

True, which is absolutely sufficient for most use cases. I'm currently doing some work for ML and data science companies where full reproducibility and introspection is very much desired. So you need to run your own source cache to provide that guarantee, because you can't count on cache.nixos.org still providing the source code from a package built 4 years ago. But that's why I love the IPFS cache efforts. [1] Running your own node to pin all required sources should then be relatively easy. [1] https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/

I would rather look at successful projects with potential, like IPFS[0] and Peertube[1], instead of failed ICO money grabs. [0] https://blog.ipfs.io/ [1] https://joinpeertube.org/

Databases, and dynamic content in general, can be done with/on IPFS. Take look at OrbitDB (https://github.com/orbitdb/orbit-db) - "Distributed peer-to-peer database for the decentralized web" or their blog post "Decentralized Real-Time Collaborative Documents - Conflict-free editing in the browser using js-ipfs and CRDTs" (https://blog.ipfs.io/30-js-ipfs-crdts.md). And all that works in the browser without running a local IPFS in the background. That's pretty amazing imo.

(Disclosure: I work for the Filecoin Foundation/Filecoin Foundation for the Decentralized Web). I do actually agree that the privacy and anonymity aspects of IPFS are not well- conveyed. I think people get hooked on the "censorship-resistant" nature of decentralized systems, without understanding that even if you have multiple sources, for instance in a content-addressable network like IPFS, aggressive censorship systems have other strategies to dissuade dissemination or punish readers. You always have to be thinking a few steps ahead. Services like Tor and, I hope, the IPFS network both try to convey what threat models they are useful for, and which they are not, but it's really hard to stop overenthusiastic re-statements that give them super-powers they do not, in fact, possess. That said, there's a bunch of careful thinking right now going on about how IPFS's privacy story could be improved: https://blog.ipfs.tech/ipfs-ping-2022-recap/ has a couple of sessions on this, and is a great summary of some other recent developments in the space. One of those improvements is in the point about nodes being high CPU, RAM, etc. (I actually find this to be more of a challenge when running the full IPFS Go node locally on my desktop, rather than on a VPS; it requires some tweaking.) The strategy right now is to encourage more implementations of IPFS to cover more use-cases; the original go-ipfs had to do everything, including maintaining some legacy decisions. Nowadays, there's a…

Matrix mitigates the threat of a hub & spoke model by not being hub & spoke (particularly in P2P!) :) > I am not sure that spanning trees or gossip protocols solve the problem I'm describing, but, if they do, I'd appreciate if you could elucidate further. Perhaps the confusion here is the expression "hub and spoke" which sounds to me like a static centralised star topology routing model. My point is that if you have 1M nodes trying to share data (e.g. follow a celebrity's personal pubsub topic), you clearly need a smarter routing algorithm than full mesh (where the celeb's node would have to do 1M parallel pokes to send its messages). You're completely right that one solution is a hub-and-spoke model (where the celeb's node would poke a big centralised hub somewhere, which would then relay the poke out to 1M followers on spokes) - but as you point out, the hub becomes a centralised chokepoint of failure/control/privacy-violation etc. So, the main other options I'm aware of are either to arrange your 1M nodes into a spanning tree (or overlapping spanning trees) of some kind, as Yggdrasil does (their current one looks like https://yggdrasil-map.cwinfo.org/#, although only ~300 nodes are live atm)... or you let the nodes self-organise into some kind of hierarchy based on gossiping and fan out the messages that way (as per https://blog.ipfs.io/2020-05-20-gossipsub-v1.1, and some of the experimental routing stuff we've been doing with Matrix).

Saying there's "no business value" in tools coming out of this space/building on IPFS is nonsense. Sure, the vast majority of people won't care about using a tool because it's decentralized in and of itself, but they _will_ choose tools that give them features they care about - like working better offline, having faster downloads/uploads, being more reliable and trustworthy, preserving their agency and control, etc. These implications from building a distributed system (ex able to collaborate when disconnected from central servers) are strong business reasons for folks to use one service over another, but that isn't clear from saying "it's decentralized". Look at Netflix using IPFS for faster container image distribution (business value: more productive developers - https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distrib… Audius & Matters.news using IPFS to give content creators more ownership over their own data, Berty & OpenBazaar using IPFS for private messaging/marketplaces -- clearly there is business value in removing middle-men/central points of failure & control in these applications. You absolutely can incentivize others to pin content. Check out Infura (infura.io), Pinata (pinata.cloud), Temporal (temporal.cloud), and Eternum (eternum.io) - these are all services you can pay to host your IPFS data with reasonable uptime. They have an incentive to keep your content around because you pay them to. Filecoin is a distributed solution to that (and…

I believe the problem is the lack of protocols implemented in browsers. It's needlessly fragile to have a single route to the data. "Web pages only last about 100 days on average"[1] http was nice, we gave it a good spin, it was certainly close enough for the cigar but in all honesty... where you have 100 days to find the content you want to consume it is not even a reasonable approach. I bet a lot of it is still available some place but whoever archived it legally may not distribute it without permission. I really don't care if TOR, IPFS, freenet or zeronet work out of the box. If it means access to more content its great. I don't even know how to use gopher atm. Good stuff is happening[2] but where they apparently chose to make it an extension is pretty lame. also.. If the user types "salvation army" into the browser we know what they want. Selling the rights to deny access is not what we need. [1] - http://blog.archive.org/2015/02/11/locking-the-web-open-a-call-for-a-d… [2] - https://blog.ipfs.io/2019-10-08-ipfs-browsers-update/

This looks neat! IPFS contributor here. I had a look at your code and saw how you handle downloading from multiple gateways. There's a better way to this which also leverages direct P2P retrieval in browsers, which is now a thing with IPFS! [0] If you just want to fetch, checkout @helia/verified-fetch[1] which gives you a Fetch like API that accepts CIDs. It handles all the content routing and retrieval p2p magic and can help reduce reliance on gateways. You can also pass it gateways in case it can connect to providers directly (due to transports) [0] https://blog.ipfs.tech/2024-shipyard-improving-ipfs-on-the-web/ [1] https://github.com/ipfs/helia-verified-fetch/tree/main/packages/verifi…

we don`t use a full IPFS, but more like a lite version. it allow us to retrieve data quickly, cache and duplicate. check this article https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-container-distr… When it comes to scale, ipfs basic discovery mechanism (the dht) does not do that well. This is why we have services like tns (which is a replicated registry) that allows it to scale.

> I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy, Imgur, CivitAI), don't use it. Netflix uses IPFS. [0] [0] https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-container-distr…

> Linux package distribution FWIW it seems like NixOS at least tried or is trying: https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/

Why don't they just use IPFS? I have seen Wikipedia put on IPFS and become uncensorable: https://blog.ipfs.tech/24-uncensorable-wikipedia/ I have never understood why it's called "wiki" leaks... what is wiki about it?

Not yet. There have been some efforts around adding IPFS support to Nix, as well as making its storage fully content addressable, which would allow for peer-to-peer archival and distribution of source. https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/ Guix, a similar project to Nix, has been tackling this head on by automatically falling back to the Software Heritage archive when the original upstream is unavailable: https://guix.gnu.org/en/blog/2019/connecting-reproducible-deployment-t…

I feel the IPFS blog post gave me a glimpse of how blockchain can be cool. In certain cases, the need for decentralization is so strong that it makes sense to build a blockchain system for it (or other secure P2P network thingy). For example, authentication/identity using blockchain technology seems interesting to me. [1] https://blog.ipfs.io/decentralizing-the-internet-s-root/

Almost every word in this section for starters: https://blog.ipfs.io/decentralizing-the-internet-s-root/#an-internet-o… Did you read the article?

most of wikipedia content is in public domain, how hard is it to "fork" wikipedia and start a new version without the army of their power editors and biased "admins", i.e. take an initial snapshot of the content as a seed but get rid of the centralized bureaucracy and censorship. Edit: looks like there is IPFS version https://blog.ipfs.io/24-uncensorable-wikipedia/

Some links without comment: - https://www.npmjs.com/package/ipfs-npm - https://github.com/ipfs-shipyard/apt-on-ipfs - https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distrib…

I'm going to shamelessly take this thread as an opportunity to link the work we recently did with IPFS that didn't stick when i tried submitting it before. https://blog.ipfs.io/2020-09-08-nix-ipfs-milestone-1/ I'm very keen on seeing IPFS succeed, and (unlike with economics). I think a "trickle down" approach of targeting developer workflows in hopes that devs "reverse dogfood"[1] is very viable. [1]: take the thing they use themselves and make it their program use it.

> You can pull docker images from ipfs? How? Like this https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distrib… Note that this is also similar to Kraken[1] from Uber, and Dragonfly[2] from Alibaba. Facebook also does container and artifact distribution using BitTorrent, but I can't find a good reference to it. [1] https://eng.uber.com/introducing-kraken/ [2] https://d7y.io/en-us/

I just realised that "recently" happened at the end of 2018 https://blog.ipfs.io/53-go-ipfs-0-4-18/ - oops. There was also a change later which turned off nodes being the middleman by default, but not sure which version. It's supposed to be much better these days, but I haven't tried it again in a few months.

You can torrent a full snapshot https://wiki.kiwix.org/wiki/Content_in_all_languages And also download snapshots from IPFS https://blog.ipfs.io/24-uncensorable-wikipedia/ I guess the issue with snapshots of large datasets is that they go out of date, and downloading the whole data set regularly is a bit painful

This was actually one of the first proofs of concept for IPFS, a decentralized file hosting platform built by Juan Benet at of Protocol Labs. https://blog.ipfs.io/24-uncensorable-wikipedia/

It's supposedly a top priority for IPFS for 2019: https://blog.ipfs.io/78-ipfs-2019-roadmap/

> * De-centralized hosting of the packages (news to come soon!) Given that "integration in package managers" is a big point in the IPFS 2019 Roadmap[0], I'm going to guess those two are related. [0]: https://blog.ipfs.io/78-ipfs-2019-roadmap/

I think mirroring things to IPFS is a perfect use case for this. Especially with IPFS companion[1], it's really easy for visitors to access your content. [1]: https://blog.ipfs.io/35-ipfs-companion-2-2-0/

It's not hard for IRC-style chats, but it gets harder if you want to handle: * Asynchronous messaging: what if the two devices are not online at the same time, or on disjointed networks? Store the message on a server somewhere? Secure Scuttlebutt[1] relies on devices pulling encrypted data that does not belong to them. * NAT or firewall hole-punching, though it can be remediated by leveraging other nodes. Some implementations use a DHT[2], but you're often relying on other servers of some sort. * What you call contact discovery is also typically handled through a DHT of some kind. Yggdrasil-like (or hyperboria, cjdns, .onion) overlay networks are usually able to route to a public key, regardless of how it moves around on the network. * Push notifications. Either you accept the use of an external server (like the Tox client TRIfA, which has an add-on[3] that also supports UnifiedPush), or you have to rely on a separate persistent connection that will drain your battery faster, especially if there's some computation involved. The last point is why I uninstalled Briar: I had almost no contacts, and didn't want an extra battery-draining service. [1] https://scuttlebutt.nz/docs/introduction/detailed-start/#more-about-so… [2] https://blog.ipfs.tech/2022-01-20-libp2p-hole-punching/ [3] https://github.com/zoff99/tox_push_msg_app

It's "not already there" as what torrents do is not the same as what Netflix does even though both are related to media delivery. Try seeking on a torrent which only caches the local blocks and getting the same seek delay, try getting instant start and seek when 1 million people are watching a launch, try getting people to pay you the same amount to use their upload, try getting the same battery life on a torrent as a single HTTPS stream using burst intervals. As I said P2P doesn't offer as many experience consistency guarantees and has many other problems. Licensing is just one of many issues. Of course for free many people are willing to live with starting and seeking being slow or battery life being worse or having to allocate the storage up front or using their upload and so on but again the question is can it fit in the cost offset of some of the centralized infrastructure not what you could do for free. I don't have anything against torrents, quite the opposite I am quite a heavy user of BOTH streaming services and torrenting due to DRM restrictions on quality for some of my devices, but it isn't a single issue problem like you are pressing to make it be. For some internal distributed action Netflix has made IPFS into something they can use but not for end delivery https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distrib…

Time to crawl! You can use something like Webrecorder, which IPFS also uses to pin tweets: https://blog.ipfs.tech/announcing-pin-tweet-to-ipfs/ See https://webrecorder.net Example https://replayweb.page/?source=https%3A%2F%2Freplayweb.page%2Fdocs%2Fa…