HowIpfsActuallyWorks

Roadmap: Explain EVERYTHING.

  • Start with how nodes find and talk to each other, and how information is propagated between them (libp2p?)
  • Then talk about how addressing works and all the million of various bits that go together for it (concentrate on a single block for now)
  • Then talk about how IPLD works and connects together multiple blocks
  • Then talk about how higher-level things get built atop IPLD, like unixfs

Okay now we have to talk about how the software works.

  • go-ipfs
  • js-ipfs

Now talk about more advanced features that are in development but not perfect. What it is, how they work currently, how they should work in the end, what problems they solve and how they should be composed together into bigger solutions:

  • Pubsub
  • IPNS
  • IPRS
  • Mutable-ish databases (CRDT’s)

Last updated: Nov 6 2017

Verified accurate by IPFS devs as of: ???

License for this document: Public domain

IPNS

How it works: A node can publish a value, using the node’s crypto key as the key. (Nodes can have multiple keys; ipfs key gen and ipfs name publish -k.) Then anyone can say “Find me the node with this key and give me what it’s saying”, and the IPFS network does that. If this node’s value is, say, an IPFS content ID, then that means any node can advertise an IPFS object for you to receive. Then you can stick a SRV record with the node ID for a site in that site’s DNS record, but I forget how exactly that works. Still, that closes the loop from ipfs.io -> node ID running ipfs.io -> content that node wishes to give you when you go to ipfs.io.

Glossary

References

FAQ

Is IPFS permanent?

Not more permanent than anything else in the world. The address for an IPFS object will never change, but someone has to actually be providing that object on the network for it to be retrievable.

How private is IPFS?

Not very, as far as I know. Anyone who knows a content ID for an IPFS object can ask for it to be provided. Nodes have to tell other nodes what objects they have, and don’t try to obfuscate origin and transmission of data. IPFS is not designed to be Tor or Freenet.

If you want to use IPFS to transmit sensitive information, encrypt the information beforehand.

You CAN use IPFS to set up private networks that don’t talk to nodes outside of that network; that’s not terribly difficult.

How does IPNS know which node is publishing a record?

Each node has a crypto key

Can I publish the same IPNS record from multiple nodes?

Yes but it’s a pain. Basically you just shove the same crypto key into multiple running nodes and set them all to broadcast the same thing. ipfs-cluster is a tool to help with this sort of thing, eventually.

What happens if I do that but make each node broadcast something different?

Nothing good.

How do I reduce CPU/mem usage?

05:44 < Magik6k> If you mean 0.4.11 - try 0.4.12-rc1 - https://dist.ipfs.io/go-ipfs/v0.4.12-rc1, it improves some things, run daemon with `--routing=dhtclient` flag, and setup connection limiter to something like 20/40/1minute (conn limiter docs: https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#connmgr)
06:05 <@Kubuxu> Magik6k: 20/40 is way too low
06:05 <@Kubuxu> Magik6k: it won't populate the DHT and everything will work badly
06:15 < Magik6k> Kubuxu: it should work fine for dhtclient with large GracePeriod
06:16 <@Kubuxu> no it won't as kbuckets won't get filled and DHT requests will take ages
06:16 <@Kubuxu> configs like that can be experimented with but don't recommend them to someone asking to fix other problems
06:17 < Magik6k> That is what was suggested by whyrusleeping in https://github.com/ipfs/go-ipfs/pull/4154#issuecomment-340870408
06:19 <@Kubuxu> it should be tested first

How do I search for something in IPFS?

Use some external service

How do I search for something in IPRS?

Use some external service (once IPRS exists)

What is a dweb:/ URL/URI and how does it relate to /ipfs/Qm… ?

Dunno

How does pubsub work?

  • https://github.com/libp2p/research-pubsub
  • https://github.com/libp2p/js-libp2p-floodsub/
  • https://github.com/libp2p/go-floodsub/
  • https://blog.ipfs.io/29-js-ipfs-pubsub/
  • https://blog.ipfs.io/30-js-ipfs-crdts.md

How do CRDT’s work?

CRDT’s just mean that you structure your data so that it doesn’t matter what order you get updates in, or whether you get duplicates of them.

Rating systems such as Reddit are a great example of a CRDT. Each vote is (postid, username, +/-). To figure out the score of a post, you just collect all the votes, then tally them by post ID and username, discarding multiple votes by the same person to the same post. It doesn’t matter how many duplicates you have, and it doesn’t matter what order you get the votes in; as long as everyone involved gets at least the full set of votes, everyone will calculate the same score for the post.