Web3 Workshop

IPFS Lab

Requirements

  • 2 nodes for testing
  • GNU/Linux Debian or Arch Based Distro (x86_64)
  • curl
  • tmux

Optional a third public node in case you want to connect 2 computers from diferent LANs by using it as a circuit-relay

Instalation

  1. Install curl and coding base programs

    $ sudo apt install curl build-essential

  2. Download the Linux Kubo binary from dist.ipfs.tech.

    $ curl -O https://dist.ipfs.tech/kubo/v0.32.1/kubo_v0.32.1_linux-amd64.tar.gz

  3. Untargz the file:

    $ tar -xvzf kubo_v0.32.1_linux-amd64.tar.gz

> x kubo/LICENSE
> x kubo/LICENSE-APACHE
> x kubo/LICENSE-MIT
> x kubo/README.md
> x kubo/install.sh
> x kubo/ipfs

  4. Move into the go-ipfs folder and run the install script:

    $ cd kubo

$ sudo bash install.sh

> Moved ./ipfs to /usr/local/bin

  5. Test that IPFS has installed correctly:

    $ ipfs --version

> ipfs version 0.32.1

  6. Setup ipfs config profile First time config setup:

    $   ipfs init

> generating ED25519 keypair...done
> peer identity: 12D3KooWAc3ayFNYwMxooWU6tPi6fcAJaMqhBPx5oBMpdzkpq9ek
> initializing IPFS node at /home/nodemaster/.ipfs

Optionally use the --profile server flag to remove local discovery requests

Usage

We will use the following basic commands as shown in Table 1 for our example.

Command Description
ipfs init Initialize a new IPFS repository
ipfs id Show information about your node
ipfs daemon & Run IPFS daemon in the background
ipfs add <file> Add a file to IPFS
ipfs add -r <folder> Recursively add a folder to IPFS
ipfs pin add <hash> Pin a file or directory to keep locally
ipfs pin ls [<hash>] List all pinned files or directories
ipfs pin rm <hash> Remove the pin on a file or directory
ipfs get <hash> Retrieve a file or folder from IPFS
ipfs dht findprovs <hash> Find providers of a hash
ipfs dht provide <hash> Announce that you have a hash
`ipfs dht findpeer Check if a node is reachable
ipfs swarm addrs Show addresses connected to your node
ipfs swarm addrs listen Show addresses connected to your node
ipfs swarm connect <multiaddr> Manually connect to another node

Table 1. Shows a simplied based CheatSheet for the IPFS.

Linking nodes

Verify if you can pin the node A from B and the oposite way

$ ipfs ping /ipfs/12D3KooWAc3ayFNYwMxooWU6tPi6fcAJaMqhBPx5oBMpdzkpq9ek
> .PING 12D3KooWAc3ayFNYwMxooWU6tPi6fcAJaMqhBPx5oBMpdzkpq9ek.
> .Pong received: time=0.58 ms
> .Pong received: time=0.18 ms
> .Pong received: time=0.68 ms
> .Pong received: time=0.75 ms
> .^C
> .Average latency: 0.55ms

Optionally use the --profile server flag to remove local discovery requests

If this doesn't work we can try to manually connect to the node by using ipfs swarm connect:

$ ipfs swarm connect /ipfs/12D3KooWAc3ayFNYwMxooWU6tPi6fcAJaMqhBPx5oBMpdzkpq9ek
> connect 12D3KooWAc3ayFNYwMxooWU6tPi6fcAJaMqhBPx5oBMpdzkpq9ek success

and you can verify the node is connected to your swarm

$ ipfs swarm addrs | grep 12D3KooWAVkXNR9Ym2rw2vrdnjW845AhtkjFN6BT9duLwN9ehPFo
> 12D3KooWAVkXNR9Ym2rw2vrdnjW845AhtkjFN6BT9duLwN9ehPFo (8)

Pinning (decentralizing files)

lets asume we have a file named axolotl-thoth.jpg which we want to share from node A to node B, first we add the file with the following command to add and wrap it into the IPFS local node:

 $ ipfs add --cid-version 1 -r -w  axolotl-thoth.jpg
 > added bafkreihiuou4poc45gqxg7fsdsd733cmmottsppr65ldqmbaxapx2vyfgm axolotl-thoth.jpg
 > added bafybeidry6pgmpjebqtqkq6g2lu3fu2ym64wfot5jvyuncyizvel3p2n5m
 > 86.23 KiB / 86.23 KiB [========================================] 100.00%

we can verify the file is alleady pinned (added to the ipfs local node):

 $ ipfs pin ls
 > QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn recursive
 > bafkreihiuou4poc45gqxg7fsdsd733cmmottsppr65ldqmbaxapx2vyfgm indirect
 > bafybeidry6pgmpjebqtqkq6g2lu3fu2ym64wfot5jvyuncyizvel3p2n5m recursive

if you have many files you can | grep <CID> after the verification command.

Now on node B and once verified that bouth nodes are linked, we proceed with pinning the hash which will mean we have a copy in our IPFS node as tracked and shared (decentralized).

 $ ipfs pin ls
 > QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn recursive
 $ ipfs pin add bafkreihiuou4poc45gqxg7fsdsd733cmmottsppr65ldqmbaxapx2vyfgm
 > pinned bafkreihiuou4poc45gqxg7fsdsd733cmmottsppr65ldqmbaxapx2vyfgm recursively
 $ ipfs pin ls
 > bafkreihiuou4poc45gqxg7fsdsd733cmmottsppr65ldqmbaxapx2vyfgm recursive
 > QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn recursive

as shown we can se the file is already pinned and added to our nodes ipfs and we can get it into our file system by using the ipfs get <CID>

 $ ipfs get bafkreihiuou4poc45gqxg7fsdsd733cmmottsppr65ldqmbaxapx2vyfgm -o axolotl-thoth.jpg
 > Saving file(s) to axolotl-thoth.jpg
 >  86.23 KiB / 86.23 KiB [=====================================] 100.00% 0s
 $ ls
 > axolotl-thoth.jpg  kubo_v0.32.1_linux-amd64.tar.gz      -O
 > kubo               kubo_v0.32.1_linux-amd64.tar.gz.sig
 $ sha256sum axolotl-thoth.jpg
 > e8a3a9c7b85ce9a1737cb21c87fdec4c63a7393df1f756383020b81f7d570533  axolotl-thoth.jpg

here we also show the file and verify the sha256 checksum

Static Site Decentralization

Requirments

  • IPFS setup (example above)
  • curl
  • unzip
  • deno
  • lume

Instalation

First we go to the deno website and install it as it specifies for our OS:

 $ curl -fsSL https://deno.land/install.sh | sh
 > ######################################################################## 100.0%
 > Archive:  /home/nodemaster/.deno/bin/deno.zip
 > inflating: /home/nodemaster/.deno/bin/deno
 > Deno was installed successfully to /home/nodemaster/.deno/bin/deno
 > info: backing '/home/nodemaster/.profile' up to '/home/nodemaster/.deno/.shellRcBackups/.profile.bak'
 > info: backing '/home/nodemaster/.bashrc' up to '/home/nodemaster/.deno/.shellRcBackups/.bashrc.bak'
 >
 > Deno was added to the PATH.
 > You may need to restart your shell for it to become available.
 >
 > Run '/home/nodemaster/.deno/bin/deno --help' to get started
 >
 > Stuck? Join our Discord https://discord.gg/deno

After restarting the shell we select a theme from lume land themes for the current example we will use the simple blog theme:

 $ mkdir my-blog && cd my-blog
 $ deno run -A https://lume.land/init.ts --theme=simple-blog

 > Welcome to Lume v2.5.1!
 >
 > File saved: _data.yml
 > File saved: 404.md
 > File saved: favicon.png
 > File saved: posts/instructions.md
 >
 > 🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥
 >
 >   BENVIDO - WELCOME! 🎉🎉🎉
 >
 >   Lume has been configured successfully!
 >   Theme installed: Simple Blog
 >
 > 🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥
 >
 > Quick start:
 >
 >   deno task serve to start a local server
 >   deno task cms to start the CMS
 >
 > See https://lume.land for online documentation
 > See https://discord.gg/YbTmpACHWB to propose new ideas and get help at Discord
 > See https://github.com/lumeland/lume to view the source code and report issues
 > See https://github.com/lumeland/theme-simple-blog to view the theme source code and report issues
 > See https://opencollective.com/lume to support Lume development

Setup and Testing

The next steps is that we require to setup relative paths for our static website this is require to work properly on any IPFS gateway. First edit the file _config.ts which looks like the following in the decentralized science website:

1 import lume from "lume/mod.ts";
2 import relativeUrls from "lume/plugins/relative_urls.ts";
3 import wiki from "wiki/mod.ts";
4
5 const site = lume({
6     location: new URL("https://decentralizedscience.org"),
7 });
8
9 site.copy("img");
10
11 site.use(relativeUrls());
12 site.use(wiki());
13
14 export default site;

Notice that the lines to make the paths relative are 2 and 11.

While you work on your website you can live watch the changes but executing the command deno task serve:

 $ deno task serve
 > Task serve deno task lume -s
 > Task lume echo "import 'lume/cli.ts'" | deno run -A - "-s"
 > Loading config file file:///home/nodemaster/_config.ts
 > 🔥 /kubo/readme/ <- /kubo/README.md
 > 🔥 /404.html <- /404.md
 > 🔥 /posts/instructions/ <- /posts/instructions.md
 > 🔥 / <- /index.vto
 > 🔥 /styles.css <- /styles.css
 > 🔥 /archive/instructions/ <- /archive_result[0].page.js
 > 🔥 /author/oscar-otero/ <- /archive_result[1].page.js
 > 🔥 /archive/ <- /archive[0].page.js
 > 🔥 /pagefind/pagefind.en_facec4f167.pf_meta <- (generated)
 > 🔥 /pagefind/wasm.en.pagefind <- (generated)
 > 🔥 /pagefind/fragment/en_6e896e2.pf_fragment <- (generated)
 > 🔥 /pagefind/index/en_8cead82.pf_index <- (generated)
 > 🔥 /pagefind/filter/en_5b2039e.pf_filter <- (generated)
 > 🔥 /pagefind/pagefind.js <- (generated)
 > 🔥 /pagefind/pagefind-highlight.js <- (generated)
 > 🔥 /pagefind/pagefind-ui.css <- (generated)
 > 🔥 /pagefind/pagefind-modular-ui.js <- (generated)
 > 🔥 /pagefind/pagefind-modular-ui.css <- (generated)
 > 🔥 /pagefind/pagefind-entry.json <- (generated)
 > 🔥 /sitemap.xml <- (generated)
 > 🔥 /robots.txt <- (generated)
 > 🔥 /feed.xml <- (generated)
 > 🔥 /feed.json <- (generated)
 > 🔥 /pagefind/pagefind-ui.js <- (generated)
 > 🔥 /pagefind/wasm.unknown.pagefind <- (generated)
 > 🔥 /favicon.png <- /favicon.png
 > 🔥 /js/main.js <- https://deno.land/x/lume_theme_simple_blog@v0.15.10/src/js/main.js
 > 🔥 /js/comments.js <- https://cdn.jsdelivr.net/npm/@oom/mastodon-comments@0.3.2/src/comments.js
 > 🍾 Site built into ./_site
 >   28 files generated in 5.92 seconds
 >   Server started at:
 >   http://localhost:3000/ (local)
 >   http://10.97.239.92:3000/ (network)

Decentralization

The process requires to build the project where we will get an output directory that contains the our static site named _site, and once we have it we will do an ipfs pin add with some parameter for a better compatibiliy as shown:

 $ deno task build

 > Task build deno task lume
 > Task lume echo "import 'lume/cli.ts'" | deno run -A -
 > Loading config file file:///home/nodemaster/my-blog/_config.ts
 > 🔥 /404.html <- /404.md
 > 🔥 /posts/instructions/ <- /posts/instructions.md
 > 🔥 / <- /index.vto
 > 🔥 /styles.css <- /styles.css
 > 🔥 /archive/instructions/ <- /archive_result[0].page.js
 > 🔥 /author/oscar-otero/ <- /archive_result[1].page.js
 > 🔥 /archive/ <- /archive[0].page.js
 > 🔥 /pagefind/pagefind.en_facec4f167.pf_meta <- (generated)
 > 🔥 /pagefind/wasm.en.pagefind <- (generated)
 > 🔥 /pagefind/fragment/en_6e896e2.pf_fragment <- (generated)
 > 🔥 /pagefind/index/en_8cead82.pf_index <- (generated)
 > 🔥 /pagefind/filter/en_5b2039e.pf_filter <- (generated)
 > 🔥 /pagefind/pagefind.js <- (generated)
 > 🔥 /pagefind/pagefind-highlight.js <- (generated)
 > 🔥 /pagefind/pagefind-ui.css <- (generated)
 > 🔥 /pagefind/pagefind-modular-ui.js <- (generated)
 > 🔥 /pagefind/pagefind-modular-ui.css <- (generated)
 > 🔥 /pagefind/pagefind-entry.json <- (generated)
 > 🔥 /sitemap.xml <- (generated)
 > 🔥 /robots.txt <- (generated)
 > 🔥 /feed.xml <- (generated)
 > 🔥 /feed.json <- (generated)
 > 🔥 /pagefind/pagefind-ui.js <- (generated)
 > 🔥 /pagefind/wasm.unknown.pagefind <- (generated)
 > 🔥 /favicon.png <- /favicon.png
 > 🔥 /js/comments.js <- https://cdn.jsdelivr.net/npm/@oom/mastodon-comments@0.3.2/src/comments.js
 > 🔥 /js/main.js <- https://deno.land/x/lume_theme_simple_blog@v0.15.10/src/js/main.js
 > 🍾 Site built into ./_site
 >   27 files generated in 0.12 seconds

Now we ipfs pin add on our local node and save the output to a file so that we remember which are the assets hashes, specifically the _site which is the one of our highest interest because it links the others.

 $ ipfs add --cid-version 1 -r -w _site/ > myBlogCID.txt

 > 368.93 KiB / 368.93 KiB
 > [================================================================================================================]
 > 100.00%

 $ tail myBlogCID.txt

 > added bafybeiammkufyfabclfpkhiyuqvi33daoy63eghnm6ek5hpbt25n6hfkle _site/author
 > added bafybeiheyhjqooii5rz6fycli3e5ut2dxn5aoei7pkgeeul4h33m53ne2y _site/js
 > added bafybeibb4zvebkd34fggnigvay2nequ64isc5wfwlx27xzlqfvunf4e5ha _site/pagefind/filter
 > added bafybeiaivvexmb7hnw2nl2s36z5u5rb36vdfmg3xolle22bctkyewlipca _site/pagefind/fragment
 > added bafybeiacg6235tzuvg7rkpjmq7ii7ry56pl2fx6thczxit624vqbsrgrna _site/pagefind/index
 > added bafybeifcqk7srojaphi7hiz4rswsg564ax63hsjgmlhlnb66zpqimy5snq _site/pagefind
 > added bafybeiglzvnozx2w42gxjm3f3dhf7qselu5lia2tt267glsi42qibculfm _site/posts/instructions
 > added bafybeighw2tyct7pe5dezbooxywwkni2l6kqu5esaj3tnu7exgdie7skae _site/posts
 > added bafybeicvxfaxdmtpormu4yn33pn3apyqu6wbn7nksnn26bgoiuadnsa4je _site
 > added bafybeigloh3jxjxxzcmypgga6hqcnu3w4pkoyqlsydshpulvpf22banise

You can verify in your local node that the site is already being served. For this example url notice we write the CID of the _site after /ipfs/

http://127.0.0.1:8080/ipfs/bafybeicvxfaxdmtpormu4yn33pn3apyqu6wbn7nksnn26bgoiuadnsa4je/

And finally now you can ipfs pin add CID hash bafybeicvxfaxdmtpormu4yn33pn3apyqu6wbn7nksnn26bgoiuadnsa4je in other nodes or online pinning services which probably also serve the site through their gateway.

ToDo

  • [x] Setup incus in both laptops with debian stable
  • [x] Install requirements
  • [x] Install IPFS
  • [x] IPFS Usage and tests (share a file)
  • [ ] Setup lume land blog
  • [ ] Decentralize the blog
  • [ ] pinning services
  • [ ] [Blockchain integrationAlloy
  • [ ] integrate dapp i ndexer
  • [ ] setup ddns and domain names
  • [ ] alt setup with nginx (future)

References

  1. dist.ipfs.tech.
  2. circuit-relay