One file system, one CID, any chain

How onchfs stores files on-chain, why the same file got different CIDs on Tezos and Ethereum, and moving it all to Tezos X.

The Rune Team ·

How onchfs stores a file: chunks in a content store, files and directories as pointers, one CID for the whole tree

onchfs is a file system that lives on a blockchain. It was built by the people behind fxhash so that generative art could be stored on-chain, but it works for any files: a web page, an image, a whole website. This blog is stored on it.

I have spent the last couple of days moving my onchfs setup to Tezos X. Here is what onchfs is, what was wrong with it, and what changed.

How onchfs works

Files are split into chunks of 16 KB. Each chunk is written once to a content store contract and addressed by its hash. A second contract, the file system, keeps track of files and directories: a file is a list of chunk pointers plus some metadata (content type, encoding), and a directory maps names to files and other directories.

Everything gets a content identifier, a CID, computed from what it contains. Change one byte and you get a different CID. The CID of a directory covers everything below it, so a single CID pins down an entire website, exactly as it was published.

To read it, you need a gateway: a small web server that takes onchfs://<cid>/index.html, reads the chunks from the chain and serves them as a normal web page. Mine runs at onchfs.io.

The problem: one file, two CIDs

onchfs exists for Tezos and for Ethereum-style chains, and the two contracts disagree about how to compute the CID of a file.

The Tezos contract hashes the contents of the file: when a file is created, it reads every chunk back from the content store and hashes the bytes. The Ethereum contract hashes the chunk pointers instead, which are themselves hashes of the chunks. Both are content addressed, but the results differ, so the same file got one CID on Tezos and another on Ethereum.

That breaks one of the nicest properties of content addressing: a CID should identify the content, no matter where it is stored.

The fix

Before: Tezos hashed the contents and Ethereum the pointers, so one file had two CIDs. After: every chain hashes the pointers, one CID everywhere

I patched the create_file entrypoint of the Tezos contract to use the same formula as the Ethereum contract. Instead of reading every chunk back, it checks that the chunks exist and hashes their pointers. Directory CIDs were already identical, so this one change is enough. As a bonus it is cheaper, since the contract no longer reads every chunk back when a file is created.

Storage layout, views and the other entrypoints are unchanged, so the new contract can use the existing content store and reuse every chunk that is already there. The contract and its tests are on GitHub: asbjornenge/onchfs-contracts. The tests check that the CIDs match what onchfs-js computes, and an independent implementation of the Ethereum formulas.

One caveat: these CIDs depend on how a file is chunked, so every uploader on every chain has to use the same chunk size. The onchfs-js default (16384 bytes) is what everyone uses.

Tezos X

Tezos X is a new layer on top of Tezos that runs Michelson contracts, the same contracts as Tezos itself, with much cheaper storage. On Tezos L1 storage costs 250 mutez per byte. On Tezos X it costs 1. Operation fees still depend on the size of the data, so in practice inscribing data on Tezos X costs around 5 ꜩ per megabyte all in, still a fraction of L1.

I deployed the patched contract both to Tezos L1 and to Tezos X, and migrated my existing files from the old contract. Migrating 31 files and directories cost about 1.1 ꜩ on L1 and about 0.19 ꜩ on Tezos X, contract deployments and data included. Both gave exactly the same CIDs, which is the whole point.

contract
Tezos L1 KT1M3JKHiEzo8iLve8AWtRVHd1vW2jiCjXd1
Tezos X KT1JYZ2o6keUiTChNKUQnWDcMkjzES3djjRY

The gateways know about both: onchfs.io reads from L1 (falling back to the old contract for older files), and tezosx.onchfs.io reads from Tezos X. The onchfs command line tool learned about Tezos X too, as of version 1.3.0.

Why bother

I have been thinking about my own mortality lately, and what might happen to my content when I die. My old blog runs on a server I pay for. When I stop paying, or stop being around to pay, it goes dark. Domains expire, hosting accounts close, and years of writing quietly disappear. Most of what we put online lives exactly as long as someone keeps paying for it.

I wanted a place to write that does not depend on me keeping anything alive. This post is published with Rune, a blogging platform built on onchfs. Every post becomes a small onchfs directory on Tezos X, and the blog index links them together by CID. If rune.pub disappears, and if I do, the posts are still on-chain, and any onchfs gateway can serve them.

It is a strange feeling to press publish on something that can never be changed. It makes you read the post one more time.

#onchfs #tezos #tezos-x