Hello world it's Siraj! And it's time to upgrade the entire internet. i'm going
To explain how the interplanetary file system or IPFS works. We'll also use it to build a simple video streaming web app
Onwards, towards the permanent web. the web is the backbone of our
civilization so much of our economy our mission critical systems our most basic
communication channels depend on it to function the orchestra of apps that give us such amazing capabilities today are only
Possible because we built the necessary infrastructure to support them what we call the web is just a collection of
protocols these are sets of rules ideas on how to best
Transmit data that have won over the hearts of developers and then made standard by browser adoption and tool
dependency anyone can write a protocol and the existing ones were written by people
Aiming, to solve small problems for themselves and in the process ended up being adopted by
everybody else because they worked really well there's one protocol in particular however that is in need of a serious upgrade
HTTP developed by Tim Berners-Lee
(Music)
we use HTTP to access most of the web it delivers virtually all file types and is considered a
Client-Server protocol a client, you on your laptop, with the browser for example can send a request message to a server say
Amazon.com and the server will then return a response message like a home page it worked well enough but we're starting
To see some real problems with http the first problem is
bandwidth more and more of the world is coming online fast and that means more people are demanding more data
Web sites crash all the time because too many people are trying to access them at the same time
imagine this scenario let's say i'm sitting in a classroom
with
Some other students we're studying sex robots i want to share a useful picture with them so i upload it to facebook
Then give them all the link so they can fetch the image
assuming there are say eight links and the total bandwidth required for my upload is 8 megabytes
Then when 30 students request that image now that's
240 megabytes of bandwidth usage
If i uploaded a video and that took 200 meg's of bandwidth when all the other people watch it it'll take 48
gigs of bandwidth this is a horribly inefficient routing scheme since they are so
Physically close to me and it's getting
To the point where providing this amount of bandwidth is no longer profitable
For companies if we look at how much bandwidth has improved over the past few years
It's not very significant compare that to how fast our storage space is decreasing in cost and it's clear that we are fast
Saturating these pipes so the web will start to feel slower which leads to the second problem latency
unfortunately we can't get around the speed of light it's a constant could we just do an ic o---- know what
So the only way to make data transfer faster is to move the data closer to you which is why some cloud services offer storage
Locations by region still if i need a file and it just so happens to be on a device one meter away from me
Having my device make a request through the backbone of the web to some nearby city
isn't nearly as fast as just pulling it from that device directly the third problem is resiliency
of, connectivity if your ISP has an outage and your towns connection to the global internet goes down you're out of luck there's nothing you
can do about that in terms of data permanence if
wikipedia was hacked and all its data
Deleted we'd lose that knowledge forever it'd be akin to book burnings when only a few copies of certain books were available
But it doesn't even have to be that drastic an event does this error look familiar to you
you've likely seen a 404 before which signifies that some content has been deleted or moved
http links get broken all the time
It's like mini book burning all the time if data is it manually copied by someone that data is gone forever
the last problem is
Centralization because data has become so centralized a few services own most of our data and we have no?
say in how that data is used. this is
unimaginable power these services can manipulate large swaths of the populace with a few clicks and
access to them can be easily blocked by governments and other
oppressive forces like what happened during the arab spring or during the pro-democracy
protests in hong kong in 2014 we have got to upgrade the internet it's too important not to
IPFS was designed to help fix these problems and help us move towards the permanent web a web where links never die and no?
entity controls your data while the
http web is ip addressed the
ipfs web is content addressed when
requesting data from a content address you'll receive it faster since it'll route from whoever owns a copy of that content address
closest to you this helps make bandwidth usage much more efficient when you upload a file to
ipfs you get back it's immutable hash each file in the network is identified by its content using this hash
The hash is cryptographically
guaranteed to represent only that file anyone can provide storage for ipfs data these storage providers are
incentivized with a crypto-token and data is replicated and chunked
across all of them by the network to maintain data permanence
To achieve this IPFS combines several great ideas from computer science in the past few years
first to store data it uses a distributed hash table similar to one proposed at mit called cord a
Dht is a dictionary like interface to data that is stored on nodes that are
distributed across the network the node that gets to store a
Particular key is found by hashing that key so in effect the hash table buckets are now independent nodes in a network
data is exchanged between these nodes using a mechanism called bit swamp inspired by the popular bittorrent
protocol that de facto a way to transfer large data sets across the web a user can discover nearby
Peers to share data with then when requesting a file they can download many small bits of it for many different peers all at once
this
format compensates for bottleneck points which makes it much faster than downloading a large file from a single source to give structure to the
DHT and let users find the data they need when they need it IPFS uses a data structure called a Merkle DAG
inspired by the git protocol. Git uses a Merkle DAG to enable distributed version control for source code and
IPFS uses it to give structure to the entire network. It's simple and flexible we can conceptualize it as a series of nodes
connect it together in the form of a directed acyclic
graph it can look like a singly linked list or a tree so whenever data is added to the DHT the system generates a
public/private key pair and the user gets both alright let's get our hands dirty with ipfs
we'll first install the IPFS client on our machines there's a tarball available for mac and linux and an exe for installation for windows
Once i've downloaded it i can untar it and move the IPFS binary to somewhere in my
executables so i can just run it directly from terminal like so
it's installed we can now initialize IPFS from terminal and it will create our node
generate an identity for us and allow us to read files on the network if i read an address i can see the text i
Just, retrieved in terminal to go online we'll run the daemon in another
we can now discover our peers in the network by their peer identities
we can read files directly from these peers and there's even a fancy web console for us to be able to see
where all the peers are in the world
We can use this ui to manage our instance let's say we want to upload a video that we have locally
We can use the ip fs add command to add that video to the network and use the resulting hash we get as its identity
Then in terminal we can view it directly or we can watch it via a local gateway in the browser
let's say we have a simple static web app that lets users
upload video and view it it's just a simple html web page with some assets we can add the whole directory
Containing our web app to ipfs directly the very last hash next to the folder name is the one we want
it's the highest level hash of the Merkle DAG we've just created
we could create a simple dns text record and then view our site locally or at the online
Http to ipfs gateway at ipf seo it's called
interplanetary because if and when we do become an interplanetary species
It's much more efficient to use a web that finds the data you need that's
closest to you not light-years away the three closing points are the http web is a few decades old now and has become slow
impermanent and centralized
ipfs is a newer protocol that aims to create a permanent web where links are content not ip addressed and it uses a
Distributed hash table to store data a Merkle DAG to give it structure and a Bittorrent like data exchange mechanism to maximize
bandwidth last week's coding challenge winner is two chars tony open my needed to be able to compute
Operations on a tensor of arbitrary type so tashard demonstrated how to do this
programmatically
such an epic pr great job and the runner-up is rohan sawat who implemented some much-needed and awesome styling updates to open
lines documentation this week's coding challenge is to create your own simple web app using ip fs
post your github link in the comments because i'm announcing the top two entries
next friday. I really hope you like this video and if you did hit the subscribe button and all your dreams will come true
For now i've got, to make a crypto rap so thanks for watching