Usually the issue of media storage (photos, videos, etc) is brought in as an Issue. For now I’ll skirt the “legal ramifications” including copying media and privacy, as those are an ever changing landscape of legal wanking that wankers can speak of much better than one can (and evil wanking still needs to be fought against).
One idea I’ve seen floated around is to have some sort of cooperative CDN for instances. Let’s say four or five relatively kindred instances, make a commitment to last and pool their resources to maintain a joint CDN from from which they’ll get their “media federation” from. This would reduce costs and issues a lot, since by the very nature of the fediverse, if everyone builds their own caches most of those caches are going to be hosting most of the same content. Basically: deduplication, but the poor man’s version.
Another alternative is to just ditch storage of videos and images. Just take links to Elsewhere and let Elsewhere handle it.
I mean, I’d say that all instances copying media by default, to be stored forever, is kind of unnecessary. (And as far as I’m aware Mastodon is the only one configured like this by default anyway)
The largest instances? Sure. I’d say they have an obligation to not DoS smaller instances by simply hotlinking or proxying without any kind of cache. But smaller ones can get away with short lived middleware-level caches, and single user ones can often get away with hotlinking (oh boo hoo your firewalled IPv4 behind enough CGNATs to block any incoming connections got exposed)
One idea I’ve seen floated around is to have some sort of cooperative CDN for instances. Let’s say four or five relatively kindred instances, make a commitment to last and pool their resources to maintain a joint CDN from from which they’ll get their “media federation” from. This would reduce costs and issues a lot, since by the very nature of the fediverse, if everyone builds their own caches most of those caches are going to be hosting most of the same content. Basically: deduplication, but the poor man’s version.
Interesting! Didn’t know that was what it was for. I always thought it was merely a storage backend.
Any metrics on how many instances are using it and how much deduplication is it doing? EDIT: I see the numbers on their page, I was wondering more about people or instances using their own copy of it, since it’s open source.
Usually the issue of media storage (photos, videos, etc) is brought in as an Issue. For now I’ll skirt the “legal ramifications” including copying media and privacy, as those are an ever changing landscape of legal wanking that wankers can speak of much better than one can (and evil wanking still needs to be fought against).
One idea I’ve seen floated around is to have some sort of cooperative CDN for instances. Let’s say four or five relatively kindred instances, make a commitment to last and pool their resources to maintain a joint CDN from from which they’ll get their “media federation” from. This would reduce costs and issues a lot, since by the very nature of the fediverse, if everyone builds their own caches most of those caches are going to be hosting most of the same content. Basically: deduplication, but the poor man’s version.
Another alternative is to just ditch storage of videos and images. Just take links to Elsewhere and let Elsewhere handle it.
I mean, I’d say that all instances copying media by default, to be stored forever, is kind of unnecessary. (And as far as I’m aware Mastodon is the only one configured like this by default anyway)
The largest instances? Sure. I’d say they have an obligation to not DoS smaller instances by simply hotlinking or proxying without any kind of cache. But smaller ones can get away with short lived middleware-level caches, and single user ones can often get away with hotlinking (oh boo hoo your firewalled IPv4 behind enough CGNATs to block any incoming connections got exposed)
https://jortage.com/ already exists, and the code behind is open.
Interesting! Didn’t know that was what it was for. I always thought it was merely a storage backend.
Any metrics on how many instances are using it and how much deduplication is it doing? EDIT: I see the numbers on their page, I was wondering more about people or instances using their own copy of it, since it’s open source.