Drafting a [proposal][1] to add API support in #Fedify for the [ActivityPub Media Upload extension][2], the SocialCG-incubated #C2S companion that lets clients upload media via a dedicated `endpoints
-
@hongminhee like I mentioned in a thread where Evan and Reiver were talking about this, I think having a mediaUpload that has a very similar behaviour to an inbox/outbox, but with one **small** changed detail is a bad API.
I would prefer there's either a two step process: upload media first, use resulting token in an object create, or use the outbox with for the binary data upload directly... The first one seems saner to me.
@mariusor @hongminhee you don't even need a "token" concept specifically, the media upload could internally as:Create an as:Document (but not broadcast it) you can then refer to or inline as needed -
@mariusor@metalhead.club I agree. We probably need a FEP for this?
@hongminhee I haven't reached that point in GoActivityPub development to really think about it.
Currently for binary uploads I use world's hackiest hack
️, where I encode the media as a base64 media URI and set it as the Content of a Image/Audio/Video object that I send through client to server. 
So far this has worked well enough between the clients and servers *I* developed, but I can't really ask anyone else to support that.
-
@mariusor @hongminhee you don't even need a "token" concept specifically, the media upload could internally as:Create an as:Document (but not broadcast it) you can then refer to or inline as needed
@kopper token can be anything: an URL at which the media can now be found, the "template" json document Evan was talking about with the URL filled in, etc... Whatever would make the most sense I guess.
-
@kopper token can be anything: an URL at which the media can now be found, the "template" json document Evan was talking about with the URL filled in, etc... Whatever would make the most sense I guess.
@mariusor @hongminhee URLs to AS objects makes the most sense to me, as you can Just (assuming valid JSON-LD handling across the ecosystem) paste the link into whatever property to federate it immediately without needing to load more data
and the media uploader can do things like introspect width/height/blurhash out of the uploaded file into the document itself, instead of moving that logic to the client (although I imagine it'd be a good idea to allow the client to as:Update the resulting documents to add more metadata if it has to) -
@mariusor @hongminhee URLs to AS objects makes the most sense to me, as you can Just (assuming valid JSON-LD handling across the ecosystem) paste the link into whatever property to federate it immediately without needing to load more data
and the media uploader can do things like introspect width/height/blurhash out of the uploaded file into the document itself, instead of moving that logic to the client (although I imagine it'd be a good idea to allow the client to as:Update the resulting documents to add more metadata if it has to)@hongminhee @mariusor although "URL to media blob directly" is simpler to implement and has more flexibility in cases where you're gonna be using that media outside well-formed AS (say: inline media in HTML) -
@kopper token can be anything: an URL at which the media can now be found, the "template" json document Evan was talking about with the URL filled in, etc... Whatever would make the most sense I guess.
@kopper now that I had to think about it for 10m, I really like the idea of getting back a templated object that can further be amended by the client and sent in a Create.
*By templated I mean an "incomplete" object which has filled only the properties that make sense: ID, URL, MediaType, Type, AttributedTo,etc...
-
@hongminhee @mariusor although "URL to media blob directly" is simpler to implement and has more flexibility in cases where you're gonna be using that media outside well-formed AS (say: inline media in HTML)
@kopper for me ActivityPub servers are just ActivityPub servers, there's no using that media outside ActivityPub. There is only one GOD.
-
@kopper now that I had to think about it for 10m, I really like the idea of getting back a templated object that can further be amended by the client and sent in a Create.
*By templated I mean an "incomplete" object which has filled only the properties that make sense: ID, URL, MediaType, Type, AttributedTo,etc...
@mariusor @hongminhee this makes a lot of sense assuming the client would want to edit the object often, but I'm a little worried about the case where you upload a file and not as:Create the object afterwards (accidentally, obviously. e.g. network loss). feels a little fragile although I can see how you may still want to deal with that case even if the server Created the object itself (but it never got referenced anywhere) -
@kopper for me ActivityPub servers are just ActivityPub servers, there's no using that media outside ActivityPub. There is only one GOD.
@mariusor @hongminhee as:content is HTML and a lot of servers (including mine) do let you attach inline media in there -
@hongminhee @mariusor although "URL to media blob directly" is simpler to implement and has more flexibility in cases where you're gonna be using that media outside well-formed AS (say: inline media in HTML)
@kopper @hongminhee i think you could upload the binary media to one endpoint, get an https: url for its Location header, and turn around and Create an Image or whatever with that Location header as the url.href of your new Image
although now that this topic comes up: there is a lack of clarity around "image" vs "image representation", since Image assumes "image" and not "image representation". as:url is nominally that "representation" and i wish it was better defined that this was the case...
-
@hongminhee I haven't reached that point in GoActivityPub development to really think about it.
Currently for binary uploads I use world's hackiest hack
️, where I encode the media as a base64 media URI and set it as the Content of a Image/Audio/Video object that I send through client to server. So far this has worked well enough between the clients and servers *I* developed, but I can't really ask anyone else to support that.
I’m not developing against ap specifically but do keep tabs
I think the two part process makes sense and that’s how I’m approaching my ap adjacent media explorations
The core is the idea of an array of posts, where each post could be a different media type, and so I upload media first and once I have what we can call a permalink, I put that in the actual post to create the reference from the top level post to embed the media
My biggest annoyance as someone that really enjoys media production, is how bad media is across the fediverse.
I’m a firm believer that how bad these tools are for people like me is why this place doesn’t grow beyond people that can stomach primarily text based relationships.
-
I’m not developing against ap specifically but do keep tabs
I think the two part process makes sense and that’s how I’m approaching my ap adjacent media explorations
The core is the idea of an array of posts, where each post could be a different media type, and so I upload media first and once I have what we can call a permalink, I put that in the actual post to create the reference from the top level post to embed the media
My biggest annoyance as someone that really enjoys media production, is how bad media is across the fediverse.
I’m a firm believer that how bad these tools are for people like me is why this place doesn’t grow beyond people that can stomach primarily text based relationships.
@mariusor @hongminhee fwiw my entire architecture is a c2s so I’m very excited to be compatible with a flourishing c2s ap future
A single central personal server makes so much sense to me and I’ve really enjoyed what I’m doing even if I’m only a personal computer and a party of 1
-
I’m not developing against ap specifically but do keep tabs
I think the two part process makes sense and that’s how I’m approaching my ap adjacent media explorations
The core is the idea of an array of posts, where each post could be a different media type, and so I upload media first and once I have what we can call a permalink, I put that in the actual post to create the reference from the top level post to embed the media
My biggest annoyance as someone that really enjoys media production, is how bad media is across the fediverse.
I’m a firm believer that how bad these tools are for people like me is why this place doesn’t grow beyond people that can stomach primarily text based relationships.
@tychi that sounds a lot a like of what I'm doing with my BOX client to server client and ONI ActivityPub server.
https://git.sr.ht/~mariusor/box
https://git.sr.ht/~mariusor/oni -
R relay@relay.mycrowd.ca shared this topic
-
I’m not developing against ap specifically but do keep tabs
I think the two part process makes sense and that’s how I’m approaching my ap adjacent media explorations
The core is the idea of an array of posts, where each post could be a different media type, and so I upload media first and once I have what we can call a permalink, I put that in the actual post to create the reference from the top level post to embed the media
My biggest annoyance as someone that really enjoys media production, is how bad media is across the fediverse.
I’m a firm believer that how bad these tools are for people like me is why this place doesn’t grow beyond people that can stomach primarily text based relationships.
@tychi also I submitted my proposal alongside Reiver's submission: https://github.com/w3c/activitypub/issues/578#issuecomment-4366469692
-
@tychi that sounds a lot a like of what I'm doing with my BOX client to server client and ONI ActivityPub server.
https://git.sr.ht/~mariusor/box
https://git.sr.ht/~mariusor/oni@mariusor @hongminhee super cool, yeah we’re definitely tracking in similar veins in adjacent spaces
The core of my c2s philosophy is a small js lib I use for custom elements because I do think react, vue, svelte, angular, htmx, lit are all too complicated for 90% of things.
It’d be actually pretty easy for me to embed my whole os just as oni client assets, just some js.
AFK for a couple weeks but I’m sure I’ll clone and poke sometime this summer. I’ll share back when I do
-
@hongminhee 대박!
-
@hongminhee like I mentioned in a thread where Evan and Reiver were talking about this, I think having a mediaUpload that has a very similar behaviour to an inbox/outbox, but with one **small** changed detail is a bad API.
I would prefer there's either a two step process: upload media first, use resulting token in an object create, or use the outbox with for the binary data upload directly... The first one seems saner to me.
I hope you write that out! But people should implement the API we have now.
-
I hope you write that out! But people should implement the API we have now.
@evan I did on reiver's comment on github: https://github.com/w3c/activitypub/issues/578#issuecomment-4366469692
It's a compromise between the current SocialCG proposal, and what I said above.
Do you think it warrants it's own ticket?
-
@evan I did on reiver's comment on github: https://github.com/w3c/activitypub/issues/578#issuecomment-4366469692
It's a compromise between the current SocialCG proposal, and what I said above.
Do you think it warrants it's own ticket?
@mariusor @hongminhee yes, it should be a separate ticket.
I don't like your proposal for multiple reasons:
1) It doesn't include addressing, so the upload server doesn't know how to enforce authorization requests. Is the file for followers? Only addressed accounts? The public?
2) What happens if the client doesn't post the `Create` activity?
I think it's better to keep it all in one transaction together.
-
@mariusor @hongminhee yes, it should be a separate ticket.
I don't like your proposal for multiple reasons:
1) It doesn't include addressing, so the upload server doesn't know how to enforce authorization requests. Is the file for followers? Only addressed accounts? The public?
2) What happens if the client doesn't post the `Create` activity?
I think it's better to keep it all in one transaction together.
> 1) It doesn't include addressing,
As always it's the Clients that are responsible for addressing, therefore the second step should cover it, if the user/client chooses to perform it.
2) What happens if the client doesn't post the `Create` activity?
Whatever the server desires: cleanup after a while, keeping the media, etc.
Why do you think it's relevant for the specification itself?ActivityPub, to my reading, is not about how to *store* content, but about how to *distribute* content.
So after it was uploaded, it's no longer the concern of the spec, unless operated further through other ActivityPub requests.
