I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
-
@kopper@not-brain.d.on-t.work @hongminhee@hollo.social expansion is actually really annoying because the resulting JSON has annoyingly similar keys to lookup in a hashmap, wasting cache lines, and CPU time
@natty @hongminhee i would imagine a Good hash algorithm wouldn't care about the similarity of the keys, no? -
@hongminhee take this part with a grain of salt because my benchmarks for it are with dotNetRdf which is the slowest C# implementation i know of (hence my replacement library), but JSON-LD is slower than RSA validation, which is one of the pain points around authorized fetch scalability
wetdry.world/@kopper/114678924693500011@hongminhee i put this in a quote but people reading the thread may also be interested: json-ld compaction does not really save that much bandwidth over having all the namespaces explicitly written in property names if you're gzipping (and you are gzipping, right? this is json. make sure your nginx gzip_types includes ld+json and activity+json)
RE: not-brain.d.on-t.work/notes/aihftmbjpxdyb9k7 -
@evan@cosocial.ca I don't remember exactly, but I think I came across it while doing research before developing Fedify. I probably didn't use it because the TypeScript type definitions were missing. In the end, I ended up making something similar in Fedify anyway.
@hongminhee that's great!
-
@hongminhee from the point of view of someone who is "maintaining" a JSON-LD processing fedi software and has implemented their own JSON-LD processing library (which is, to my knowledge, the fastest in it's programming language), JSON-LD is pure overhead. there is nothing it allows for that can't be done with
1. making fields which take multiple values explicit
2. always using namespaces and letting HTTP compression take care of minimizing the transfer
without JSON-LD, fedi software could use zero-ish-copy deserialization for a majority of their objects (when strings aren't escaped) through tools like serde_json and Cow<str>, or System.Text.Json.JsonDocument. JSON-LD processing effectively mandates a JSON node DOM (in the algorithms standardized, you may be able to get rid of it with Clever Programming)
additionally, due to JSON-LD 1.1 features like @type:@json, you can not even fetch contexts ahead of time of running JSON DOM transformations, meaning all JSON-LD code has to be async (in the languages which has the concept), potentially losing out on significant optimizations that can't be done in coroutines due to various reasons (e.g. C# async methods can't have ref structs, Rust async functions usually require thread safety due to tokio's prevalence, even if they're ran in a single-threaded runtime)
this is after context processing introducing network dependency to the deserialization of data, wasting time and data on non-server cases (e.g. activitypub C2S). sure you can cache individual contexts, but then the context can change underneath you, desynchronizing your cached context and, in the worst case, opening you up to security vulnerabilities
json-ld is not my favorite part of this protocolhm, we really need to differentiate between users responsibility and dev responsibility.
Not sure if Hong saw the draft about the AP kv thing, it supports either JSON-LD fields _or_ as:attachment / as:context …
wtf do I want to say.user story:
We are working on 2 major and 3 projects fulltime which is
- federation of wikibase / wikidata
- federation of Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/
and these https://codeberg.org/Menschys/fedi-codebaseLet's say we want to federate a Country, then all the knowledge is sent in `attachment` with the fully qualified qikidata url in `context` [as:context - not @context ! - this is so confusing :)]
For example the according entries from the PressFreedomIndex `collection` (co-founder of freelens here
But anyway, the idea about having
"wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/" in the `@context` was that any user can consume and federate wikibase
incl.
🧵 1/2 -
hm, we really need to differentiate between users responsibility and dev responsibility.
Not sure if Hong saw the draft about the AP kv thing, it supports either JSON-LD fields _or_ as:attachment / as:context …
wtf do I want to say.user story:
We are working on 2 major and 3 projects fulltime which is
- federation of wikibase / wikidata
- federation of Public Broadcasters https://www.publicmediaalliance.org/public-broadcasters-create-public-spaces-incubator/
and these https://codeberg.org/Menschys/fedi-codebaseLet's say we want to federate a Country, then all the knowledge is sent in `attachment` with the fully qualified qikidata url in `context` [as:context - not @context ! - this is so confusing :)]
For example the according entries from the PressFreedomIndex `collection` (co-founder of freelens hereBut anyway, the idea about having
"wd": "https://www.wikidata.org/wiki/Special:EntityData/",
"wdt": "https://www.wikidata.org/prop/direct/" in the `@context` was that any user can consume and federate wikibase
incl.
🧵 1/2incl.
- the properties in all the languages of the world
- the knowledge of the world in all the languages
- the wikidata relations and qualified statements including the nameMap etc. and all the urls to all wikiprojects incl. their languages and knowledgeHow else could I say to other softwares if they want all users qualified data, use wikidata vocabulary?
wikipedia, wikidata, EBU, Public Broadcasters, taxi data is _all_ JSON-LD … -
incl.
- the properties in all the languages of the world
- the knowledge of the world in all the languages
- the wikidata relations and qualified statements including the nameMap etc. and all the urls to all wikiprojects incl. their languages and knowledgeHow else could I say to other softwares if they want all users qualified data, use wikidata vocabulary?
wikipedia, wikidata, EBU, Public Broadcasters, taxi data is _all_ JSON-LD …@sl007 @hongminhee @julian i feel like you're falling into a trap i've seen a lot around AP spaces: just because the data can be contorted to represent something does not mean software will interpret it as such.
any software who wants to support wikidata statements and relations will have to go out of their way to implement that manually with or without json-ld in the mix, and interoperability between those software will have to specify how that works. and in your specification you can indeed make it so Simply Linking to the wikidata json-ld (which i don't believe it provides out of the box, it does for xml, turtle, and n-triples, if we're talking about rdf. if not, their bespoke json format is just as authoritative) can work (but i'd say using the Qxxx and Pxx IDs and letting the software figure out how to access it would be better!)
if you have the dream of making an as:Note and having it's as:attributedTo be the wikidata entity for alan turing... sorry, nobody other than maybe your own software will even attempt interpreting that -
@sl007 @hongminhee @julian i feel like you're falling into a trap i've seen a lot around AP spaces: just because the data can be contorted to represent something does not mean software will interpret it as such.
any software who wants to support wikidata statements and relations will have to go out of their way to implement that manually with or without json-ld in the mix, and interoperability between those software will have to specify how that works. and in your specification you can indeed make it so Simply Linking to the wikidata json-ld (which i don't believe it provides out of the box, it does for xml, turtle, and n-triples, if we're talking about rdf. if not, their bespoke json format is just as authoritative) can work (but i'd say using the Qxxx and Pxx IDs and letting the software figure out how to access it would be better!)
if you have the dream of making an as:Note and having it's as:attributedTo be the wikidata entity for alan turing... sorry, nobody other than maybe your own software will even attempt interpreting that@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authors -
@sl007 @hongminhee @julian i feel like you're falling into a trap i've seen a lot around AP spaces: just because the data can be contorted to represent something does not mean software will interpret it as such.
any software who wants to support wikidata statements and relations will have to go out of their way to implement that manually with or without json-ld in the mix, and interoperability between those software will have to specify how that works. and in your specification you can indeed make it so Simply Linking to the wikidata json-ld (which i don't believe it provides out of the box, it does for xml, turtle, and n-triples, if we're talking about rdf. if not, their bespoke json format is just as authoritative) can work (but i'd say using the Qxxx and Pxx IDs and letting the software figure out how to access it would be better!)
if you have the dream of making an as:Note and having it's as:attributedTo be the wikidata entity for alan turing... sorry, nobody other than maybe your own software will even attempt interpreting thatah, no - that is a misunderstanding!
Anyone can feel free to represent the texts only and the user at least "knows" it.
But the thing for Public Broadcasters means 47mio. users in DE alone and given the unified codebase for the 5 projects _these_ softwares will interpret it.
It does JSON-LD you could just check by asking for any JSON-LD e.g. Q1055 (Hamburg) - it is content-negotiation.
The taxiteam software is funded by the German yellow cabs - the official ones (!) the codename is FCKUBR
and I have no doubt about adoption fortunately.Maybe we can work out better examples …
-
@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authorsit is just damned simple, your as: Client can do so much by asking wikidata, OSM, federated geocoding and not our system. When you use a property for the first time, the client can cache its names in the languages of the user etc.
-
it is just damned simple, your as: Client can do so much by asking wikidata, OSM, federated geocoding and not our system. When you use a property for the first time, the client can cache its names in the languages of the user etc.
@sl007 @hongminhee @julian i genuinely can't see where json-ld is relevant here. if your client wants to support wikidata and OSM then it can do that with or without json-ld being involved. you are going to have to document how this integration works anyhow if you want anyone else to do so -
@hongminhee @sl007 @julian attempting to support this kind of "data contortion" (i made this up and prolly isnt the right way to describe this) would rapidly balloon the scope of every fedi software ever. i don't believe anyone would want to develop for such ecosystem
a similar example i saw was someone attempting to explain how you can partially inline an as:object you as:Like'd in order to specify you only liked that past version of it and if it changed your like shouldn't count. without describing this exact scenario i don't believe any software, json-ld capable or not, would interpret that Like as such. same thing with the long-form text FEP which attempts to support non-activitypub authorsjust btw, we had many W3C Social CG meetings about the importance and how to use the as:context property - not the JSON-LD @context and we all agreed.
About 30-40 devs attended.
Between 2016 and 2024 I attended basically any meeting. I felt that using wikidata urls in as:context was nice for anyone. -
@sl007 @hongminhee @julian i genuinely can't see where json-ld is relevant here. if your client wants to support wikidata and OSM then it can do that with or without json-ld being involved. you are going to have to document how this integration works anyhow if you want anyone else to do so
if I see wd: in lets say 3 of 12 AP software, I know tha I can give the user wikibase support.
-
if I see wd: in lets say 3 of 12 AP software, I know tha I can give the user wikibase support.
anyway, if you like RDF and `content` is html how about RDFa ?
For us it would work similar. If we have any "convention" before we stop writing it might save time of rewriting -
just btw, we had many W3C Social CG meetings about the importance and how to use the as:context property - not the JSON-LD @context and we all agreed.
About 30-40 devs attended.
Between 2016 and 2024 I attended basically any meeting. I felt that using wikidata urls in as:context was nice for anyone.@sl007@digitalcourage.social eh? You use
context?ForumWG decided to use context to represent threads in threadiverse software (and Mastodon too, now).
Just FYI.
-
@sl007@digitalcourage.social eh? You use
context?ForumWG decided to use context to represent threads in threadiverse software (and Mastodon too, now).
Just FYI.
hm, strange. I would really not ignore all the official ActivityPub meetings between 2016 and 2014
Maybe it would be worth to read the W3C minutes of SocialCG 2019 ff
Think, it _should_ have been 2019 or 2021 cause 2021 it wasn't on the "waitlist" anymore :https://socialhub.activitypub.rocks/t/2021-01-09-socialcg-meeting-fep/1246But:
It dates back to 2020 https://socialhub.activitypub.rocks/t/context-vs-conversation/578 and after mastodon and pleroma agreed to the us , the 2 Social CG meetings are linked by me in the thread.
Then we had the 2020 brilliant Conf.So, well, we use it for ActivityPub spec says and what was decided there by all …
mastodon is _not_ able to introduce breaking changes to a W3C standard
Just FYI. -
hm, strange. I would really not ignore all the official ActivityPub meetings between 2016 and 2014
Maybe it would be worth to read the W3C minutes of SocialCG 2019 ff
Think, it _should_ have been 2019 or 2021 cause 2021 it wasn't on the "waitlist" anymore :https://socialhub.activitypub.rocks/t/2021-01-09-socialcg-meeting-fep/1246But:
It dates back to 2020 https://socialhub.activitypub.rocks/t/context-vs-conversation/578 and after mastodon and pleroma agreed to the us , the 2 Social CG meetings are linked by me in the thread.
Then we had the 2020 brilliant Conf.So, well, we use it for ActivityPub spec says and what was decided there by all …
mastodon is _not_ able to introduce breaking changes to a W3C standard
Just FYI.Can you specify "threadiverse" software cause like all properties which are not marked as "functional" in the ActivityPub Spec, `context` is an Array and I do not see any problem …
-
@liaizon@social.wake.st actually, oddly, I did receive @silverpill@mitra.social's response, so it seems to be I can access some replies from Mitra, but not all.
-
@silverpill@mitra.social POSTing that inbox sends a 404?
-
I have deeply mixed feelings about #ActivityPub's adoption of JSON-LD, as someone who's spent way too long dealing with it while building #Fedify.
Part of me wishes it had never happened. A lot of developers jump into ActivityPub development without really understanding JSON-LD, and honestly, can you blame them? The result is a growing number of implementations producing technically invalid JSON-LD. It works, sort of, because everyone's just pattern-matching against what Mastodon does, but it's not correct. And even developers who do take the time to understand JSON-LD often end up hardcoding their documents anyway, because proper JSON-LD processor libraries simply don't exist for many languages. No safety net, no validation, just vibes and hoping you got the
@contextright. Naturally, mistakes creep in.But then the other part of me thinks: well, we're stuck with JSON-LD now. There's no going back. So wouldn't it be nice if people actually used it properly? Process the documents, normalize them, do the compaction and expansion dance the way the spec intended. That's what Fedify does.
Here's the part that really gets to me, though. Because Fedify actually processes JSON-LD correctly, it's more likely to break when talking to implementations that produce malformed documents. From the end user's perspective, Fedify looks like the fragile one. “Why can't I follow this person?” Well, because their server is emitting garbage JSON-LD that happens to work with implementations that just treat it as a regular JSON blob. Every time I get one of these bug reports, I feel a certain injustice. Like being the only person in the group project who actually read the assignment.
To be fair, there are real practical reasons why most people don't bother with proper JSON-LD processing. Implementing a full processor is genuinely a lot of work. It leans on the entire Linked Data stack, which is bigger than most people expect going in. And the performance cost isn't trivial either. Fedify uses some tricks to keep things fast, and I'll be honest, that code isn't my proudest work.
Anyway, none of this is going anywhere. Just me grumbling into the void. If you're building an ActivityPub implementation, maybe consider using a JSON-LD processor if one's available for your language. And if you're not going to, at least test your output against implementations that do.
#JSONLD #fedidev
@hongminhee i struggling badly with rust cuz it's rust being rust... i can imagine a duck typing lang might have easier time
-
@hongminhee i struggling badly with rust cuz it's rust being rust... i can imagine a duck typing lang might have easier time
@hongminhee most of the time, i cant compile if really do jsonld as it is. unless i box everything onto the heap. but then it became a de-optimization point to the rest of the software
