sure this is all very bad for activitypub but this is truly amazing content
-
@trishalynn Sure! So, there are two main events that happen here:
- The server receives the third-party data
- One of the server's users reads the third-party dataThere are usually at least a few minutes, and sometimes a few hours or even days between these two events.
@trishalynn The question is, when should the server *verify* the third-party data?
To be conservative, at the very least, the third party data should be verified before one of the server's users reads it.
-
@trishalynn The question is, when should the server *verify* the third-party data?
To be conservative, at the very least, the third party data should be verified before one of the server's users reads it.
@trishalynn If the user is online when the data is received, there may be no time between the time the data is received and when the user reads it.
However, most users aren't online most of the time. There's a strong chance that there are minutes, hours, or days between when the data is received and when it is read.
-
@trishalynn If the user is online when the data is received, there may be no time between the time the data is received and when the user reads it.
However, most users aren't online most of the time. There's a strong chance that there are minutes, hours, or days between when the data is received and when it is read.
@evan I should think that the server should verify first even if the user is not active online.
-
@trishalynn If the user is online when the data is received, there may be no time between the time the data is received and when the user reads it.
However, most users aren't online most of the time. There's a strong chance that there are minutes, hours, or days between when the data is received and when it is read.
@trishalynn Most ActivityPub implementations today lean waaaaaay into the early part of this gap -- verifying the data as soon as it is received.
The problem with this is that sometimes hundreds or even thousands of servers receive the data within a few seconds -- and if they all verify the data with the third-party server immediately, it can swamp that server with requests.
-
@evan I should think that the server should verify first even if the user is not active online.
@trishalynn Before it's read by a user, yes.
-
@trishalynn Most ActivityPub implementations today lean waaaaaay into the early part of this gap -- verifying the data as soon as it is received.
The problem with this is that sometimes hundreds or even thousands of servers receive the data within a few seconds -- and if they all verify the data with the third-party server immediately, it can swamp that server with requests.
@trishalynn One way to relieve this pressure on the third party server is to space out all these requests by seconds or even minutes. There are a couple of ways to do this.
-
@trishalynn One way to relieve this pressure on the third party server is to space out all these requests by seconds or even minutes. There are a couple of ways to do this.
@trishalynn One is to wait until the first reader reads the data. That event is going to vary wildly across servers, so it will spread out the requests and lower the load on the third-party server. The downside of this technique is that it introduces some extra time for that first read. Usually not a lot, but some.
-
@trishalynn One is to wait until the first reader reads the data. That event is going to vary wildly across servers, so it will spread out the requests and lower the load on the third-party server. The downside of this technique is that it introduces some extra time for that first read. Usually not a lot, but some.
@trishalynn Another is for the receiving server to wait a random number of seconds or minutes before doing the verification request. This spaces out the requests, and hopefully avoids the little delay for the user on first read. At worst, if a user tries to read the data before the verification timeout, you can do the verification then -- it's no worse than the previous method, and will usually be better.
-
@trishalynn One is to wait until the first reader reads the data. That event is going to vary wildly across servers, so it will spread out the requests and lower the load on the third-party server. The downside of this technique is that it introduces some extra time for that first read. Usually not a lot, but some.
@evan (Could you please let me know when you’re done explaining? I don’t want to jump in with clarifying Qs till you’re done.)
-
@trishalynn Another is for the receiving server to wait a random number of seconds or minutes before doing the verification request. This spaces out the requests, and hopefully avoids the little delay for the user on first read. At worst, if a user tries to read the data before the verification timeout, you can do the verification then -- it's no worse than the previous method, and will usually be better.
@trishalynn So, the last part, which I think is most controversial, is showing the unverified data to the user -- doing the verification *after* the first read.
This requires a lot of trust between the actors. But if a sending actor has sent 10 or 1000 or 10,000 shares, all of which have previously verified correctly, there's a very good chance that share number 10001 is also going to verify correctly.
-
@trishalynn So, the last part, which I think is most controversial, is showing the unverified data to the user -- doing the verification *after* the first read.
This requires a lot of trust between the actors. But if a sending actor has sent 10 or 1000 or 10,000 shares, all of which have previously verified correctly, there's a very good chance that share number 10001 is also going to verify correctly.
@trishalynn This requires a lot more tracking on the receiving server's part. I'm not even sure the performance benefits are that great, compared to waiting for first-read instead of verifying on receipt. But for high-volume servers, it might be a valuable strategy in the future.
-
@evan (Could you please let me know when you’re done explaining? I don’t want to jump in with clarifying Qs till you’re done.)
@trishalynn I think I'm done!
-
@trishalynn This requires a lot more tracking on the receiving server's part. I'm not even sure the performance benefits are that great, compared to waiting for first-read instead of verifying on receipt. But for high-volume servers, it might be a valuable strategy in the future.
@evan What's the effect on a high-volume server versus a lower-volume server when the ethos of "trust, then verify" is used to implement a solution?
-
@evan @anders @promovicz @laurenshof It doesn't need to break backwards compatibility tho
But anyway
Long conversation potentially
@cwebber The original conversation was about removing JSON-LD and potentially using another schema language or making one up, or throwing away extensibility altogether. That would break backwards compatibility.
I agree, we might be able to add digital signatures without removing JSON-LD.
-
@evan What's the effect on a high-volume server versus a lower-volume server when the ethos of "trust, then verify" is used to implement a solution?
@trishalynn OK, so, you're good with the idea that the data doesn't have to be verified until the first user reads it, correct? We're good up until there?
-
@trishalynn OK, so, you're good with the idea that the data doesn't have to be verified until the first user reads it, correct? We're good up until there?
@trishalynn Most of the benefits happen there. It would be great to see more ActivityPub implementations take that approach, because it would ease up on smaller servers. (Christine gave the example of when she shares posts by her friend Viv, which kills Viv's server.)
-
@trishalynn Most of the benefits happen there. It would be great to see more ActivityPub implementations take that approach, because it would ease up on smaller servers. (Christine gave the example of when she shares posts by her friend Viv, which kills Viv's server.)
@trishalynn I think that maintaining trust metrics has some resource requirements -- you have to track by server and maybe by actor how many times you've received third-party data from them, and how many times it has verified correctly.
-
@trishalynn I think that maintaining trust metrics has some resource requirements -- you have to track by server and maybe by actor how many times you've received third-party data from them, and how many times it has verified correctly.
@trishalynn I think there are limited benefits to using these trust metrics to verify even *after* the first read. So, it would only be on a server with a lot of scale, where those benefits multiply out over thousands or millions of interactions, where that technique might pay off.
-
@trishalynn I think there are limited benefits to using these trust metrics to verify even *after* the first read. So, it would only be on a server with a lot of scale, where those benefits multiply out over thousands or millions of interactions, where that technique might pay off.
@trishalynn I hope that answers your question.
-
@trishalynn I hope that answers your question.
@trishalynn Oh, I should probably say: trust is what we do when we are not certain. If I receive my 10 millionth share from mastodon.social, and I decide to delay verifying it, there's a non-zero chance that this is the time that mastodon.social takes its heel turn and sends me fake data. Trust is accepting that non-zero chance. For users or developers that can't accept that chance, waiting to verify when the first user reads the data is still a great benefit, and also a lot easier to code for.
