#Deepfakes are everywhere, but #DigitalForensics investigators are fighting back:
-
@tk @nartagnan @legendarybassoon @grototo @AudeCaussarieu
Générer plein d'images par IA, demander a des petites sous payées de dessiner les lignes fuites. On fait deux jeux de données : les images avec un seul point d'intersection et les autres. On rajoute des vrais images dans la première catégorie. On lance l'entraînement d’un modèle ou un fine tunning d’un modèle existant.
@youen
@tk @legendarybassoon @grototo @AudeCaussarieuOui, c'est faisable.
Mais se concentrer sur X c'est délaisser Y.
Au début, quand il fallait compter les doigts des mains, les modeles qui étaient bons sur les mains étaient mauvais sur le reste.L'amélioration n'est venue qu'en multipllant le nb de paramètre des modèles. Et donc le coût de génération d'une seule image.
C'est exponentiel.
Et j'ose croire qu'il n'y a plu moyen de multiplier encore par 2 leurs coûts, sans revenus.
-
@aearo @FabMusacchio What's interesting to me is WHY AI generated images will maybe never get it right.
Put simply, the consumers of the AI generated images do not care whether or not all the lines properly converge onto a vanishing point. Human vision may care about weird extra fingers, but vanishing point convergence? Nope. Don't care.
Human viewers will never notice these perspective errors, so AI models have no incentive to fix them.
That, but I also think it's a really hard, abstract thing to train the models on regardless.
I could be wrong about this! Maybe it's easier than I think. But it's not like you can just say to the model "oh yeah, and make sure all the edges of things follow the rules of perspective." It has to learn those rules the same way it learns everything else - basically, by looking at a bunch of examples and getting a "feel" for what's right. (Well, "a feel" = "the values of the model's weights updated to produce this result" and so forth, but yunno.)
But it's not the kind of detail that immediately jumps out, as long as it's not *too* wrong. Observing it requires both figuring out which lines are relevant, and knowing how those lines should behave, and image-gen AI has no special ability to do either of those things. It has no ability to follow rules precisely.
The fact that human brains can also look at the pictures and not immediately go "wait, that's wrong" gives me confidence that AI models won't get it either. Even humans generally need to get out a ruler and start measuring. I think it's hard for human brains to just see it for pretty much the same reason it's hard for AI, but until AGI is a thing, strategies like "know the rules concretely" and "draw a line with a ruler" are more or less out of reach for the AI.
-
That, but I also think it's a really hard, abstract thing to train the models on regardless.
I could be wrong about this! Maybe it's easier than I think. But it's not like you can just say to the model "oh yeah, and make sure all the edges of things follow the rules of perspective." It has to learn those rules the same way it learns everything else - basically, by looking at a bunch of examples and getting a "feel" for what's right. (Well, "a feel" = "the values of the model's weights updated to produce this result" and so forth, but yunno.)
But it's not the kind of detail that immediately jumps out, as long as it's not *too* wrong. Observing it requires both figuring out which lines are relevant, and knowing how those lines should behave, and image-gen AI has no special ability to do either of those things. It has no ability to follow rules precisely.
The fact that human brains can also look at the pictures and not immediately go "wait, that's wrong" gives me confidence that AI models won't get it either. Even humans generally need to get out a ruler and start measuring. I think it's hard for human brains to just see it for pretty much the same reason it's hard for AI, but until AGI is a thing, strategies like "know the rules concretely" and "draw a line with a ruler" are more or less out of reach for the AI.
@aearo @FabMusacchio My guess is that there might be some "secret sauce" to improving stable diffusion generated 3D CGI models. Right now they're kinda crap but there is WAY LESS training data available.
But if normal typical 2D images could be "reverse engineered" into 3D models, then that could be a plausible path to fixing all the perspective and lighting errors, as well as allowing better looking stuff with reflections and refraction and such.
-
#Deepfakes are everywhere, but #DigitalForensics investigators are fighting back:
This kind of thing will only be useful briefly for forensics, because immediately after that they can become feedback for the image generators — keep refining until this analysis passes.
-
#Deepfakes are everywhere, but #DigitalForensics investigators are fighting back:
@FabMusacchio @peterdrake In so-called AI-generated or OCR-based image descriptions, AI is often spelled Al.
-
@FabMusacchio @peterdrake In so-called AI-generated or OCR-based image descriptions, AI is often spelled Al.
@pesh @FabMusacchio ... and in the font used by Tusky, those are indistinguishable.
-
@NatureMC @ilusenn @FabMusacchio Definitely not a fan either but I am aware of where this one leads (or, at least, should).
@JonnyT
On Mastodon, real URL length does not count in the character limit of a post. URL shorteners on Mastodon have many drawbacks but no advantage.
@NatureMC @ilusenn @FabMusacchio -
@Steel_Virgin @FabMusacchio The goal wasn't to show that picture was fake. The goal was to show the technique of analyzing vanishing point perspective errors.
@isaackuo @FabMusacchio I understand. But I'm not sure it so relevant to test the efficiency of this technique on pictures that are so bad you can tell by looking at it.
I have another technique to show you ! I personally test if pictures are real photographs by counting the fingers of the hands of human figures.
Here are an example of the efficiency of my method. Very solid proof.
-
It's not the sun's rays that meet at a point, it's the lines from the objects' shadows to the corresponding points on the objects that should meet at a point.
The statement about the sun's rays being effectively parallel just means that the direction of the light source can be considered the same for all objects.
@leadore @leah @f4grx @FabMusacchio I'm still bothered though: the shadows of those cubes are all splayed-out relative to one another, but in actual sunlight they should be parallel to one another. But the "proof that it's fake" seems to ignore that?
-
#Deepfakes are everywhere, but #DigitalForensics investigators are fighting back:
@FabMusacchio Flat-earthers will not understand this
-
#Deepfakes are everywhere, but #DigitalForensics investigators are fighting back:
@FabMusacchio but this doesnt work.
once held a school presentation on moon landing conspiracy theories. on of the main arguments is "the lines dont match".
this is an issue inherently in photography.
here is a site talking about it (theory 1) https://www.rmg.co.uk/stories/space-astronomy/moon-landing-conspiracy-theories-debunked -
@JonnyT
On Mastodon, real URL length does not count in the character limit of a post. URL shorteners on Mastodon have many drawbacks but no advantage.
@NatureMC @ilusenn @FabMusacchio -
Good points… except the bad one: the dinosaur graphic shows a line connecting different toes to the horizon
@DavidM_yeg @FabMusacchio To be fair, it’s hard to find matching toes in that picture. AI images are notoriously bad at producing identical reflections, and this one is no exception.
-
@DavidM_yeg @FabMusacchio To be fair, it’s hard to find matching toes in that picture. AI images are notoriously bad at producing identical reflections, and this one is no exception.
The matching toe isn’t necessarily visible even in a real image. In this case, it’s definitely a human error because they chose an inside toe on the figure and an outside toe on the reflection. That would be an analysis error whether the image was real or generated.
-
@leadore @leah @f4grx @FabMusacchio I'm still bothered though: the shadows of those cubes are all splayed-out relative to one another, but in actual sunlight they should be parallel to one another. But the "proof that it's fake" seems to ignore that?
@seachaint @leadore @leah @f4grx @FabMusacchio because the sun is at a distance, parallel lines meet at a vanishing point, like train tracks.
-
@Deixis9 what's the origin of the image?
Thanks
I see that it's commercialised e.g. <https://lovingsquad.com/product/ai-slop-is-not-art-chatbots-are-not-your-friends-shirt/>, <https://pagtee.com/product/t-shirt/1926944-ai-slop-is-not-art-chatbots-are-not-your-friends-cyborg-painting-flames>, <https://zerevia.com/product/t-shirt/1926944-ai-slop-is-not-art-chatbots-are-not-your-friends-cyborg-painting-flames>, <https://www.spellingmistakescostlives.com/product-page/copy-of-ai-slop-is-not-art-chatbots-are-not-your-friends-sticker>, however there's no real acknowledgement of an original artist; no credit, as far as I can tell.
@fuzzy @Deixis9 @FabMusacchio It's my original image, but thanks for finding that page, it's a counterfeit t-shirt website that rips off artist designs. I'll try get them to take it down
-
@seachaint @leadore @leah @f4grx @FabMusacchio because the sun is at a distance, parallel lines meet at a vanishing point, like train tracks.
@ghoppe @leadore @leah @f4grx @FabMusacchio While that is technically true, the splay of shadows on Earth is negligible and nearly imperceptible in real life. In the example image it's as if the sun is a lamp a few metres away.
-
@leadore @leah @f4grx @FabMusacchio I'm still bothered though: the shadows of those cubes are all splayed-out relative to one another, but in actual sunlight they should be parallel to one another. But the "proof that it's fake" seems to ignore that?
@seachaint
Yeah, I noticed that. All the example images have other things about them that are wrong and give them away as fake.I think the OP wanted to just focus specifically on how to check for lines that should meet at a point, as one kind of objective test we can use, and not get into all the other stuff.
-
@AudeCaussarieu
J'aurais jamais pensé que les points de fuites pouvaient être une façon de vérifier la véracité d'une image. 🤯@mathieu_ @AudeCaussarieu fondamentalement, les AI generatives fonctionnent a l'echelle des pixels d'une image (ou video), et n'ont pas vraiment un model abstrait d'une scene (pas de plan 3D en tete)
C'est pour ca aussi sur les model plus vieux les doigts surnumeraires, les 3e mains mal placees, etc. Meme-si ces erreures la sont plus facilement corrigibles.
-
#Deepfakes are everywhere, but #DigitalForensics investigators are fighting back:
@FabMusacchio in the first picture, the lines may be off by measurements inaccuracies. a mirror image depends on a perfect mirror on the physical mirror. and the shadows may be off by camera lens distortion.
