i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
-
@whitequark @deborahh @danlyke ie, the sort of thing a linter does?
-
@whitequark "92 boosts, 115 favourites" damn
I swear to god sometimes Mastodon is just "old person yells at thing".
Researchers spend tons of time and money trying to solve Sudoku in polynomial time not because Sudoku is such a important problem to humanity, but because it's a NP-Hard problem, and you can thus reduce all other NP-complete problems to Sudoku and solve them all in polynomial time if you can solve Sudoku in polynomial time.
The research challenge is disentangling content from style in a learned embedding space, it's a classic representation learning problem that's genuinely hard: 1) Two functions that do the same thing should have identical content embeddings but different style embeddings, 2) Style must generalise to unseen code patterns, not just pattern-match known rules, 3) It's unsupervised, so there's no labeled (code_A, same_code_in_style_B) training pairs.
Code formatting is actually a very good medium to test this hypothesis, because you have an infinite latent space of code that does the exact same thing but is stylistically different.
@budududuroiu the reason I was reading the paper is because I'm working on the same problem and I think the encoding presented in the paper makes no sense at all to use
-
@budududuroiu the reason I was reading the paper is because I'm working on the same problem and I think the encoding presented in the paper makes no sense at all to use
@whitequark to use for what? It's research, it's not meant to create something for industry use. Academia already suffers from the "File-drawer problem". I also did research on using GANs for Outlier Detection, when most of the time Outlier Detection is a classification problem, not a learned representation problem.
-
@whitequark to use for what? It's research, it's not meant to create something for industry use. Academia already suffers from the "File-drawer problem". I also did research on using GANs for Outlier Detection, when most of the time Outlier Detection is a classification problem, not a learned representation problem.
@budududuroiu yes yes i know you're here because you look at trends and start arguments, now move on to something else and stop wasting my time
-
@budududuroiu yes yes i know you're here because you look at trends and start arguments, now move on to something else and stop wasting my time
@whitequark lmao, have fun "clowning" on stuff you don't understand
-
@whitequark lmao, have fun "clowning" on stuff you don't understand
@budududuroiu go take a short walk off a long pier
-
@porglezomp you'll love Fig. 6
@whitequark "If" right next to "if"
-
i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
the "ideal" (their choice of words) case is 64.2%
@whitequark if I have understood you correctly, they're saying 64% functional is a satisfactory result?
-
@whitequark if I have understood you correctly, they're saying 64% functional is a satisfactory result?
@FibroJedi that's my read of it yeah
-
i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
the "ideal" (their choice of words) case is 64.2%
@whitequark Whenever I hear about these benchmarks I can't help but wonder how people can say these things with a straight face. -
@FibroJedi that's my read of it yeah
@whitequark Maybe they'd like their phone and car 64% functional as a real world test
.Some of those logic misses/switches are disturbing. I don't know how it's allowable.
If the code works 100%, and "reformatting" it reduces that % then it's wrong by definition.
-
@whitequark @porglezomp I'm spitting out my drink at j++ → j--. Holy shit.
@xgranade @whitequark @porglezomp
I think reversing the `j` for loop is actually wanted by them? It's labelled "ground truth", and it is a potential valid optimisation -
i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
the "ideal" (their choice of words) case is 64.2%
@whitequark But... why? Why not just use a linter?
-
@whitequark because "the thing we're promoting is incredibly dangerous, and not in fun ways" is not really the thing anyone wants to be cited for
@ireneista @whitequark Now, show me the numbers on the effort to make a rule-based style file compared to this. Because I'm sure that A_c is 100.0 in that case.
-
@whitequark But... why? Why not just use a linter?
@DaKangaroo see edit
-
i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
the "ideal" (their choice of words) case is 64.2%
@whitequark I cannot even
-
@porglezomp you'll love Fig. 6
@whitequark @porglezomp long live the new flesh
-
@ireneista @whitequark Now, show me the numbers on the effort to make a rule-based style file compared to this. Because I'm sure that A_c is 100.0 in that case.
@GeoffWozniak @ireneista so the problem i'm solving is that while for C++, you have tools like clang-format which are nice and flexible, for Rust you have rustfmt which is rigid and makes your code look like ass. I do not like my code looking like ass but I am also receptive to the idea that introducing as many knobs as clang-format has into rustfmt would make it unmaintainable
-
i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
the "ideal" (their choice of words) case is 64.2%
@whitequark this technology is going to be amazing for the competitive advantage of the few software firms that refuse to use it
-
i'm at a loss of words after reading a paper about reformatting code using an ML model that has a measured statistical quantity A_c which says how often the reformatted code behaves the same as the original
the "ideal" (their choice of words) case is 64.2%
@whitequark Saw your edit with the motivation for reading research. I doubt there's anything out there doing this well, but I think the smart approach to doing it well would be to evaluate and score a bunch of candidate standard-class rules across the codebase, solve for a set that maximally approximates what's already there, then apply some sort of pattern learning for the remaining instances that "break the rules", hopefully identifying correlations between them.
Basically, going as far as you can with simple comprehensible deterministic rules before you start throwing magical statistics at it.
