Just read a paper that included an "I trust companies..." measure in their "AI receptivity" outcome.
-
@grimalkina I feel a similar pain so hard when it comes to claimed results in academic software, claims about amazing progress that we can't see or re-use at all because of some and sometimes none reasons. I spent a decade at Moz begging academics to show their work, and got crickets. I have stickers, you want a shirt, we can get you the shirt! I will _make you famous_, just give me something I can use!
@mhoye it's ................ something I try to find a productive way to express thoughts about. It's a lot
-
@grimalkina Yup. I also don't understand why you should be measuring that in the first place anyway.
@GinevraCat in this case they were asking what might be useful predictors of who is more or less receptive to AI which I think is something that's interesting to understand. Just don't trust the measure validity
-
@GinevraCat in this case they were asking what might be useful predictors of who is more or less receptive to AI which I think is something that's interesting to understand. Just don't trust the measure validity
@grimalkina Thanks for the clarification.
-
@mhoye it's ................ something I try to find a productive way to express thoughts about. It's a lot
@grimalkina A decade ago now I found journal of geological sciences that accepted paper submissions via github, and the criteria was that the papers - the text, the graphs, all of it - was built from data in the repository in one step. So it was all there, raw data, the calculation methods, the text, you could see how it was all processed... it was the holy grail of published science. And they study rocks! How are these people miles ahead of us using our own tools and they study rocks!?!?
-
@grimalkina A decade ago now I found journal of geological sciences that accepted paper submissions via github, and the criteria was that the papers - the text, the graphs, all of it - was built from data in the repository in one step. So it was all there, raw data, the calculation methods, the text, you could see how it was all processed... it was the holy grail of published science. And they study rocks! How are these people miles ahead of us using our own tools and they study rocks!?!?
@grimalkina Wildly unfair to geologists, obviously, but "why can we not have the nice things that the people who rub rocks on other rocks have" still haunts me.
-
@grimalkina Wildly unfair to geologists, obviously, but "why can we not have the nice things that the people who rub rocks on other rocks have" still haunts me.
@mhoye @grimalkina Is this a thing that would actually be useful? That is, some kind of public managed git repository that does what the geologists do?
I guess what I'm asking is this thing not happening just because there's no clearly designated place and set of procedures that publications/fields/whatever can point to and say "do this, put it here, this is where the requirements and instructions on how live"?
-
@mhoye @grimalkina Is this a thing that would actually be useful? That is, some kind of public managed git repository that does what the geologists do?
I guess what I'm asking is this thing not happening just because there's no clearly designated place and set of procedures that publications/fields/whatever can point to and say "do this, put it here, this is where the requirements and instructions on how live"?
@mhoye @grimalkina (Because frankly if the only big thing stopping this is $20k/year for the services and someone going "goddamn it, we are just going to do it", well... things can be done)
-
@mhoye it's ................ something I try to find a productive way to express thoughts about. It's a lot
I ended up writing punk songs for a lot of stuff that has transcended the realm of polite conversation.
-
@mhoye @grimalkina (Because frankly if the only big thing stopping this is $20k/year for the services and someone going "goddamn it, we are just going to do it", well... things can be done)
The core challenge of changing academic publishing is that academic institutions have _actively sought out_ this situation specifically because it let those academics avoid personal accountability for denying publication, promotion, and tenure positions to people they'd be working with for decades.
One fundamental reason for the managerial-caste capture of the academy is that academics didn't want the job, actively sucked at doing it and abdicated whenever possible.
-
The core challenge of changing academic publishing is that academic institutions have _actively sought out_ this situation specifically because it let those academics avoid personal accountability for denying publication, promotion, and tenure positions to people they'd be working with for decades.
One fundamental reason for the managerial-caste capture of the academy is that academics didn't want the job, actively sucked at doing it and abdicated whenever possible.
@wordshaper @grimalkina It takes a tectonic, haha, commitment from groups wanting to change that, but the hosting is not the difficult thing, the difficult thing is changing systems of recognition in academic settings to include what are still generally novel methods of publication as "a publication".
-
@grimalkina Wildly unfair to geologists, obviously, but "why can we not have the nice things that the people who rub rocks on other rocks have" still haunts me.
@mhoye @grimalkina Most researchers I meet avoid getting involved with IT like the plague. I have shown people, over ten years ago, that you can almost completely automate the data gathering. Surveys, mobile apps and sensors integrated. Experiments running for months so they could have people from specific groups participate if and when they found them. Most of this work landed in the trash 🤨
-
@mhoye @grimalkina Most researchers I meet avoid getting involved with IT like the plague. I have shown people, over ten years ago, that you can almost completely automate the data gathering. Surveys, mobile apps and sensors integrated. Experiments running for months so they could have people from specific groups participate if and when they found them. Most of this work landed in the trash 🤨
@sandorspruit @mhoye if I had managed to survive academia I would've loved your work
-
@sandorspruit @mhoye if I had managed to survive academia I would've loved your work
@grimalkina @mhoye Thank you. I have presented this on several occasions, and people responded positively. Yet, when it came to actually discussing and developing this further there was mostly silence. For some reason, researchers still have to take onboard the fact that IT can play a great role in their work, but you need to get involved if you want to develop truly useful “nice” things.
-
@wordshaper @grimalkina It takes a tectonic, haha, commitment from groups wanting to change that, but the hosting is not the difficult thing, the difficult thing is changing systems of recognition in academic settings to include what are still generally novel methods of publication as "a publication".
@mhoye @wordshaper I think you can see useful change lessons in fields that have changed on this. Neuroscience and psychology both have made enormous strides on setting norms toward open science since I was a grad student. I mean BIG changes, really. It is very expected that you share data in most prestigious outlets. People still have plenty of criticism, but compared to other fields I interact with, night and day.
-
@mhoye @wordshaper I think you can see useful change lessons in fields that have changed on this. Neuroscience and psychology both have made enormous strides on setting norms toward open science since I was a grad student. I mean BIG changes, really. It is very expected that you share data in most prestigious outlets. People still have plenty of criticism, but compared to other fields I interact with, night and day.
@mhoye @wordshaper what is hard is definitely social and cultural, but I wouldn't underestimate the lack of access to even knowing about these systems. I have taught almost every not cs former academic I've worked with about the entire existence of eg GitHub. Then we have our own platforms (eg, OSF, https://www.cos.io/products/osf ) that I regularly get criticism from developers for using because "well I don't know what that means/I don't know that place". It's hard to speak to both sides and survive
-
@mhoye @wordshaper what is hard is definitely social and cultural, but I wouldn't underestimate the lack of access to even knowing about these systems. I have taught almost every not cs former academic I've worked with about the entire existence of eg GitHub. Then we have our own platforms (eg, OSF, https://www.cos.io/products/osf ) that I regularly get criticism from developers for using because "well I don't know what that means/I don't know that place". It's hard to speak to both sides and survive
@grimalkina @mhoye I am then wondering if setting something like this up and layering on some ease-of-use tooling (git is... not the friendliest of things to use) and actual documentation, in conjunction with an org of some sort inclined to push it, would help.
(What, me looking to add *another* post-retirement project to my summer todo list? Nah, not me!
-
@mhoye @wordshaper what is hard is definitely social and cultural, but I wouldn't underestimate the lack of access to even knowing about these systems. I have taught almost every not cs former academic I've worked with about the entire existence of eg GitHub. Then we have our own platforms (eg, OSF, https://www.cos.io/products/osf ) that I regularly get criticism from developers for using because "well I don't know what that means/I don't know that place". It's hard to speak to both sides and survive
@mhoye @wordshaper at any rate I'm going to actually try to talk about this at the DevEx research forum at UC Irvine this Friday! I feel these open science moves are really key to an evidence ecosystem that we can rely on, use to drive change, preserve and protect
-
@mhoye @wordshaper at any rate I'm going to actually try to talk about this at the DevEx research forum at UC Irvine this Friday! I feel these open science moves are really key to an evidence ecosystem that we can rely on, use to drive change, preserve and protect
@mhoye @wordshaper and I would like more methodological sharing and clarity even more than data sharing tbh. That I feel is a really culturally overlooked problem even by people building many technical solutions toward reproducibility
-
@grimalkina @mhoye I am then wondering if setting something like this up and layering on some ease-of-use tooling (git is... not the friendliest of things to use) and actual documentation, in conjunction with an org of some sort inclined to push it, would help.
(What, me looking to add *another* post-retirement project to my summer todo list? Nah, not me!
@wordshaper @mhoye all I can say is Ashley teaches programming for biologists AND "how to teach programming" for biology teachers and the needs are vast enough that I constantly worry about how to make sure we're both not working so hard we can't sustain it all!
I've been working next to folks like you two for so many years and *I* am still afraid of fucking it up on a repo or tool in some big public way. It's so hard to do all the work of science AND try to figure out better infrastructure.
-
@mhoye @wordshaper and I would like more methodological sharing and clarity even more than data sharing tbh. That I feel is a really culturally overlooked problem even by people building many technical solutions toward reproducibility
@grimalkina @mhoye It sounds like there are many things that need updating/reworking/someone shouting at, tbh. Which isn't particularly surprising, since that's pretty much describing everything.
I suspect, as a regrettable history of volunteering has taught me, that a lot of these problems can be ameliorated by someone just going "goddamn it, fine, I will Do The Thing." (Doesn't work for everything but is sadly effective for the things people circle 'round and wish were actually working)
