I make it part of my job to educate my graduate students about open access, open-source software, open data, practices for #reproducibility, and #OpenScience
SymfonyCon has been a fantastic experience, especially for the opportunity to meet so many incredible people for the first time. Our live discussions have already yielded tangible results. After PHPUnit, GrumPHP is now including a composer.lock for achieving total reproducibility.
This underscores the importance of the social aspect in our work, it's crucial and should never be underestimated.
I am super excited about this mini-conference on #reproducibility in #linguistics that I am organising this evening: Four of my M.A. students will be reporting on their attempts to reproduce the results of four published quantitative linguistics papers for which the data is available, but not the code!
Colleagues, they have a lot of things to report! So, if you're in the area (Cologne), do come along! There will be #ReproducibiliTea and Christmas biscuits! 🍵 🍪 #OpenScience
As promised, here are some of the (anonymised) highlights from my students' attempts to reproduce the results of four published #linguistics studies using the authors' original data, which my students brilliantly presented at our mini-conference on #reproducibility yesterday. #OpenScience
1/ One student perfectly replicated the statistics and plots for RQ1 of the paper she chose, but could not replicate RQ2 because some data for this was missing. She contacted the authors. They never replied.
I've released version 1.19.0-3 of my @nlnetlabs#unbound#docker image with updated build environments and unbound base to #alpinelinux 3.19.0. I have reduced two image layers by adding a separate build stage. The tags of the build environments got pinned for better #reproducibility, too.
Good point from Benjamin Haibe-Kains on unreproducible research: "The problem is not so much that editors waive rules about transparency,…but that editors and reviewers might be 'poorly educated on the real versus fictitious obstacles for sharing data, code and so on, so they tend to be content with very shallow, unreasonable justifications [for not sharing such information]'." https://www.nature.com/articles/d41586-023-03817-6
An inspiring talk by @khinsen at The First Workshop on Reproducible Software Environments for Research and High-Performance Computing
"We shape our tools and then our tools shape us"
"Convivial tools empower their users. Radical monopolies make users dependent on a technological elite"
"We should come back to the goal of having scientific computing environments that normal scientists can manipulate without the help of expert programmers"
🎉 Just got back from the International PHP Conference in Munich!
An amazing experience, I even gave a talk (see previous toot)!
🎤 Noted some recurring themes:
1️⃣ Reproducibility is often overlooked or ignored, sadly 🤔
2️⃣ The benefits of functional programming in PHP aren't widely understood 🔄
3️⃣ Docker ≠ Full reproducibility 🐳
Stay tuned! I'm planning to write some articles to delve deeper into these topics.
What better time than #OpenAccessWeek for Computo's first toot?
Hello, fediverse! Computo is a journal of the French Statistical Society (but don't worry, everything is in English), and our goal is to publish computational/algorithmic contributions in statistics and machine learning in a reproducible, open access way. Our papers are submitted as notebooks, and you can read them as html or pdf files.
If you're interested in reproducibility and have nothing to watch while eating your lunch today, then why not watch this video of mine https://m.youtube.com/watch?v=zs6LtT0PavM
hot take: #cs academics should be expected not only to publish the full source of the programs that demonstrate their research (I can't believe this still needs to said), but they should also be expected to offer #ReproducibleBuilds of the environments so that the results may be reproduced at a later point in time. "it works on my machine" is no longer a legitimate problem and "I'm self-conscious about the code because its messy" both holds you to a lower standard than undergraduate CS students are held to for their assignments, while holding the reader responsible for reconstructing your research from some small set of source snippets, which amounts to imposing trust in the authority of university discourse, a kind of metaphysics.
taking a few weeks to learn how to use #guix for #ReproducibleResearch should not be too much to ask, knowledge production is mere discourse production when the results cannot be scrutinized beyond a peer review process without a good deal of effort, which we all know that few people have the time for. if you can't be bothered to get it working, then why should your readers be expected to assume that it works?
And while I welcome scientific CS papers, I am more worried that we take Big Pharma's word for their most profitable drugs to also be the most safe and effective ones available.
If you are a research psychologist please help promote awareness of my open-access editorial, A Plea to Psychology Professional Societies that Publish Journals: Assess Computational Reproducibility. My fantasy is to inspire a groundswell of pressure from the membership of, say, the Psychonomic Society. https://open.lnu.se/index.php/metapsychology/article/view/4020
If open data and code support reproducibility (and they do), then so do standardized data and code 𝘤𝘪𝘵𝘢𝘵𝘪𝘰𝘯𝘴. But what are the best ways to implement them? https://www.nature.com/articles/s41597-023-02491-7