1

I think it is important for a research paper to include raw data and code for scientific replicability, verifiability, and falsifiability. However, recently, most of the research paper I read does not give raw data or source codes.

Those papers are from: European Journal of Operation Research; IEEE; International Conference on Cloud Computing and Big Data; MDPI

Some of them are highly reputable outlets. Is this common that an applied CNN and DRL research does not disclose the code and data they used? Are there any journal with a data/code policy on that publishes applied research?

High GPA
  • 163
  • 6
  • 1
    You're asking here 2 distinct questions, both of them are barely on-topic here. I would remove the second one from this post "Are there any journal with a data/code policy on that publishes applied research?", as it's not answered by the answer below that you accepted, and it's more off-topic, and, nevertheless, could be asked on another post. Having said that, your question is a (partial) duplicate of [this](https://ai.stackexchange.com/q/21159/2444). – nbro Jul 10 '22 at 08:10
  • @nbro Hi nbro, thanks for pointing out the related questions! I will write another post. – High GPA Jul 10 '22 at 09:57

1 Answers1

1

Many papers provide links to code, typically on GitHub.com. I have often found that even if the authors don't give a link, I can search the authors' names on GitHub and find a repository for the paper. My wild guess is that about 50% of papers provide reproducible code.

Papers With Code is a good website for finding papers with code.

Brian O'Donnell
  • 1,853
  • 6
  • 20
  • 2
    When you trust Papers With Code's [section](https://paperswithcode.com/trends) on publishing trends among frameworks code availability only 25-30% come with code of all ML related topics. I guess that it might by roughly the same for only deep learning (sadly you can't filter that chart). Accounting the fact that they include community implementations biases the number slightly higher than they should be. – OuttaSpaceTime Jul 08 '22 at 13:36
  • Wow! wonderful link! Truly helpful. Love this answer so much – High GPA Jul 09 '22 at 08:08
  • 1
    I also think that saying that 50% of the papers come with "reproducible code" is probably wrong. There may be some implementations, not by any of the authors, as you mention, on Github or other code hosting services, but those do not necessarily _reproduce_ the experiments or even the exact same models. In terms of really reproducible code, that percentage may be 5%. – nbro Jul 10 '22 at 08:14