Papers with Code today announced that preprint paper archive arXiv will now allow researchers to submit code alongside research papers, giving computer scientists an easy way to analyze, scrutinize, or reproduce claims of state-of-the-art AI or novel advances in what’s possible.
An assessment of the AI industry released a week ago found that only 15% of papers submitted by researchers today publish their code.
Maintained by Cornell University, arXiv hosts manuscripts from fields like biology, mathematics, and physics, and it has become one of the most popular places online for artificial intelligence researchers to publicly share their work. Preprint repositories give researchers a way to share their work immediately, before undergoing what can be a long peer review process as practiced by reputable scholarly journals. Code shared on arXiv will be submitted through Papers with Code and can be found in a Code tab for each paper.
“Having code on arXiv makes it much easier for researchers and practitioners to build on the latest machine learning research,” Papers with Code cocreator Robert Stojnic said a blog post today. “We also hope this change has ripple effects on broader computational science beyond machine learning. Science is cumulative. Open science, including making available key artefacts such as code, helps to accelerate progress by making research easier to build upon.”
Started in 2018, Papers with Code focuses on encouraging reproducibility of AI model results and, as the name states, submitting research with code. The Papers with Code website shares nearly 2,000 papers and code from across major fields in AI like natural language processing, computer vision, adversarial machine learning, and robotics. Papers with Code was initially founded in part by members of Facebook AI Research. Last year, Facebook and Papers with Code launched PyTorch Hub to encourage reproducibility.
In the past year or so, sharing code along with a research paper manuscript has become standard at major AI research conferences. At ICML 2019, nearly 70% of authors submitted code with their papers by the start of the conference. ICML organizers found that 90% of researchers who submitted code came from academia, and about 27% included an author from industry. Conversely, nearly 84% of authors overall came from industry and about 27% from academia. People developing software or AI inside companies may be more likely to view secrecy as important to protecting intellectual property or financial interests.
NeurIPS began to experiment with a code submission policy in 2019 and put an official code submission policy into effect this year. In other news about evolving standards for AI researchers, earlier this year NeurIPS began to require all paper submissions to include social impact statements and declare any potential financial conflicts of interest.
.jpg "School of Rock, One Beat CPR Share Their Front Line Action Plans for Adaptation and the Future")
