Making your code publicly available: Does it have to undergo review?

For computer science conferences recently ACM introduced Artifact Review and Badging which incorporates code review in the usual publication review process. As far as I know currently it is not mandatory to submit your code for review but it is recommended and their future direction is to make code review mandatory.

From the ACM website:

... there remain many circumstances in which such enhanced review will be either infeasible or not possible. As a result, such review processes are encouraged, but remain completely optional for ACM journals and conferences, and when they are made available, it is recommended that participation by authors also be made optional.

If the venue you are interested in publishing at has the option of code review I would highly recommend it. Having your code reviewed by the conference reviewers will give your work more credibility if it passes the review process as your experiments are reproducible. Also the code review will potentially prevent any missed logical errors if any in your code, its always a good idea to have someone revise your work.

From the ACM website:

A variety of recent studies, primarily in the biomedical field, have revealed that an uncomfortably large number of research results found in the literature fail this test, because of sloppy experimental methods, flawed statistical analyses, or in rare cases, fraud.


In response to your addendum side note: the best you can do is posting your code to your website or a public code repository such as GitHub which gives you the option of choosing from several copyright licenses you can publicize your code under, the list includes but is not limited to (GNU v3.0, MIT, BSD). These licenses will not prevent someone malicious from stealing your code or work and claiming that it is theirs, but it will give you the bases to claim that someone tried to steal your work. Also a common practice I have seen is adding a summary or a paragraph in each file of your source code saying that the following code is protected by xyz license.


This is in part a field-dependent issue, but if you’re planning to release it anyways, why not make it available for inspection during the review process? There’s no particularly valid argument against releasing the code at that stage, especially if it’s a significant part of your new reported work.


My view is that all code should be made available at submission. There are no arguments against this that wouldn't also apply to the main manuscript as well.

So should you make code available for review at submission time? Yes. Will reviewers review the code you submit? Depends, but often they won't. Will many places require this? No.