[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW]: Matching: A Python library for solving matching games #2169

Closed
38 tasks done
whedon opened this issue Mar 23, 2020 · 57 comments
Closed
38 tasks done

[REVIEW]: Matching: A Python library for solving matching games #2169

whedon opened this issue Mar 23, 2020 · 57 comments
Assignees
Labels
accepted published Papers published in JOSS recommend-accept Papers recommended for acceptance in JOSS. review

Comments

@whedon
Copy link
whedon commented Mar 23, 2020

Submitting author: @daffidwilde (Henry Wilde)
Repository: https://github.com/daffidwilde/matching
Version: 1.3.0
Editor: @VivianePons
Reviewer: @igarizio, @mbdemoraes
Archive: 10.5281/zenodo.3755304

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/fd6f8c837c2669d1ba19e8b6ce4b096c"><img src="https://joss.theoj.org/papers/fd6f8c837c2669d1ba19e8b6ce4b096c/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/fd6f8c837c2669d1ba19e8b6ce4b096c/status.svg)](https://joss.theoj.org/papers/fd6f8c837c2669d1ba19e8b6ce4b096c)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@igarizio & @mbdemoraes, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @VivianePons know.

Please try and complete your review in the next two weeks

Review checklist for @igarizio

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@daffidwilde) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @mbdemoraes

Conflict of interest

  • I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Contribution and authorship: Has the submitting author (@daffidwilde) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • State of the field: Do the authors describe how this software compares to other commonly-used packages?
  • Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
  • References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
@whedon
Copy link
Author
whedon commented Mar 23, 2020

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @igarizio, @mbdemoraes it looks like you're currently assigned to review this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

@whedon
Copy link
Author
whedon commented Mar 23, 2020
Reference check summary:

OK DOIs

- 10.1016/j.jda.2006.03.006 is OK
- 10.1257/aer.p20171112 is OK
- 10.1109/MSP.2016.2598848 is OK
- 10.1007/s10479-017-2710-1 is OK
- 10.1016/0196-6774(85)90033-1 is OK
- 10.2307/2312726 is OK
- 10.12688/f1000research.11407.1 is OK
- 10.5281/zenodo.3691470 is OK
- 10.1006/game.1995.1006 is OK
- 10.1086/261272 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@whedon
Copy link
Author
whedon commented Mar 23, 2020

@VivianePons
Copy link

I have started the review as we have now 2 reviewers. But please note that with the special conditions these days, many of us are in complicated situations. There is no pressure to conduct the reviews right now, just do it if / when it's convenient. I will keep following and answer when needed.

@igarizio
Copy link

👋 @mbdemoraes I think you clicked my checkboxes. I transferred them to your review 😁.

@mbdemoraes
Copy link

@igarizio I'm sorry! You are right, my mistake. Thank you for letting me know. 😅

@mbdemoraes
Copy link

The authors propose a python library to solve four "matching games" problems. I could locate the repository, install and run the examples from both README and documentation. The paper is well-written and introduces the field clearly and objectively. Also, the "Discussion" section in the documentation allows the reader to understand the subject in greater depth. Good job!

Some minor aspects that need revision on the documentation:

  • There is a "How to" page that is empty;
  • Under "Indice and Tables" on the home page, there is a "Search page" that is also empty;

Also, I have a few suggestions that I believe will improve this work:

  • Remove the "code figure" from the paper. Although JOSS is a software-oriented journal, the paper is the place where you introduce motivations, theoretical and scientific aspects of your work. Software functionality should be addressed only on the documentation;

  • Include on the documentation that for people with two python installations (versions 2 and 3+), they may have to use the command "python3" to install the package and run the examples, since in that case, the "python" command is the default for python2. It would be a command like this: "python3 -m pip install matching" and to run an example "python3 example.py". This is a common problem when you do not have a scientific python distribution (such as anaconda) installed. It may be obvious for experienced users but others may experience some difficulties;

  • In the paper, the first paragraph of the "Statement of Need" section, the authors write that "Matching games have applications in a number of fields including social and market economics, wireless communication and education infrastructure". Although a practical application in education infrastructure is easily understandable given the examples itself (for instance, the student-allocation problem), applications in market economics and wireless communication are not so easily perceptible. If the authors can provide some specific applications under these fields, more researchers can understand the applicability of the library.

  • On the "Get in contact!" section of README, the authors use "PRs" as an acronym for "Pull requests". I know it is a well-known abbreviation for GitHub users, but again, some readers may not understand. I suggest using the full text "Pull requests" to improve readability.

Thank you for the opportunity to review this work! Let me know when the revision has been made.

@igarizio
Copy link
igarizio commented Mar 29, 2020

👋 Hi @daffidwilde. Thank you for submitting your paper!

The paper is well written and the code is clear, easy to use and properly documented (good job!). So far, I have only found some small issues:

  • As @mbdemoraes mentioned, the “How to” page on the documentation is empty.
  • In the setup.py file, I think you may be missing the hypothesis package (in “tests_require”). (The package is correctly listed everywhere else.) After installing it all tests pass. I should also mention that the coverage is perfect!
  • I get a ValueError when running a StableMarriage game twice. I opened an issue with more details [ValueError when re-running a StableMarriage game daffidwilde/matching#80].

Opinion:

  • While the “code figure” on the paper may look a little bulky, I do not think you need to remove it, as it highlights how easy it is to use your package (maybe you can remove the “>“ (?)).
    @mbdemoraes It is true that JOSS says not to include “software documentation such as API functionality” (link), but I think in this case, it is trying to show a strength of the package, not documenting it. What do you think? 👀
  • On game.py, you mention that the Game class is just a “base class and is not intended for use other than inheritance”. Maybe you can use a metaclass to highlight that (so users get a nice warning on their IDE when subclassing it). This is just a suggestion, your code in completely fine without it.

@mbdemoraes
Copy link

@igarizio Thank you! I suggested that because I thought "code in the paper" was a restriction. But if it's not, you are certainly right, the image sure adds to the content. 😄

@igarizio
Copy link

@mbdemoraes Perfect! As a reference, JOSS even has an example on how to add python code here, and also there are other accepted papers with code in them (for example this one).

@daffidwilde
Copy link

Hello everyone,

Thank you so much for getting back with your reviews so swiftly. I've opened a number of PRs on the matching repository that address most of the points you've raised and I'd appreciate your feedback.

I will be adding to these tomorrow but I think things are looking pretty good thus far.

Thanks 👍

@daffidwilde
Copy link
daffidwilde commented Mar 31, 2020

One thing I haven't yet changed that I'd like to discuss more is this point raised by @mbdemoraes:

Include on the documentation that for people with two python installations (versions 2 and 3+), they may have to use the command "python3" to install the package and run the examples, since in that case, the "python" command is the default for python2. It would be a command like this: "python3 -m pip install matching" and to run an example "python3 example.py". This is a common problem when you do not have a scientific python distribution (such as anaconda) installed. It may be obvious for experienced users but others may experience some difficulties;

When writing the documenation, we decided that since Python 2 is not supported in Matching (by the use of f-strings, for instance) that this was not necessary -- although I see why that could become an issue as you've highlighted. Having said that, since Python 2 has been officially sunsetted as of January 2020, I would prefer to avoid distinguishing between "Python" and "Python 3".

Perhaps adding this requirement to the setup.py and making it clearer (via a warning directive) in the docs would be sufficient? We can discuss this more on this PR if you'd like so as not to clutter this issue.

@mbdemoraes
Copy link

@daffidwilde I understand! Adding this requirement to the setup.py and/or making it clearer in the documentation would sure be sufficient. 👍

@daffidwilde
Copy link

Fantastic! I'll get on that now.

@daffidwilde
Copy link

👋 Hello all!

I've now completed making the changes requested here in Matching. The changes have been split amongst seven pull requests.

It'd be great if you could have a look at these changes whenever you get the chance.

Thanks again for all the suggestions and discussion 😄

@igarizio
Copy link
igarizio commented Apr 5, 2020

@daffidwilde All the pull requests seem good to me. Thank you for making the suggested changes!
Let me know when you merge them.

@daffidwilde
Copy link

@VivianePons I've made a new release on GitHub and the Zenodo archive is here

@drvinceknight
Copy link

DOI is 10.5281/zenodo.3755304
Version is v1.3.0

@VivianePons
Copy link

Hi @daffidwilde : I need the metadata to exactly match the paper (title and author) thanks!

@daffidwilde
Copy link

Oh yes, of course. I've sorted that now @VivianePons 👍

@VivianePons
Copy link

@whedon set 10.5281/zenodo.3755304 as doi

@whedon
Copy link
Author
whedon commented Apr 17, 2020

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

@VivianePons
Copy link

@whedon set 1.3.0 as version

@whedon
Copy link
Author
whedon commented Apr 17, 2020

OK. 1.3.0 is the version.

@VivianePons
Copy link

@whedon set 10.5281/zenodo.3755304 as archive

@whedon
Copy link
Author
whedon commented Apr 17, 2020

OK. 10.5281/zenodo.3755304 is the archive.

@VivianePons
Copy link

Congratulations @daffidwilde for having your paper accepted at JOSS! thanks again to the reviewers @mbdemoraes and @igarizio for their great work

@VivianePons
Copy link

@whedon accept

@whedon whedon added the recommend-accept Papers recommended for acceptance in JOSS. label Apr 17, 2020
@whedon
Copy link
Author
whedon commented Apr 17, 2020
Attempting dry run of processing paper acceptance...

@whedon
Copy link
Author
whedon commented Apr 17, 2020
Reference check summary:

OK DOIs

- 10.1016/j.jda.2006.03.006 is OK
- 10.1257/aer.p20171112 is OK
- 10.1109/MSP.2016.2598848 is OK
- 10.1007/s10479-017-2710-1 is OK
- 10.1016/0196-6774(85)90033-1 is OK
- 10.2307/2312726 is OK
- 10.12688/f1000research.11407.1 is OK
- 10.5281/zenodo.3751325 is OK
- 10.1086/261272 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@whedon
Copy link
Author
whedon commented Apr 17, 2020

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1424

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1424, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

@arfon
Copy link
Member
arfon commented Apr 17, 2020

@whedon accept deposit=true

@whedon whedon added accepted published Papers published in JOSS labels Apr 17, 2020
@whedon
Copy link
Author
whedon commented Apr 17, 2020
Doing it live! Attempting automated processing of paper acceptance...

@whedon
Copy link
Author
whedon commented Apr 17, 2020

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

@whedon
Copy link
Author
whedon commented Apr 17, 2020

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02169 joss-papers#1425
  2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02169
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

@arfon
Copy link
Member
arfon commented Apr 17, 2020

@igarizio, @mbdemoraes - many thanks for your reviews here and to @VivianePons for editing this submission ✨

@daffidwilde - your paper is now accepted into JOSS ⚡🚀💥

@arfon arfon closed this as completed Apr 17, 2020
@whedon
Copy link
Author
whedon commented Apr 17, 2020

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02169/status.svg)](https://doi.org/10.21105/joss.02169)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02169">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02169/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02169/status.svg
   :target: https://doi.org/10.21105/joss.02169

This is how it will look in your documentation:

DOI

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

drvinceknight added a commit to drvinceknight/matching that referenced this issue Apr 17, 2020
drvinceknight added a commit to drvinceknight/matching that referenced this issue Apr 17, 2020
drvinceknight added a commit to drvinceknight/matching that referenced this issue Apr 17, 2020
daffidwilde pushed a commit to daffidwilde/matching that referenced this issue Apr 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted published Papers published in JOSS recommend-accept Papers recommended for acceptance in JOSS. review
Projects
None yet
Development

No branches or pull requests

7 participants