challengeutils
challengeutils is a set of tools and commands that provides an interface for managing crowd-sourced challenges administered on Synapse, including but not limited to, DREAM Challenges. Its main purpose is to ease the process of creating, monitoring, and ending a challenge, as well as provide useful functions for post-competition analysis.
challengeutils is written in Python and uses the synapseclient library to pull information from Synapse (account required).
Installation
To install from pypi:
$ pip install challengeutils
To install from the source:
$ python setup.py install
Attention
synapseclient < 2.3.0
is no longer supported, so some features may
break if you are not on the latest client.
Contributing
Interested in contributing? Awesome! We follow the typical GitHub workflow of forking a repo, creating a branch, and opening pull requests. For more information on how you can add or propose a change, visit our contributing guide.
- Participant Built-in Commands
- Admin Built-in Commands
- Create a challenge
- Mirror wiki changes
- List evaluations
- Set an evaluation quota
- Update an evaluation ACL
- Query an evaluation
- Download a submission
- Annotate a submission
- Update a submission status
- Stop a Docker submission
- Update an entity ACL
- Validate a Project Submission
- Archive a Project Submission
- Delete a submission
- Validate a Docker Submission
- Pull a Synapse Project’s Wiki
- Push a Synapse Project’s Wiki
- Scan an evaluation queue for cheating
- Source