challengeutils

challengeutils is a set of tools and commands that provides an interface for managing crowd-sourced challenges administered on Synapse, including but not limited to, DREAM Challenges. Its main purpose is to ease the process of creating, monitoring, and ending a challenge, as well as provide useful functions for post-competition analysis.

challengeutils is written in Python and uses the synapseclient library to pull information from Synapse (account required).

Installation

To install from pypi:

$ pip install challengeutils

To install from the source:

$ python setup.py install

Attention

synapseclient < 2.3.0 is no longer supported, so some features may break if you are not on the latest client.

Contributing

Interested in contributing? Awesome! We follow the typical GitHub workflow of forking a repo, creating a branch, and opening pull requests. For more information on how you can add or propose a change, visit our contributing guide.

Python Client

• Challenge Wiki
  • Create
  • Updating
  • Synapse Wiki Tools
• Submissions
• Discussion Service
• Annotations
• Permissions
• Teams
• Utils
• Evaluation Queues
• Docker Tools

Command-Line Client

• Participant Built-in Commands
  • List challenges
• Admin Built-in Commands
  • Create a challenge
  • Mirror wiki changes
  • List evaluations
  • Set an evaluation quota
  • Update an evaluation ACL
  • Query an evaluation
  • Download a submission
  • Annotate a submission
  • Update a submission status
  • Stop a Docker submission
  • Update an entity ACL
  • Validate a Project Submission
  • Archive a Project Submission
  • Delete a submission
  • Validate a Docker Submission
  • Pull a Synapse Project’s Wiki
  • Push a Synapse Project’s Wiki
  • Scan an evaluation queue for cheating
• Source
  • Participant
  • Admin

Demos & Examples

• General Challenge Workflow

About

• Changelog
• Authors